Don’t take the easy out. Making informed decisions and tackling today’s real problems is possible if you know how to use data properly.
“No one knows.”
“Someone should figure that out.”
“If we had access to that information, then we could do this. Too bad it’s impossible.”
“When we know more, we’ll make a change. For now, there’s nothing we can do.”
We have a strange relationship with data. All around us, “Top 10” lists and flashy statistics compete to grab our attention. We know our personal data is valuable and should be protected. At the same time, data has a certain mystique. We think people either are or aren’t “data people” and often take quantitative findings as fact without questioning how the data was collected, interpreted, or presented. Our mindset about data is simultaneously trusting and distrusting.
How we think about data affects how we use data and the outcomes that follow. In some cases, when someone isn’t a “math person,” they think using data, let alone using it well, is just too difficult. In other cases, even when someone recognizes the importance of data, they encounter a complex challenge and think, “Oh, there’s just no data for that,” and, therefore, it isn’t possible to reach a good answer. The pursuit of missing knowledge is abandoned with an assumption that “no one knows.”
In many ways, we are living in a time when we have access to too much data. It’s overwhelming. It’s difficult to know what is valid, trustworthy, and useful—and what isn’t. This environment of abundant and questionable data can make using data seem intimidating or risky.
That’s a shame. As new technologies emerge that provide even greater access to knowledge, expertise, and data, the ability to recognize how to use data that is trusted and available to us to formulate better questions or test new hypotheses will serve us well. The alternative approaches of making uninformed decisions or postponing action will not serve us as we continue to navigate complex workplace and social challenges.
Today, identifying the skills that Canadians will need as we move into an increasingly uncertain future has become an enormous priority. These efforts often focus on training for specific technical skills. However, instilling confidence, problem-solving, and good judgement are critical to knowing what to do when there is simply no way to know “for sure” what do to next.
In this white paper, we explain what data proxies are in research and decision-making—for business success and shaping the Future of Work—and how they can be used to fill gaps in data and knowledge. Then, we show this work in action through two case studies about overcoming the “no one knows” trap. The first case study concerns building an evidence base that can help shape the future of Canada’s career development sector. The second case study concerns remembering lessons learned during crisis to shape the future of Canada’s oldest coworking community. Lastly, we provide a tool that you can use to avoid falling into the “no one knows” trap.
When perfect data doesn’t exist, turn to proxies
Challenge Factory frequently partners with organizations that have fallen into the “no one knows” trap when it comes to questions about the Future of Work. From what the future of remote and hybrid work will be to how the housing market will impact future labour supply, there’s no shortage of issues and questions that feel difficult or impossible to answer.
These types of questions usually build on each other. Like a ball of twine, each unknown becomes another length of messy, knotted string tangled together. Pulling on the end of one thread only causes the knots to get tighter. While it may be easy to see that particular thread and pull on it, it’s not helpful in addressing the bigger concern of how to unravel the knots or, more ambitiously, the entire ball.
Using a variety of tools and techniques, our task is to loosen the right knots at the right time so that the entire network of entanglements loosens and can once again become useable material. To accomplish this goal, we use data, proxies, and a research-to-practice model. Let’s begin by exploring the importance of knowing what type of data you need, followed by how you can begin to move forward when it feels like that data doesn’t exist.
Data for making business decisions versus data for shaping the Future of Work
There’s a big difference between the type of data someone needs to shape the Future of Work and the type of data they need to predict future business performance. Making effective business decisions is about placing bets on what will happen next, whereas shaping the Future of Work is about taking action to create workforces, workplaces, and labour markets that are more equitable and sustainable for all.
On the one hand, when predicting the future, you need data about what’s easily defined and already known. Questions focus on what will be different, by when, and who it will impact. When we predict something, we make bold statements that can later be proven true or false. Pundits who predict what will happen at the beginning of each new year engage in this type of activity: Interest rates will rise by a predicted percentage, a specific brand will dominate the market, or certain issues will be the most important to average citizens.
On the other hand, when shaping the Future of Work, you need indicators that tell you when to make a change. The focus is less about predicting a definitive outcome or metric and more about identifying what is desired, how we might get to that desired state, what obstacles are likely to be encountered, and how we will know if we are making progress.
Using proxies is one way to bridge this gap, tackle problems that feel impossible to solve, and challenge the assumption that certain aspects of business, work, or the future are simply unknowable.
What is a data proxy?
In data collection and analysis, a data proxy refers to a substitute or stand-in variable that can be used when it is difficult or impossible to directly measure or access the variable of interest. Proxies are used in research to estimate or represent an unobservable or difficult-to-measure phenomenon. They serve as a practical means to gather data and produce insights when collecting the actual data is challenging or costly.
Challenge Factory often uses proxies to navigate the complex issues and ecosystems in which we work, where new and tailored approaches are valued over standard templates. They allow us to help clients and partners find solutions when they believe “there is no data for that.”
The research results are not the only valuable outcome of using proxies. Learning about the concept of proxies and thinking through their application also helps leaders shift their mindsets about and relationship to data. Proxies allow us to make the intangible concrete by providing a score or metric that can be compared across organizations, or within one organization over time, to demonstrate how attitudes or behaviours might change.
Two examples of data proxies
Net Promoter Score
Customer loyalty is a good example of where accepted proxies are used in place of precise measurement. It is actually impossible to measure exactly how much someone likes a product, brand, or service and what might cause them to substitute one product—even while saying they love it—for another.
In 2003, Bain and Company consultant Fred Reichheld created the Net Promoter Score (NPS) metric to represent customer loyalty. The metric works on a 9-point scale and has only one question: How likely are you to promote or recommend this product, service, or brand to others? A score of eight or nine indicates the respondent is a “net promoter,” loyal and connected to what is being sold. A score of six or lower suggests the respondent is a “net detractor,” likely to undermine what is being sold.
The “Recommend Question,” as it has become known, has served as a proxy for customer loyalty ever since. Consumers experience variations of this approach everywhere, including in market research surveys and when they press a green, yellow, or red button on the way out of a department store to indicate how likely they are to recommend it to others based on their most recent shopping experience.
B Impact Assessment
Another example of proxies are the metrics used to evaluate prospective and certified B Corporations. The B Impact Assessment administered by non-profit organization B Lab asks a series of questions about a company’s practices and outputs across five categories: governance, workers, community, the environment, and customers.
Some questions are empirical at a practical level, such as whether all of the company’s employees are paid a living wage. However, B Lab’s interest in confirming living wages is not purely a salary question. What they are seeking to measure is whether the company is a fair employer, responsive to local market conditions, and focused on the well-being of their staff.
Stating that they pay a living wage requires companies to (a) know what a living wage is as a concept, (b) know what the actual living wage is for each city where their employees reside, and (c) consider the obvious, but sometimes overlooked, connection between compensation and community quality of life. Paying employees a living wage is seen as a minimum standard and a low bar. But the value of this metric is not only in the empirical information it provides, but in the actions it triggers. It serves as a proxy for how committed employers are to building, updating, and maintaining fair social contracts with their employees.
How to overcome the “no one knows” trap: Two case studies
Case study 1: Career transitions – Build a reliable evidence base
In the professional field of career development, we see many examples of well-intended leaders and organizations falling into the “no one knows” trap. Today, a lot of energy, funding, and focus is being placed on how Canadians will transition from the current economy to a fair, green economy that can thrive through AI transformations, ageing workforces, and more disruption. This is a major issue where lack of data and “no one knows” thinking prevents breakthrough action.
If this transition to a new, sustainable economy is to succeed, it is important to first know who influences and assists Canadians with their career transitions. This seems like it should be simple to identify; all we have to do is answer the question: Who provides career guidance and support to Canadians? But the answer can lead to all sorts of follow-up questions: What is the quality of the guidance? How responsive is it to changes in the world of work and labour markets? Who’s left out or lacks access? What’s the right level of funding and staffing for a country of Canada’s size and situation?
The career development sector in Canada, as in many countries, is hidden. If you aren’t actively involved in career development research or practice, you likely aren’t aware that this field is comprised of formal professionals, with standards, methodologies, and innovation that directly address the question: How do we navigate a changing world of work and ensure everyone shifts into the new skills economy?
Due to the sector’s invisibility, many skills and employment-focused programs, funding initiatives, and the need to advance the sector and culture of career development in Canada are treated as if “no one knows” how to move forward. Sometimes, this means little is being done. Other times, actions are being taken, but they are ill-informed or not optimized. The intentions are undeniably good; policymakers, educators, employers, and others want to provide a better path to a prosperous Future of Work for Canadians. But taking a “no one knows” approach leads to slow progress, wasted resources, and uncoordinated efforts.
For a long time, Canada lacked a reliable evidence base about the overall size, scope, and composition of its career development sector. With funding from CERIC and in partnership with the Canadian Career Development Foundation, we set out to map the sector and answer fundamental questions about who is engaged in career development across the country and, more broadly, how the sector can shape its own future.
To build a reliable evidence base about the sector, three research strategies contributed greatly to our capacity to overcome the “no one knows” trap:
1. The use of research personas to distinguish different types of career development work within the sector: This approach helps make visible the nuances, hidden work, and sector members who are doing a variety of unique types of work. The persona serves as a proxy for large categories of professionals, ensuring their work is seen and its impact is valued. This strategy creates the borders of the ecosystem map, making the large and complex challenge of scoping an entire sector more manageable.
2. The use of proxy data to tackle impossible-to-answer questions: Many career conversations and informal mentoring sessions take place between managers and employees. It is impossible to know for certain how many managers actually engage in these types of meetings with their staff across Canada, let alone whether the career guidance they provide is of high quality and based on current labour market trends. However, we can make calculated assumptions based on:
- the general level of awareness among Canadians about career development best practices;
- the number of Canadians in management positions reported by Statistics Canada’s Census of Population;
- the number of organizations that have more than 10 employees;
- the average number of career conversations that managers have with staff in a given year; and,
- a manager’s average span of control within organizations of various sizes.
Using proxy data points that are known, we can work towards a reliable estimation for an answer that, on the surface, seems impossible to know for sure. In this sector mapping work, we did not explore the career development impact of people managers in detail, but future research certainly could do this.
3. The use of top-down and bottom-up data sources: Top-down and bottom-up approaches to data collection differ in their research starting points, participants, processes, and levels of control over the data collection process. We collected data from both government funders at the “top” of the sector and frontline providers of career services at the “bottom” (better understood as the foundation or core) of the sector. This strategy of “meeting in the middle” ensures we engage a wide range of career development interest holders and positions us to identify what data exists (or doesn’t exist) and what data is accessible (or inaccessible) within the scope of the research. By looking at data from both perspectives and focusing on ranges, we can confidently overcome inertia caused by missing data.
This initial work to create an evidence base about how many people provide career development support to Canadians has the potential to begin moving entire systems forward. It is the equivalent of untangling a major, central knot in the ball of twine that prevents innovative solutions from being implemented at scale to solve some of Canada’s trickiest workforce problems. When “no one knows” the extent of the full ecosystem of actors who can help solve a problem, we tend to work on small snags without ever releasing the tension on the main knot.
This career development research will mobilize those within the sector and others who care about careers, employment, and labour market transitions to take action with a sense of knowing. This is what having good data and a reliable evidence base enables.
Case study 2: Community building – Don’t discount what we learn during crisis
The Centre for Social Innovation (CSI) is Canada’s first coworking space and a community for social-purpose organizations. It had the incredible foresight to conduct a survey of its members at the height of the COVID-19 pandemic. The comprehensive survey received hundreds of responses and resulted in a rich dataset that provides insight into what members needed, how they thought, and what they felt in the darkest days of 2021.
CSI turned to Challenge Factory to analyze the more than 700 pages of data that had been gathered. As we examined the data and imagined who it might be useful to, we recognized that simply creating charts and graphs would not suffice. To really have impact, the stories from that unusual and unprecedented moment in time needed to be told. So, we created a narrative story with complex characters who brought the data findings to life in a way a typical report couldn’t. The final product is a compelling story supported with data elements, charts, callouts, and graphs. Readers can empathize and relate to the core characters. The story and its characters filled in gaps in data, based on a deep understanding of the culture and people who call CSI their workplace home. It presents a different way to make sense of data gathered at a very specific moment in the midst of a crisis, where key plot points and learnings can serve as proxies to indicate just what makes CSI different to its members.
A main question that arose during the project was whether a story told about crisis is still valuable or of interest once the crisis has passed. If we are no longer experiencing the days of lockdowns and isolation, do we need a story that reminds us of those times…or do we just move on?
We end up in circumstances where we feel “no one knows” the answer to unprecedented events or crises precisely because we tend not to document or keep key reflections once the event is over and the crisis is resolved. Revolutions follow patterns, and how we respond in one cycle of revolutionary change can be very useful to learn from and use to guide our choices as we experience the next wave of disruption. The story of the CSI community in 2021 can seem out of date with where the community is today. Or it can serve as a reminder of how enduring values and relationships sustained a community in dark times and continues to be CSI’s superpower and secret weapon.
The next time CSI is faced with a new, unprecedented situation, the documented learnings from the pandemic, now in the rear-view mirror, might be just the kind of information needed to navigate into an uncertain future.
Believing “no one knows” no longer serves us
When no one knows what to do, where to look for the answers, or how to tackle a problem, it’s easy to feel like either we can’t do anything about it or, since there’s no right answer, we have licence to do whatever we want. This provides a ready way to avoid committing to solving today’s real problems in business and across society. System-wide impact gets sacrificed for short-term or easy wins.
At Challenge Factory, we fundamentally disagree with the “no one knows” phenomenon and challenge the assumption that certain aspects of business, work, society, or the future are simply unknowable. While specific outcomes or exact answers may not be within our grasp right at this moment, we can watch for patterns, sense shifts and changes, and set conditions that will lead to better information in the future. Only then will we be able to unravel the knotted balls of twine that urgently need unraveling.
What’s your “no one knows” assumption?
Being able to recognize when you may be incorrectly assuming that data, knowledge, and a path forward does not exist is an invaluable business advantage for company leaders. It is also a key to success for everyone setting out to shape the Future of Work. Use this tool to help you move forward when you feel stuck in the “no one knows” trap.