Tension - Cost of not making Data-based Decisions

Part
01
of one
Part
01

Tension - Cost of not making Data-based Decisions

While a list of the most significant impacts is not readily available in the public domain, information pulled from various sources suggests that businesses that rely on gut feel - or intuition-based decision-making instead of data-driven decision-making - have laggard overall performance, lower margins, reduced competitiveness, and higher chances of making poor decisions. As illustrated below, publicly available statistics suggest that these are the biggest consequences of making decisions based on intuition rather than data.

METHODOLOGY

We began our research with a press search, taking into account that the geographic focus should be the United States and looking for articles or reports covering the impacts, effects, or consequences of gut- or intuition-based decision-making. We also researched the benefits, advantages, or importance of data-driven decision-making, as the two types of decision-making can be contrasted with each other. This initial step led us to a number of relevant statistics, but the statistics are not specific to the United States. The scope of these statistics was either global or undefined. Statistics in articles published by The BI Survey, People HR, Towards Data Science, ZDNet, SmartSheet, iTransition, and Forbes were among the quantitative information we were able to find.

We wanted to find more authoritative sources, so for our second approach, we scoured the websites of the most reputable consulting firms, including Mckinsey, The Boston Consulting Group, Bain, Deloitte, PwC, Booz Allen Hamilton, Ernst & Young, Accenture, KPMG, Oliver Wyman, A.T. Kearney, and Gartner. This step turned up only a few additional sources. We found that Gartner has released a research document detailing how chief data officers (CDOs) are driving business impact, but unfortunately, this document is paywalled. It has also published an article about how bad financial decisions impact businesses. Again, the statistics in this article were based on a global study.

Lastly, we checked if academic researchers have covered the topic. We did this by checking databases of academic literature. This last step led us to a number of articles about Nobel Prize winner Daniel Kahneman, who has worked on the subject of decision-making. Articles published by The Washington Post and the CFA Institute were among the sources we found. We also came across a paywalled research report from the Massachusetts Institute of Technology.

To find out which statistics are the most significant, we listed all the data we were able to gather from these sources. We then classified the statistics into just four major categories: laggard overall performance, lower margins, reduced competitiveness, and higher chances of making poor decisions. As we were unable to find statistics specific to the United States, we assumed that global statistics are also indicative of the behavior of companies in the United States.

LAGGARD OVERALL PERFORMANCE

Data-driven decision-making appears to be associated with best-in-class performance, so businesses that do not make data-driven decisions will likely fall behind their data-driven counterparts in terms of performance. Most best-in-class companies use data-driven decision-making, while most laggard companies use gut-based decision-making. According to The BI Survey, 60% of best-in-class companies base their decisions on data. In contrast, 70% of laggard companies base their decisions on gut feel. Moreover, based on a survey conducted by the Economist Intelligence Unit, 39% of companies whose use of data was behind peers are either somewhat behind or substantially behind peers in terms of financial performance.

LOWER MARGINS

Businesses that do not make strategic use of consumer behavior insights perform worse than their peers in terms of gross margins and sales growth margins. Based on an article published by Towards Data Science, they have 85% lower sales growth margins and at least 25% lower gross margins. Compared with data-driven companies, companies that are not data-driven are likely to be less profitable and productive. Based on a study from the Massachusetss Institute of Technology, which analyzed 179 publicly-traded companies, companies relying more on gut feel or intuition are 6% less profitable and 4% less productive. Also, according to a survey conducted by the Economist, 83% of companies indicate that their use of data had resulted in the increased profitability of their existing offerings.

Poorly-informed operational decisions made by mid-level managers are causing companies to lose profits. According to Gartner, the frequent occurrence of these poorly-informed decisions is costing companies over 3% of profits. Also, businesses that are not data-driven are likely to have lower EBITDAs. Based on an article published by Forbes, data-driven companies are observing over 20% to 30% increases in EBITDA, and these increases can be attributed to efficiencies and more detailed financial data.

REDUCED COMPETITIVENESS

Businesses that do not make data-driven decisions are likely to lose out on competitiveness. Based on an article published by custom software developer iTransition, 59% of organizations believe that "business analytics creates a competitive advantage." Also, over 80% of analytical innovators report that they engage customers, use customer feedback, enhance customer intelligence, and satisfy customers. The article published by Towards Data Science also shows that data-driven businesses have 23 times higher odds of acquiring customers, six times higher odds of retaining these customers, and 19 times higher odds of becoming profitable. This means that businesses that rely mostly on intuition have lower chances of acquiring and retaining customers and of becoming profitable, thus these businesses are less competitive.

HIGHER CHANCES OF MAKING POOR DECISIONS

Businesses relying on gut feel or intuition have higher chances of making wrong decisions. Nobel Prize winner Daniel Kahneman says: "Algorithms beat individuals about half the time. And they match individuals about half the time." Human judgment is not as reliable as algorithms, as judgment is subjective and inconsistent across people. The name Daniel Kahneman surfaced several times in our searches, as he is a Nobel Prize-winning psychologist and a well-known proponent of delaying intuition or gut feel until an informed choice can be made. Kahneman, along with two other researchers, recently penned a study detailing how to approach strategic decision-making. Unfortunately, the full text of this study is not publicly accessible, and articles and reports about his work do not contain statistics or quantitative information on the impact of gut-based decision-making.

Compared to those that make data-driven decisions, human resource (HR) teams that rely on gut feel or intuition are four times less likely to be respected by leaders and colleagues, according to an article published by People Apps, an HR software provider.


Sources
Sources

Quotes
  • "We found that “best-in-class” companies base their decisions more on information (60%) while “laggard” companies base a worryingly high 70% of their decisions on gut feel."
Quotes
  • "Research has shown that HR teams that rely on data-based decision making are, ‘4 times more likely to be respected by colleagues and leaders’."
Quotes
  • "Research from Mckinsey suggests that organizations that are using their consumer behavior insights strategically are outperforming their peers by 85 percent in sales growth margins and by more than 25 percent in gross margins. "
  • "Recent research also shows that data-centered organizations are twenty three times more likely to acquire customers, six times as likely to retain those customers, and nineteen times as likely to be profitable as a result."
Quotes
  • "Nobel Prize-winning psychologist Daniel Kahneman -- along with Amos Tversky, his longtime collaborator who died in 1996 -- changed the way the world thinks about economics, upending the notion that human beings are rational decision-makers. Along the way, his discipline-crossing influence has altered the way physicians make medical decisions, political scientists think through foreign policy and investors evaluate risk on Wall Street. "
  • "Their suggested approach, labeled with the wonky name “Mediating Assessments Protocol,” or MAP, has a simple goal: To put off gut-based decision-making until a choice can be informed by a number of separate factors."
  • "The structured process calls for analyzing a decision based on six to seven previously chosen attributes, discussing each of them separately and assigning them a relative percentile score, and finally, using those scores to make a holistic judgment."
Quotes
  • "Given how unreliable human judgment is, all evaluations are susceptible to errors. These errors can stem from known cognitive biases — or they can be random errors, sometimes called “noise.” "
Quotes
  • "Kahneman described a study of underwriters at a well-run insurance company. While not an exact science, underwriting is a domain with learnable rules where expertise can be developed. The underwriters all read the same file and determined a premium."
  • "The underwriters all read the same file and determined a premium. That there would be divergence in the premium set by each was understood. The question was how large a divergence."
  • "“What percentage would you expect?” Kahneman asked. “The number that comes to mind most often is 10%. It’s fairly high and a conservative judgment.”"
  • "Yet when the average was computed, there was 56% divergence."
  • "Algorithms beat individuals about half the time. And they match individuals about half the time."
  • "There are very few examples of people outperforming algorithms in making predictive judgments. So when there’s the possibility of using an algorithm, people should use it. "
Quotes
  • "Chronic occurrence of poor operational decisions by midlevel managers is eroding margins and costing firms upward of 3 percent of profits, according to Gartner, Inc."
  • "Twenty-two percent don’t consider a single financial implication when making such a decision. These factors translate to a company with $5 billion in revenue sacrificing upward of 3 percent of earnings through poor decision making across the thousands of material business decisions it will face each year."
Quotes
  • "In fact, a study from MIT has proven the benefits of being data-driven. Prof. Erik Brynjolfsson and his colleagues studied 179 large publicly-traded companies and concluded that those companies are 5% more productive and profitable than their competitors."
Quotes
  • "Andrew McAfee and Erik Brynjolfsson, professors at the MIT Sloan School of Management, conducted a survey and found that the companies that were mostly data driven had six percent higher profits and four percent higher productivity than average."
  • "Eighty-three percent of respondents to a survey by The Economist said that using data had made their existing offerings more profitable"
Quotes
  • "Between 2016 and 2017, the share of organizations reporting that analytics creates a competitive advantage rose 2 percentage points."
Quotes
  • "Data-driven organizations are seeing upwards of 20% to 30% improvements in EBITDA due to unlocked efficiencies and more granular financial insight. "