I recently came across this post with inspirational data-related quotes by Emily Stevens. Whilst most quotes listed preach about the potential of data and its growth over the years, I find that most organizations fail to use data effectively. Everyone is talking about the value of data, but most people I work with feel that they are barely scratching the surface of the opportunities it presents.
We are surrounded by data, but starved for insights – Jay Baer
Our goal at Data Crafts is to help people use data to add value to their organizations. During our many years consulting, we found common patterns that prevent companies from utilizing data effectively and efficiently. In this post, I want to share the worse mistakes we saw companies make while using data. Contrary to what many believe, it is seldom down to a technology choice.
Not defining a purpose
Data is like garbage. You’d better know what you are going to do with it before you collect it – Mark Twain
Data can answer questions, provide insights and shed light over what we don’t know. However, there is one thing it won’t do: define a purpose. That one is on us. We need to define what questions we want answered, what processes we want to automate, or what elements we want to optimize.
Many organizations expect a new trendy analytics solution, shiny cloud data warehouse, or slick visualization tool will solve all their data problems. Unfortunately, this isn’t the case. Of course, there are cases where technologies alone unveil valuable insights, but these are infrequent and usually the result of lucky coincidences.
Not defining a purpose makes it impossible to assess whether an initiative was successful or not. Unknown objectives may be achieved, but they will go unnoticed and unvalued if they are not defined in advance. Having clarity on the goals makes the same output more likely to be considered a success.
Imagine you want to use a session recording tool because your conversion rate is low, and you suspect there are UX issues. After implementing the tool and spending some time looking at the session replays, you don’t find any evidence of significant UX problems. In the end, you conclude that a traffic quality problem is causing the low conversion rates. If you made clear you were trying to find out if there was a UX issue, this is a success since you answered the question and ruled out that possibility. However, if you implemented the tool without defining an objective, you might believe the session recording tool didn’t add value since it seems like you haven’t found anything.
When working with data, ensure you know exactly why you are doing things. If possible, share the objectives with the broader team and get their buy-in to be aligned.
There isn’t enough talent
Big data isn’t about bits, it’s about talent.” — Douglas Merrill
Getting value from data involves a lot of work, including designing strategies, selecting, implementing, and maintaining tools, generating and presenting analyses and reports, automating pipelines, and running optimization programs. All these require a ton of effort from highly qualified people. Without them, organizations fail to use data effectively. This common scenario comes in two flavors: insufficient resources or wrong skillset.
Some companies have massive data demand but not enough engineers to fulfill it; the technical team becomes a bottleneck, and frustration spreads amongst the data consumers. Conversely, some companies have great data engineering teams that create amazing data warehouses, but they barely get used because there aren’t enough analysts to use it; this will result in a very expensive team that doesn’t add value, and a small ROI from data.
It is a challenging industry, and talent is scarce and expensive, but the right team, skillset balance, and culture are necessary if you want to use data effectively and efficiently.
Bad communication
Data on its own is useless, we need people to interpret it and then tell stories with it – Denholm Hesse
When communication isn’t good enough, data doesn’t reach the right audience with the right message and becomes useless.
Many factors can cause this: a lack of data literacy, wrong communication channels, organizational siloes, or an immature data culture, to name a few.
The journey from data collection to action is long and complex, and it is vital to understand what is blocking the information flow and amend it if you want to utilize data fruitfully. One way to identify blockers is to frequently speak with people alongside the data chain. They will surely be thrilled to share their successes and frustrations, helping identify improvement opportunities.
Lack of investment
Every company has big data in its future, and every company will eventually be in the data business – Thomas H. Davenport
The infrastructure and the people required to collect, transform and activate data are expensive. And that’s that. Many organizations are unwilling to invest enough money in data, hindering their chances of achieving any success.
Without money, you can’t hire the right team, buy the right tools and get adequate help. Asking a graduate to implement all the free tools without guidance is a recipe for disaster.
That said, data will soon become an essential asset for every company. Without it, companies won’t be able to compete and will eventually go out of business. From an expenditure perspective, data should be considered a utility rather than an optional strategic asset. Nowadays, no company questions if they need an accounting department or electricity in the building. They know these are essential resources they need to stay in business. Similarly, data should be considered a fundamental asset to manage organizations and improve their results, and companies shouldn’t hesitate to invest strongly in it.
Bad implementations
Data! Data Data! I can’t make bricks without clay! – Artur Conan Doyle
This mistake is one of the most prevalent reasons organizations fail to use data. Outdated, wrong, messy, convoluted, undocumented, indecipherable, inconsistent, fragile… The list can go on and on. You can’t work well with bad implementations, period.
How will you know if new features perform better? How can you allocate a budget to channels and campaigns? How do you know if your personalization algorithm is not making grave mistakes? If your implementation isn’t good enough, you make uninformed decisions at best. If the implementation is faulty and lacks credibility, people will question the good results and deny the bad ones.
Bad implementations have a long list of dire consequences: inability to get the required information, longer time to insight, high maintenance costs, team frustration and demoralization, and lousy decision-making. I even saw situations where bad implementations spiraled down and resulted in multiple resignations that had a terrible impact on the team, from which they didn’t recover for several years.
If your implementation is in bad shape, the best you can do is to fix it as soon as possible before moving to more advanced use cases.
Excessive perfectionism
Errors using inadequate data are much less than those using no data at all – Charles Babbage
So many times, I see companies getting obsessed about the discrepancy between 2 or more systems. Their focus shifts from using data to make decisions to understand why the numbers don’t match. They start questioning every report and dashboard and enter a rabbit hole where they spend countless hours without getting anywhere.
Data quality is essential, and making decisions based on inaccurate data is dangerous. However, not all use cases are created equal. Discrepancies and inaccuracies in data have a different impact depending on its use: financial data has to be 100% accurate; the orders and the logistics database should be close to 100% match (maybe there are some lost or damaged shipments); orders registered in web analytics, on the other hand, will show discrepancies with the backend system, and it is normal.
Some factors hinder data capture in some environments: ad blockers, cookie rejection, cross domains, connectivity issues, and old and privacy-friendly browsers, to name the most common. In this context, it is impossible to get 100% accuracy, and it is okay. Embrace that fact, understand the origin of the discrepancies, and define a threshold for acceptable differences under which you can operate without questioning the results.
Unrecognized biases
It is a capital mistake to theorize before one has data – Sherlock Holmes
I had a boss that used to say: “if you torture data long enough, it’ll confess anything.” I believe the quote isn’t his, but it is still very relevant. I often hear things like: “we just need to show that the new funnel outperforms the old one,” or “we will first publish the update, and then we will compare the before and after to measure the improvement.” The problem with the first one is that it assumes that the new funnel will perform better than the old one; the problem with the second one is that before and after comparisons are seldom accurate and should only be used very carefully and when there is no alternative, but never to make up for bad planning.
This excellent article explains some biases we can experience when using data. They are all excellent mechanisms people use to ignore the facts and get data to tell us what we want. Often, people make decisions based on hunches and wills, not on evidence, which defeats the purpose of using data in the first place. When biased, we don’t perceive value in data when it validates our hypotheses since we already “knew” what it would tell us. Even worse, when data refutes our preconceived idea, we use everything we can to invalidate the results and refuse to learn from them.
Failing to identify and correct biases make organizations fail to use data because no value is perceived if it “validates” hypotheses and prevents them from learning and rectifying when it refutes them.
There isn’t a single solution for this, but being open to being wrong, promoting a culture of experimentation, and rewarding learning instead of punishing failures go a long way.
Need help?
At Data Crafts, we have a wealth of experience helping companies make the most of their data. We offer in-depth audits to assess the state of the technology, the processes, and the organization to identify opportunities for improvement and design a roadmap to bring your company from zero to hero at leveraging its data assets.