A business can’t be managed based on data alone

As much as we rely on figures for making business decisions, you shouldn’t sanctify KPIs

Shaul Olmert 18:4724.01.21

TAGS:

Managing a tech company nowadays is more a science than an art. The managerial culture that has taken root in the industry sanctifies data at the expense of intuition, with everything becoming target driven and information-based. When an artist sets their canvas on the easel and starts painting, they don’t need to know too much about how the painting will end up. They can go with the flow, get started and see where the brushstrokes take them. In the worst-case scenario, they throw away the painting and start again. The costs and operative impact of throwing out a sketch are usually negligible so there’s nothing stopping them from carrying out trial and error, there’s no imperative to plan everything in advance and manage the process according to use and consumption figures. A company that employs tens, hundreds, or thousands of employees, which operates in a highly competitive and fluctuating market and manages a high sales volume, meanwhile, doesn’t have the luxury of going with the flow and because of that must adopt a strict and calculated operating style, based on precise data. This management imperative is not limited to tech companies, of course, but in the digital world, where every action that takes place on the product is recorded, it’s possible to closely track every step of interaction, allowing for a high degree of monitoring.

“Anything that isn’t being monitored isn't being managed,” many managers, who are quick to install screens displaying performance charts and graphs at every corner of the office so that employees constantly have their fingers on the pulse of the system, would say. Automatic usage reports concentrate ungodly volumes of figures in a wide range of formats, present them, analyse them and send out alerts when volumes deviate from what’s expected. It's gotten to be so that employees spend most of their time examining and analyzing the information and most of the internal communications and investor and partner relations are carried out based on those figures. In modern tech companies, data is king. The more mature a company gets, the more the data becomes akin to religion.

A manager looking at figures while ignoring his employee. Photo: Shutterstock

A new startup like “Flying Pigs” possesses very little data. The product hasn’t been launched yet, we don’t have any customers or users, we haven't earned a single dollar and we can’t even properly guess precisely how the product will be used when it’s fully developed. Our challenge at this stage is to define exactly which parameters we want to measure in order to base our product and service management on in the future. In other words, we have to build an initial thesis about the use of our future product and define the parameters that properly represent its use and ensure that they are measurable, recorded, fed into the cloud-based monitoring system, so that when the product is eventually operational, we’ll be able to examine the data and determine where we stand and adjust accordingly. At my previous company, Playbuzz, we worked with an outstanding reporting system, one that took me several hours to present to potential investors who I wanted to impress with my data-based managerial abilities. Of course, less is more and with time I learned that not everything can be measured properly and moreover you simply can’t follow and be familiar with too many figures. You must choose a small number of critical parameters and make do, adding or removing some with time. In the professional jargon, these parameters are known as Key Performance Indicators (KPI). A good organization can closely track their main indicators, quickly examine the causes behind any change to them and calibrate the system accordingly. Choosing the correct indicators to track is essential. In a mature organization such as YouTube, there is an ordered system and teams dedicated to the various indicators. They have managers that constantly track how many new videos are being uploaded and others who monitor how many minutes on average a video is viewed for. When these indicators go up or down, the managers must determine what caused the change and take action accordingly. In our case, due to a small number of team members and the fact that our product isn’t mature yet, we grasp on a very small number of indicators so that we can focus on what seems critical and not waste our time chasing data that’s not that important.

What managers tend to forget is that the world can’t actually be reduced to columns in a chart. Reality is chaotic and complex and while relying on figures is necessary in order to lay out some constants and remain on track, data too has its limitations. Firstly, there are important questions concerning the collection of data and its credibility. For example, these days, we are all witness to the data-based management system that the state authorities have employed to receive indications about Covid-19 infection trends. The government decision-makers defined several main indicators, such as the number of people infected per day of testing, the number of critically ill, the number of people that require ventilators and so forth in order to guide their regulations. But we are also aware of the criticism that arises from things like tests being conducted on sections of society that don’t reflect its entirety, or the fact that among the results are some tests that aren’t credible. These are examples of challenges that are characteristic of data-based management: the figures aren’t always accurate, comparisons aren’t always apt (comparing apples to apples, as the saying goes), and occasionally other variables impact the significance that we place on different types of data. A further challenge arises from entrenching management patterns that rely solely on figures. A good organization creates a nearly automatic mechanism for data-based management and it can often take a long time to make that reliable and make necessary changes to the existing patterns. At Playbuzz, for example, we measured the degree of virality that any piece of content made with the platform achieved and defined a piece of content’s value based on the number of people who were exposed to it as a result of it being shared on social media. We assumed that the higher number of hits a piece of content got from being shared on Facebook, the higher its “viral potential” and the better it was. With time we realized that the number of clicks as a result of exposure on Facebook was impacted by many parameters, that often change when a random project manager at Facebook decided to make changes to the algorithm, resulting in the indicator that we chose no longer representing the value we gave to it.

But the primary logical failure when it comes to data-based management is derived from confusing the end with the means. When you are eager to see the indicators you defined climb upwards, suggesting that your business is on an upward trend, it is sometimes easy to “play” with the indicators, whether knowingly or not, so that they show an uptrend even if the value that they represent doesn’t climb with them.

Related Stories:

Another example from my days at Playbuzz takes me back to the days when the main indicator we monitored was the number of exposures to pieces of content produced on our platform that featured on leading websites. All of our partner sites (there were several tens of thousands of them) embedded content created on Playbuzz on some of their pages, and anytime someone would visit a page with our content on it, our counter went up by one. One day we were fretting because the number dropped or didn’t climb in accordance with our expectations and we started to come up with solutions. We suggested training the teams of our leading partner sites in order to make it easier for them to produce and embed content, we suggested to add more options to the product in order to make it easier and faster to create, and a variety of other suggestions aimed at increasing usability that would eventually lead to our indicator tracking exposures returning to climb. But nothing prepared us for a sudden increase of hundreds of percent that took place one day mostly from a small number of partners. When we examined more closely what had happened, it turned out that one of our eager and creative salespeople was able to convince some of our partners that they would benefit from having a Playbuzz feature embedded at the bottom of every one of their pages. As a result, the number of exposures skyrocketed. But the volume of exposures was never the actual goal, only an indicator of the true value which was things like higher levels of interaction with the content produced on Playbuzz platform or content of greater importance that was created on it. Of course, none of that had changed just because we artificially inflated the number of exposures by being featured on the bottom of more pages, and the boost had no real value to either our company or our partners. It was a classic case of confusing the method with the goal: the number of exposures was merely an indicator to attest to a high degree of usability. When it is turned into a goal unto itself, it can be raised, but it won’t reflect the true value, the number of interactions with our product, which didn’t actually budge. We praised the sales manager for his ingenuity, but kept on looking for solutions to generate actual value.

In addition to all that, while it is indeed important to keep your finger on the pulse, to monitor and analyze and understand where your business is heading, sometimes you can’t “see the forest for the data.” Good management is about more than just data-analysis, it's about having a strong emotional connection to your organization, your product, and your clients. A good manager must be good at reading, investigating and analyzing figures, but should also give respect to gut feelings. At the end of the day, the answers aren’t always hidden in the numbers, sometimes they’re hidden in the sentiment. Businesses that serve a large number of clients (in the case of Playbuzz, tens of thousands of partners and more than a billion monthly users) can’t really tell what’s going on without sticking to a few guiding indicators. The challenge is identifying the right indicators, ensuring that they are free from external influences that may cast doubt about their credibility, and above all, remembering that an indicator is just that - an indication of something, not the thing itself.

Shaul Olmert. Photo: Orel Cohen

Shaul Olmert is a serial entrepreneur and the co-founder and CEO of mobile app developer Piggy. He formerly founded interactive content company Playbuzz Ltd. You can find his previous columns here