Running a business is hard enough. With all the hype around information technology and how integral it is to doing anything in this world, business leaders must grapple with serious decisions as to where to place their technology investment dollars.
One would be hard-pressed to find a business process for which there isn’t a piece of software you can buy or subscribe to. And as software takes over, critically valuable information is generated. But it’s not enough that the information would just exist, it needs to be brought together into a view that accurately describes the business, thereby supporting the decision-making process.
Hence all the excitement around data. Because no transactional solution can ever claim to hold all available information, data technology evolved to occupy its own space in the world of business technology, confronting managers with serious decisions as to where to place their data technology investment dollars.
One Percenters
Most of what we learn about how to run a business is inspired by what happens in the largest, and for the most part, successful companies. A simple LinkedIn search will show that Amazon employs over 10,000 people whose job titles suggest that data manipulation and analysis is the focus of their work. Walmart, JPMorgan & Chase? Thousands each too.
Aside the fact that Walmart and Amazon each employ over 1 million people in total, the conclusion might be: if all great companies see their data as an asset so valuable that they need to employ armies of people to help them use it, then all companies that aspire to be great should have a data team too.
That type of reasoning might well be what’s driving increased demand for data professionals. But not even the strong response coming from our educational system seems to be able to alleviate what’s been a chronic scarcity of workers capable of filling open data positions.
The Job to be Done
Most companies look at the data challenge as an IT staffing issue. Any growing business will reach a point at which someone will say “it’s time we add a data team to our IT department”. Although the circumstances driving that conclusion may vary, it will probably come up when the businesspeople who’ve been responsible for reporting are no longer able to keep up. In other words, data work is always being done; it’s the amount of it that has been outgrowing existing resources.
Business and IT leaders could more easily solve that problem if they defined it in terms of desired business outcomes and then worked backwards towards finding the necessary resources to make it happen. That means that managers first need to ignore that data should be a business function at all and invest time inventorying the metrics they need to run the company, and in partnership with IT or outside help determine where to source the data that’s required to support on-demand access to key performance indicators.
Most people going through that process might be surprised to find out that the bulk of the problem—the time and the money—lies at the earlier stages of the data pipeline. Extracting data out of transactional systems and creating repositories to receive, process and deliver that to reporting tools is the critical piece. What may also come as a surprise is that once that work has been first and successfully completed, it may not require as much upkeep as one would expect; business models do change but are in general stable enough to back the assumption that you may actually get it sufficiently close the first time.
Like Uber for Data
According to the U.S. Census 99.7% of all companies in America employ less than 500 people. Of those, about 1 million employ between 20 and 500 people. These are the ones which may find themselves right at that juncture where they have accumulated enough business data to turn it into a critical asset. At that scale, standing up an in-house data team is not even a luxury anymore, it’s almost a physical impossibility. But why would you do it anyway?
