OpinionThe clock is ticking for effective GPT data management
Opinion
The clock is ticking for effective GPT data management
"Attention and investment is going to startups attempting to build on top of GPT and less to those building solutions for data management and infrastructure," warns Judah Taub
Yes, "knowledge is power," and Sir Francis Bacon’s phrasing takes on a different meaning these days, four centuries after he said it. Today, we find power in Generative AI, a technology based on large language models, or LLMs, (like GPT), which allows us to constantly generate information based on enormous amounts of data.
This ability to automatically generate content from data - from software code to illustrations to movie scripts to synthetic data - is the next evolutionary stage in which manipulating, analyzing, or building on data is simpler for everyone, not only data scientists or data engineers.
For organizations, data is the new oil, but as with the case of "black gold", the collection process itself is not where we’ll find the real value. Only through the ability to accurately utilize it with AI and ML and secure infrastructure, can data actually be turned into valuable information and financial profit. Until a decade ago, data management was not a significant challenge, and usually, a small team was sufficient to handle the limited volumes of information.
Today, the situation is more complex, and a wider spread of roles in a given organization will have a hand in the company’s data operations. The transition to cloud environments and the movement to microservices and utilization of APIs has resulted in increased volume of data and a variety of organizational applications. At the same time, the accelerated development in AI technologies has resulted in organizations implementing layers of different applications on top of the data in order to analyze the information and generate valuable insights.
What we’re seeing is most people (and the media) focus mainly on the utilization of the data and not on handling the overall management of it. Attention - and investment - is going to startups attempting to build on top of GPT, for example, and less to those building solutions for data management and infrastructure.
However, AI systems, information security processes, and automated information flows require proper data management, or else they may get stuck in a 'bottleneck’, or much worse.
But the IDF’s development units are adapting to this new reality, and we’re going to see a similar trajectory as we witnessed with cyber security coming out of the Israeli military. Until around a decade ago, the "data unit" did not exist at all in the IDF's development departments (or only for ad-hoc activity). Today, more and more IDF coding development units are no longer focused solely on cyber or telecom but also on active data management - which can shed light on the next generation of founders who will be entering Israeli high-tech after military service.
As a result, we’ll end up seeing more Israeli startups operating in several major areas of data management, including:
- Data quality and data contracts, or ensuring the quality of information and setting rules for how information is shared, helping to keep the data correct and consistent no matter where it is used in a system.
- Process automation, which is about making processes work automatically, and doing this as close as possible to where the data actually exists.
- Analytics explainability, or making sense of data analysis. For instance, if something changes in the whole data system, how can you break it down in a simple way for the person running the business?
- Cost optimization for the modern data stack. This covers a wide range of tasks, not just storing data, but also how it's moved and processed.
In meetings with entrepreneurs, we see these trends already spreading in the market as they move to develop data and infrastructure solutions that will allow applications - such as GPT - to be better and more efficiently run.
There is no doubt that when the data is cleaner, more accurate, and easier to use, the tech running on it will yield more value in the long term. Development and investment in more effective data management is the key to better utilization of the variety of AI-based applications.
And in the 21st century, that is power.
Judah Taub is the founder and managing partner of Hetz Ventures, an Israeli venture capital fund that invests in early stage data and AI startups.
