OpinionOrganizational GenAI: The challenge, the great opportunity, and the emerging killer application
Opinion
Organizational GenAI: The challenge, the great opportunity, and the emerging killer application
"Some organizations are already integrating GenAI into their products through local development, often ignoring complex issues of access permissions and governance," writes EMET Group CTO Yoel Jacobson.
The computing world is deeply entrenched in the AI revolution. Unlike previous revolutions that took years to mature, the AI revolution is self-sustaining and advancing at an unprecedented pace. Companies are continuously evaluating technologies, and making decisions regarding which to adopt and invest in is a complex challenge.
There are AI domains that have been around for several years and are tried and tested, like AI on tabular data. However, the current wave of the revolution is focusing on the newest domain of AI - Generative AI (GenAI), primarily around text and language, with its organizational use still shaping and forming during this period.
GenAI, and particularly ChatGPT, has brought AI to billions of users worldwide due to its user-friendly operation and excellent content outputs, which are also used by professionals from various fields. So far, there are several GenAI use cases that have started to establish themselves among organizations and proven to be effective:
- Code Generation: Products like GitHub Copilot and others help developers build quality code quickly and understand existing code better. Another type of code generation enables querying organizational databases through natural language queries.
- Semantic Searches: If you’ve ever wasted time searching for data in an organizational system because you didn’t use the correct sequence of letters or the exact word, you can appreciate natural language searches that return everything related to what you asked for in terms of meaning and semantic proximity.
- Organizational Versions of “Classic” GenAI Uses: Such as conversing with an organizational chatbot, summarizing long documents, automatic translation between languages, document comparison, and content creation.
However, full-scale organizational adoption faces numerous challenges. Organizations recognize the need to not fall behind in the technological race and allocate budgets for implementing GenAI applications, sometimes at the expense of budgets allocated for other uses. But a long list of considerations delays or prevents the move, as revealed by a survey from ETR - Enterprise Technology Research. According to the survey: 83% of organizations are still in the evaluation stages; 38% fear data privacy and security issues; 37% are concerned about compliance and regulatory issues, and 6% believe there is no business need for the technology.
Above all these considerations, the issue of return on investment (ROI) remains unclear. A recent Gartner study states that by 2025, at least 30% of GenAI projects will be abandoned after the proof of concept (POC) phase due to reasons like poor data quality, inappropriate risk controls, high costs, and unclear business value.
RAG Systems: The Organizational Killer Application
On the ground, there is active engagement from organizations that don’t want to be left behind and are trying various applications with positive market feedback. The hot topic today in the organizational GenAI field is RAG - Retrieval Augmented Generation. These systems are based on the collection infrastructures of previous-generation enterprise search products but are combined with modern language models and semantic retrieval to enable querying in natural language. RAG products produce coherent answers in natural language with references to all the organizational information that built the answer. It is possible to combine the organizational search with internet searches that cross-reference and enrich the information products.
RAG is implemented on the organizational data sources, which are accurate and verified, reducing the risk of incorrect information and increasing decision-making reliability. It reduces biases compared to generic LLM models and can be customized to the specific needs and tasks of users, departments, and the entire organization.
RAG is gaining significant popularity among organizations worldwide and has a chance to become the Killer Application that attracts businesses and companies of all sizes and types, potentially cementing GenAI as a mature and suitable technology for organizations.
Many companies use open public RAG models to develop their solution, but organizations concerned with security and privacy breaches can build a private solution based on various platforms available in the market.
Many organizations in Israel and around the world are currently in the evaluation and learning stages. Some are already integrating GenAI into their products through local development, often ignoring complex issues of access permissions and governance. However, the field of dedicated products still requires relevant use case proofs for the Israeli market. There are several solutions already implemented in local organizations to prevent information leakage and misuse and a variety of solutions with the possibility of local implementation for code completion and code analysis. The recommendation remains to conduct a thorough feasibility study, even if the product works well in other markets.
Yoel Jacobson is the CTO at EMET Group.
