Over the last 18 months, we have seen incredible advances in the field of Generative AI. Most of us use tools like ChatGPT or Google on a daily basis in our personal lives, but particularly in large enterprises, the manifestation of generative AI is done through a system called RAG (Retrieval Augmented Generation).
RAG systems are more common in enterprises as they operate only with the data a company owns or licenses and conform to enterprise security and data access. This approach significantly avoids the risk of hallucination (as available data is from sources owned and/or licensed by the company and thus trusted, known and credible). As a result, you can use the results from the RAG in your work because your company owns the copyright; it’s based on the data you own and you can verify the accuracy.
That said, sometimes an Enterprise IT department decides they want to build their own RAG. After all, how hard can it be? Vector DB + LLM = Done! Companies like Open AI, Microsoft and others are creating various tools and APIs so that developers can build their own.
Five Reasons Not to Build Your Own RAG
There is so much to this that goes beyond these APIs and tools. Over the last 18 months, the vast majority of these efforts have stalled or struggled badly. Here’s why:
- Lack of Resources: Virtually anyone can take a stack of PDFs and share them with an LLM for a quick demo. But to build an Enterprise System (or a department system within an Enterprise) is hard work. These projects can quickly become black holes requiring more and more resources (or die when they can’t get them).
- Conflicting Needs: Different business units have different needs and oftentimes different data sources. For example, perhaps the sales teams needs a Salesforce integration, the operations team needs ServiceNow, and the supply chain team needs SAP. Each business unit may have a source or sources that are required for a generative AI tool to be useful for them. It’s not a one size fits all. And by the way, each of those disparate systems likely has its own security model and access controls that need to be accounted for.
- Time Constraints: These systems require constant support and evolution. It’s not a 6-month project. But in the real world, IT resources often get reprioritized for the next project, leaving these custom solutions poorly supported or even abandoned.
- Inability to Support Ongoing Training: These solutions require training of the models for the various use cases they are to support. And the training is constant, requiring ongoing support and evolution.
- Change is Tough: For these systems to succeed, there is a level of change management necessary, along with a drive for user adoption. This requires experienced training resources and a commitment over time to provide support for ongoing adoption of the organization—another function often not accounted for or resourced for internally.
Don’t Go It Alone
There is an entire category of vendors that came out of the world of Enterprise & Federated Search that have been building robust solutions in this space for years. For example, Lucy (now known as the Capacity Answer Engine®) is in its 4th generation and was founded nearly a decade ago. We have more than a few competitors that have had similar journeys, building out search solutions for some of the world’s biggest companies. We have robust IP, hundreds of connectors to support safe integrations to enterprise systems, robust security models, sophisticated integrations to the access controls used in the Enterprise, unique features built based on years of customer feedback and more.
The tech, oftentimes evolving over years, is very mature and has taken tens of millions of dollars of investment to create, and has been battle tested in the most rigorous enterprises. And for us and our competitors, we have all integrated our solutions with the latest generative AI to further enhance the customer experience.
The other day, after a year of experimentation, a customer said “we make #### products, we don’t build RAGs”. The bottom line is that you don’t build your own CRM, CMS or Marketing Automation tool (at least I hope you don’t!), and you shouldn’t try building this.
And beyond the building of this tech, vendors like us and our competitors have experienced training and support organizations that are critical to drive constant evolution & adoption of these platforms—which is necessary to successfully deploy a system that actually gets used and adopted.
If you are about to start this journey, I’d suggest that the best solution will come from a close collaboration between the business stakeholders that need the solution, Enterprise IT to help manage the safe & secure access to data, and a specialized vendor that provides RAG solutions to the Enterprise.