How to run RAG projects for better data analytics results

"The arrival of generative AI-enhanced business intelligence (GenBI) for enterprise data analytics has opened up access to insights, while also increasing the speed, relevance and accuracy of those insights. But that's in best-case scenarios. Often, AI-powered analytics leads data teams to the same challenges: Hallucinations, security and governance snafus, outdated or incorrect answers, low familiarity with niche areas of expertise, and an inability to deliver answers grounded in proprietary data."

"Many of these challenges stem from a single factor: The LLMs that form the foundation for GenBI can only draw on their training data for answers, and this training data is largely static and inflexible. While retrieval-augmented generation (RAG) offers a solution, it isn't always implemented in ways that yield exemplary results. Some experts are extremely skeptical about the technology, estimating that real-world RAG implementations only produce successful outputs 25% of the time."

"A recent research paper from Google in partnership with the University of Southern California found that RAG-enhanced model output only included direct answers to users' questions 30% of the time, with problematic output most commonly attributed to perceived conflicts between internal information and retrieved information. When done correctly, RAG enhances LLM knowledge by augmenting it with data retrieved from external sources, including internal knowledge bases, proprietary databases and documentation repositories."

Generative AI-enhanced business intelligence expands access to faster, more relevant enterprise insights but faces accuracy, hallucination, security, governance and domain-knowledge gaps because base LLMs rely on static training data. Retrieval-augmented generation augments LLMs with external sources—internal knowledge bases, proprietary databases and documentation repositories—to ground answers in up-to-date, proprietary data. Real-world RAG deployments often underperform; experts estimate roughly 25–30% direct-answer success, with conflicts between internal and retrieved information a common failure mode. Achieving reliable GenBI requires clean, curated data, precise prompt engineering, robust system design, and strong security and governance practices.

Read at InfoWorld