The recipe for RAG: How cloud services enable generative AI outcomes across industries
For technology leaders, the real decision is not whether RAG can improve answers, but whether the organisation can operationalise it as a governed service. The articleโs examples point to a broader management issue: retrieval quality, document control and deployment consistency matter as much as model choice. CIOs and transformation leaders should treat RAG as part of the information architecture and operating model, not as a standalone AI feature.
That shifts the priorities. The first question is who owns source content, indexing rules, retention and auditability across business domains. If legal, customer service or research teams each curate their own data sets, the enterprise risks inconsistent outputs and unclear accountability. A practical next step is to define decision rights for what data can be used, how often it is refreshed, and which teams approve changes to prompts, retrieval layers and access controls.
There is also a portfolio trade-off. RAG can reduce hallucinations and improve usefulness, but it only works well when the underlying documents are secure, current and well structured. That means funding may need to shift away from more visible model experimentation toward less glamorous work such as content remediation, metadata standards, records management and search quality. Leaders should ask where the highest-value knowledge lives and whether the organisation is prepared to maintain it as a product.
The infrastructure message is equally important. Cloud-managed storage and serverless services can shorten delivery time, but they also create dependency on vendor controls, resilience assumptions and compliance evidence. Before scaling, executives should ask: what is the fallback if retrieval fails, which workloads require immutable records, and how will benefits be measured beyond prototype success? For most enterprises, the winning question is not โCan we build RAG?โ but โCan we govern trusted AI outputs at scale without increasing operational risk?โ
According to research from IBMยฎ, about 42% of enterprises surveyed have AI in use in their businesses. Of all the use cases, many of us are now extremely familiar with natural language processing AI chatbots that can answer our questions and assist with tasks such as composing emails or essays. Yet even with widespread adoption of these chatbots, enterprises are still occasionally experiencing some challenges. For example, these chatbots can produce inconsistent results as theyโre pulling from large data stores that might not be relevant to the query at hand.
Thankfully, retrieval-augmented generation (RAG) has emerged as a promising solution to ground large language models (LLMs) on the most accurate, up-to-date information. As an AI framework, RAG works to improve the quality ofย LLM-generatedย responses by grounding the model on sources of knowledge to supplement the LLMโs internal representation of information. IBM unveiled its new AI and data platform, watsonxโข, which offers RAG,ย back in May 2023.
In simple terms, leveraging RAG is like making the model take an open book exam as you are asking the chatbot to respond to a question with all the information readily available. But how does RAG operate at an infrastructure level? With a mixture of platform-as-a-service (PaaS) services, RAG can run successfully and with ease, enabling generative AI outcomes for organizations across industries using LLMs.
How PaaS services are critical to RAG
Enterprise-grade AI, including generative AI, requires a highly sustainable, compute- and data-intensive distributed infrastructure. While the AI is the key component of the RAG framework, other โingredientsโ such as PaaS solutions are integral to the mix. These offerings, specifically serverless and storage offerings, operate diligently behind the scenes, enabling data to be processed and stored more easily, which provides increasingly accurate outputs from chatbots.
Serverless technology supports compute-intensive workloads, such as those brought forth by RAG, by managing and securing the infrastructure around them. This gives time back to developers, so they can concentrate on coding. Serverless enables developers to build and run application code without provisioning or managing servers or backend infrastructure.
If a developer is uploading data into an LLM or chatbot but is unsure of how to preprocess the data so itโs in the right format or filtered for specific data points, IBM Cloudยฎ Code Engine can do all this for themโeasing the overall process of getting correct outputs from AI models. As a fully managed serverless platform, IBM Cloud Code Engine can scale the application with ease through automation capabilities that manage and secure the underlying infrastructure.
Additionally, if a developer is uploading the sources for LLMs, itโs important to have highly secure, resilient and durable storage. This isย especially critical in highly regulated industries such as financial services, healthcare and telecommunications.
IBM Cloud Object Storage, for example, provides security and data durability to store large volumes of data. With immutable data retention and audit control capabilities, IBM Cloud Object Storage supports RAG by helping to safeguard your data from tampering or manipulation by ransomware attacks and helps ensure it meets compliance and business requirements.
With IBMโs vast technology stack including IBM Code Engine and Cloud Object Storage, organizations across industries can seamlessly tap into RAG and focus on leveraging AI more effectively for their businesses.
The power of cloud and AI in practice
Weโve established that RAG is extremely valuable for enabling generative AI outcomes, but what does this look like in practice?
Blendow Group, a leading provider of legal services in Sweden, handles a diverse array of legal documentsโdissecting, summarizing and evaluating these documents that range from court rulings to legislation and case law. With a relatively small team, Blendow Group needed a scalable solution to aid their legal analysis. Working withย IBM Client Engineering and NEXER, Blendow Group created an innovative AI-driven tool, leveraging the comprehensive capabilities ofย toย enhance research and analysis, and streamlines the process of creating legal content, all while maintaining the utmost confidentiality of sensitive data.
Utilizing IBMโs technology stack, including IBM Cloud Object Storage and IBM Code Engine, the AI solution was tailored to increase the efficiency and breadth of Blendowโs legal document analysis.
The Mawsonโs Huts Foundation is also an excellent example of leveraging RAG to enable greater AI outcomes. The foundation is on mission to preserve the Mawson legacy, which includes Australiaโs 42 percent territorial claim to the Antarctic and educate schoolchildren and others about Antarctica itself and the importance of sustaining its pristine environment.
With The Antarctic Explorer, an AI-powered learning platform running onย IBM Cloud, Mawson is bringing children and others access to Antarctica from a browser wherever they are. Users can submit questions via a browser-based interface and the learning platform uses AI-powered natural language processing capabilities provided byย IBM watsonx Assistantโขย to interpret the questions and deliver appropriate answers with associated mediaโvideos, images and documentsโthat are stored in and retrieved fromย IBM Cloud Object Storage.
By leveraging infrastructure as-a-service offerings in tandem with watsonx, both the Mawson Huts Foundation and Blendow Group are able to gain greater insights from their AI models by easing the process of managing and storing the data that is contained within them.
Enabling Generative AI outcomes with the cloud
Generative AI and LLMs have already proven to have great potential for transforming organizations across industries. Whether itโs educating the wider population or analyzing legal documents, PaaS solutions within the cloud are critical for the success of RAG and running AI models.
At IBM, we believe that AI workloads will likely form the backbone of mission-critical workloads and ultimately house and manage the most-trusted data, so the infrastructure around it must be trustworthy and resilient by design. With IBM Cloud, enterprises across industries using AI can tap into higher levels of resiliency, performance, security, compliance and total cost of ownership. Learn more about IBM Cloud Code Engine and IBM Cloud Object Storage below.
IBM Cloud Code Engine IBM Cloud Object StorageThe post The recipe for RAG: How cloud services enable generative AI outcomes across industries appeared first on IBM Blog.
Enjoyed this article? Sign up for our newsletter to receive regular insights and stay connected.