Introduction

Large Language Models (LLMs) can generate impressive responses, but they do not automatically have access to every piece of information a user may want to ask about. When an application needs to answer questions based on a specific document or knowledge source, relying only on the model can sometimes result in incomplete or unsupported answers.

This is where Retrieval-Augmented Generation (RAG) becomes useful.

As part of my Generative AI internship at Valentius Kryptix, I worked on building and analyzing a RAG system using Python, Google Gemini, and ChromaDB. The goal of the project was to understand how an LLM can be combined with an external knowledge source so that relevant information is retrieved before generating an answer.

The project gave me practical exposure to the complete RAG pipeline, from document processing and chunking to embeddings, vector storage, retrieval, and grounded response generation.

What is Retrieval-Augmented Generation?

Retrieval-Augmented Generation is an approach that combines information retrieval with Large Language Models.

Instead of directly sending a user’s question to an LLM, a RAG application first searches a knowledge base for information relevant to that question. The retrieved information is then provided to the LLM as context.

The basic workflow can be represented as:

Document → Chunking → Embeddings → Vector Database → User Query → Retrieval → Context → LLM → Response

This approach allows an AI application to work with information contained in external documents while reducing its dependence on the model’s pretrained knowledge.

Building the RAG Pipeline

For my implementation, I used Python as the primary programming language.

The system started with a PDF document that acted as the knowledge source. The document was processed and its text was extracted before being prepared for retrieval.

Document Processing and Chunking

The first stage of the pipeline was extracting text from the PDF using PyMuPDF.

Rather than treating the entire document as one large block of text, the extracted content was divided into smaller chunks. I used page-aware sentence-based chunking so that the relationship between the text and its original page could be maintained.

The test document contained 25 pages, resulting in 25 chunks or records being stored for retrieval.

This step is important because the way a document is divided can directly affect the quality of the information retrieved later. If chunks are too large, they may contain unnecessary information. If they are too small, important context may become separated.

Creating Embeddings

After processing the document, the text chunks were converted into vector representations called embeddings.

Embeddings represent the semantic meaning of text numerically. This makes it possible to compare a user’s question with stored document chunks based on semantic similarity rather than relying only on exact keyword matches.

For this project, I used Google’s Gemini embedding model to generate embeddings for the document chunks.

The resulting vectors provided the foundation for performing semantic search over the document.

Storing Vectors with ChromaDB

The generated embeddings were stored using ChromaDB, a vector database designed for working with embedding-based search.

The system stored the document chunks along with their corresponding vector representations and metadata.

When a user submits a question, the question can also be represented as an embedding. The system then searches the vector database to identify the most relevant pieces of information.

In my implementation, the retrieval stage used top-k retrieval with k = 5, meaning that the five most relevant records were selected as context for the generation stage.

Retrieval and Grounded Generation

After the relevant chunks were retrieved, they were passed to Google Gemini as contextual information.

The model could then generate an answer using the retrieved content rather than responding only from its general knowledge.

This represents the central idea behind the RAG architecture:

Retrieve relevant information first → provide that information to the LLM → generate a response grounded in the retrieved context.

One of the most important concepts I learned from this project was that the LLM is only one part of a RAG system. The retrieval process that supplies useful context is equally important.

Evaluating the RAG System

Building the pipeline was only one part of the project. I also wanted to understand whether the retrieval process was returning useful information.

I created a small retrieval evaluation using five questions, with the retrieval results saved for analysis.

One useful experiment involved comparing a normal Gemini response with a response generated using retrieved context.

For a corpus-specific question involving Aurora, a plain Gemini response produced an unsupported interpretation related to Aurora/AWS. When the same question was processed through the RAG pipeline, the system retrieved relevant sections from the document, including pages 11 and 23, and the generated response correctly identified SectionScope as the metadata filter described in the source.

This experiment demonstrated an important advantage of RAG: the model can use retrieved domain-specific information instead of relying entirely on its general knowledge.

Key Learnings

1. Retrieval Quality Matters

A RAG system is only as useful as the information it retrieves. Even a powerful LLM cannot provide a reliable answer if the relevant information is not included in its context.

This made document chunking, embeddings, and similarity search important parts of the overall system rather than simply being preprocessing steps.

2. Embeddings Enable Semantic Search

Working with embeddings helped me understand how text can be represented in a form that allows a system to find semantically related information.

A user’s question does not necessarily need to contain the exact same words as the document. Embedding-based retrieval can identify relevant content based on semantic similarity.

3. RAG Changes the Role of the LLM

The LLM does not necessarily need to be the source of all the information in a RAG application.

Instead, the retrieval system supplies relevant knowledge, while the LLM uses that knowledge to formulate a natural-language response.

Understanding this distinction helped me see how modern AI applications can combine information retrieval systems with generative models.

Challenges and Observations

One of the main challenges in developing a RAG system is finding the right balance between retrieving enough context and avoiding unnecessary information.

If chunks are too large, retrieval may bring excessive content into the prompt. If they are too small, important context can be separated across different chunks.

This made document chunking an important design decision in the project.

The retrieval experiments also showed why evaluating a RAG system with realistic questions is useful. Looking only at whether the application runs successfully does not necessarily tell us whether it is retrieving the correct information.

Comparing responses with and without retrieved context provided a practical way to understand the impact of the retrieval stage.

Conclusion

Building this RAG system gave me practical experience with one of the fundamental architectures used in modern Generative AI applications.

The project allowed me to work through the complete process of document processing, chunking, embeddings, vector storage, semantic retrieval, and LLM-based generation.

More importantly, it helped me understand that building an effective AI application is not only about selecting a powerful language model. The quality of the data, retrieval process, context construction, and evaluation all play important roles in the final result.

This project strengthened my interest in Generative AI and provided a practical foundation for exploring more advanced systems involving RAG, LLM applications, and AI agents.

I look forward to building on these concepts and exploring how retrieval and generative models can be combined to create more useful and reliable AI applications.

#GenerativeAI #RAG #ArtificialIntelligence #Python #LLM #MachineLearning #DataScience #Internship #ValentiusKryptix