RAG Document Q&A Pipeline | Rutvik Mendpara

An end-to-end retrieval-augmented generation (RAG) pipeline for answering questions over a custom document set.

RAG pipeline: documents are chunked, embedded and indexed in FAISS offline; questions retrieve top-K chunks that are passed to an LLM

Indexing happens once, offline. At question time the top-K chunks are retrieved and passed to the LLM as context.

How it works

Documents are split into chunks and embedded with Sentence Transformers, and the embeddings are stored in a FAISS index. At query time the question is embedded the same way, the top-K most similar chunks are retrieved, and they are passed to an LLM as context for generating the answer.

What I explored

I compared fixed-size chunking with sentence-based chunking. Fixed-size chunks with overlap worked better on longer documents, because sentence boundaries sometimes split an idea that needed its surrounding context. I also measured how retrieval quality (Recall@K) changed with chunk size, evaluating retrieval on its own rather than only judging the final answers. When an answer is wrong, that separation shows whether the right chunk was never retrieved or the model failed to use it.

I wrote up more on these trade-offs in Why Chunking Matters in RAG Pipelines.

Stack

Python · FAISS · Sentence Transformers · LangChain · OpenAI API