RAG Expert Assistant

A production-grade Retrieval-Augmented Generation (RAG) pipeline built on Google Gemini, LangChain, and ChromaDB. It retrieves answers from your own documents with source citations, reranks results locally, evaluates output quality with RAGAS, and defends against prompt injection and PII leakage.

• Document ingestion - loads .txt/.md docs, chunks them with semantic boundaries
• Vector search - ChromaDB + Gemini embeddings (gemini-embedding-001)
• Local reranking - FlashRank reranker (no API key needed) to boost precision
• Grounded answers - Gemini LLM answers only from retrieved context with [Source N] citations and confidence ratings
• RAGAS evaluation - faithfulness, answer relevancy, context precision & recall with retry/backoff
• A/B comparison - benchmark Naive vs Optimized retrieval configurations side-by-side
• Security layer - prompt injection defense, PII detection/redaction, output filtering
• Config-driven - all tunables live in configs/base.yaml

data/sample_docs
      │  load_documents()          (rag_pipeline.py)
      ▼
Raw documents
      │  chunk_documents()         (512-char chunks, 50 overlap)
      ▼
Chunks
      │  build_vectorstore()       (Gemini embeddings → ChromaDB)
      ▼
Vector store
      │  build_retriever()         (retrieve top-20 → FlashRank rerank top-5)
      ▼
Retriever
      │  build_rag_chain()         (prompt + citations + Gemini LLM)
      ▼
RAG chain  ──►  query_pipeline()   (answer + sources)

.
├── configs/base.yaml        # Pipeline configuration (chunking, retrieval, LLM, security)
├── data/sample_docs/        # Sample documents (.txt/.md)
├── src/
│   ├── rag_pipeline.py      # Loading, chunking, vectorstore, retriever, RAG chain
│   ├── evaluate.py          # RAGAS evaluation setup and runner
│   ├── ab_comparison.py     # Naive vs Optimized A/B benchmark
│   ├── helper.py            # Shared retry/backoff helper for RAGAS calls
│   └── security/
│       ├── sanitizer.py     # PII detection, injection defense, output filtering
│       └── test_security.py # pytest suite for the security layer
└── pyproject.toml           # Project metadata & dependencies (uv-managed)

• Python >= 3.13
• A Google AI API key (GOOGLE_API_KEY)

# 1. Clone & enter the project
git clone <repo-url> && cd "RAG Expert Assistant"

# 2. Set up environment (uv recommended)
uv sync

# 3. Configure your API key in a .env file
echo "GOOGLE_API_KEY=your_key_here" > .env

Note: .env and .venv/ are gitignored. Runtime artifacts like chroma_db/ are created on first run.

We are currently pinning to the in-repo .venv (via uv sync --active or activating it directly) instead of resolving fresh, because ragas==0.4.3 has a broken Gemini integration: ChatVertexAI imports from a langchain_community path that no longer exists in newer langchain releases. See the upstream issue:

• ragas #2745 — "ChatVertexAI import broken — uses removed langchain_community path"

Until ragas ships a fix (or we upgrade), do not delete/regenerate the virtual environment, as a clean resolve will pull incompatible versions.

uv run src/rag_pipeline.py

Asks a sample question ("What are the available payment methods?") and prints the answer with citations and retrieved source contexts.

uv run -m src.evaluate

Scores the built-in evaluation set across faithfulness, answer relevancy, context precision, and context recall.

uv run -, src.ab_comparison

Builds and evaluates two configurations (RAGConfig) over 10 test questions and prints a metric-by-metric comparison table:

uv run src/security/sanitizer.py   # run injection & PII demo
uv run pytest                              # run pytests for the security layer

All pipeline settings live in configs/base.yaml:

• chunking - chunk size, overlap, separators
• embedding - Gemini embedding model & dimensions
• vectorstore - ChromaDB collection name & persist directory
• retrieval - top-k for retrieval and after rerank, search type
• reranker - local FlashRank model
• llm - Gemini generation model, temperature, max tokens
• evaluation - thresholds for faithfulness/relevancy/precision/recall
• security - toggles for PII detection, injection defense, output filtering

uv run pytest                                          # all tests
uv run pytest src/security/test_security.py -v        # security layer only

MIT License (add a LICENSE file to formalize).