Case Study · AI / ML
IntelliFAQ — RAG-Powered AI
A retrieval-augmented FAQ engine that answers natural-language questions grounded in an organization's own knowledge base — and turns that knowledge into courses and quizzes.
The problem
Organizations sit on large, ever-changing knowledge bases — help docs, policies, course material — but the people who need answers can't always find them. Keyword search breaks the moment someone phrases a question differently from the source text, and a generic LLM will happily hallucinate an answer that sounds right but isn't grounded in the company's actual documentation.
The goal was an API that answers questions in natural language using only the organization's own content, cites what it's drawing from, and stays current as the underlying docs change — without retraining a model every time something is edited. As a stretch, it should be able to repurpose that same knowledge into learning material (structured courses and multiple-choice questions).
How it works
Ingestion & indexing
Source documents are split into overlapping chunks, embedded into vectors, and stored in Pinecone alongside metadata. Re-indexing is the only step needed when content changes — no model retraining — which keeps answers current and the update cost near zero.
Retrieval → generation (RAG)
At query time the user's question is embedded and used to pull the most semantically relevant chunks from Pinecone. Those chunks are injected into the prompt as grounding context, and Azure OpenAI's GPT-4o generates an answer constrained to that context. LangChain orchestrates the retrieve-then-generate flow.
Because the model only reasons over retrieved, real content, answers stay grounded and hallucinations drop sharply compared to asking the model cold.
Service layer & operations
The whole pipeline is exposed through a FastAPI service with auto-generated OpenAPI docs, plus endpoints for analytics, monitoring, and AI-driven course/MCQ generation built on the same retrieval layer. The service is containerized with Docker for reproducible deployment.
Key decisions & tradeoffs
RAG instead of fine-tuning the model
Why: Retrieval keeps answers grounded in the source docs and current — updating knowledge is just re-indexing, not retraining. It also sidesteps the cost and hallucination risk of baking facts into model weights.
Tradeoff: Answer quality becomes bounded by retrieval quality, so the engineering effort shifts to chunking and embedding strategy rather than model training.
Pinecone (managed vector DB) over self-hosted FAISS
Why: A managed vector store removes infra ops, scales without babysitting, and supports metadata filtering out of the box.
Tradeoff: Adds an external dependency and per-usage cost versus running an in-process index myself.
FastAPI as the service framework
Why: Async I/O suits the network-bound calls to Pinecone and Azure OpenAI, and the auto-generated OpenAPI surface makes the API explorable and easy to integrate against.
Tradeoff: Python's concurrency model means CPU-bound work would need offloading, but this workload is almost entirely I/O.
Outcome
- Answers are grounded in the organization's own documents rather than the model's training data, sharply reducing hallucination.
- Knowledge updates require only re-indexing — no retraining cycle.
- A single retrieval layer powers three products: Q&A, course generation, and MCQ generation.
- Fully containerized and documented via OpenAPI for clean handoff and integration.
What I'd improve next
- Add a reranking step over the top-k retrieval to lift precision before generation.
- Build an evaluation harness to measure retrieval relevance and answer faithfulness over time.
- Stream responses token-by-token and cache frequent queries to cut latency and cost.