Local RAG Mubashir Hussain

LOCAL-FIRST CUSTOMER SUPPORT

Answers you
can trace.

A RAG assistant that cites the exact words behind an answer and abstains when the documentation falls short. Built around LumaDock, a fictional product.

Static showcase · Real models run locally · No hosted inference

Local frontend captureFactory reset with citations
LumaDock support interface showing the factory reset answer and three citations from the Factory Reset Procedure.
From the recorded local acceptance. View mobile capture
9/9Exact-source citations
in the live fictional evaluation
9/12Rubric-correct outcomes
in that same evaluation
5Read-only MCP tools
in the investigation workflow

ARCHITECTURE

From documents to
verifiable answers.

  1. Ingest

    Local documents

    The CLI normalizes selected Markdown and text, preserving document versions, chunk IDs and exact source offsets. Previewed changes reuse compatible embeddings.

  2. Retrieve

    Hybrid search

    FastAPI combines Nomic cosine similarity with SQLite FTS5 keyword ranking using reciprocal rank fusion, then passes bounded passages to local Qwen.

  3. Validate

    Cite or abstain

    The backend checks quote completeness, source offsets, relevance and conflicts. React shows the answer, passages and retrieval ranks.

OPTIONAL WORKFLOW

LangGraph + MCP

A bounded graph calls five allowlisted, read-only tools for indexed documents, passages, versions and evidence. Its deterministic trace keeps provenance inspectable. The scoped real Nomic query acceptance made zero generation calls. Investigation architecture

Release the reset button when the status light flashes amber three times.
Factory Reset Procedure v2.3 reset:c000 · source characters [203, 276)

EVIDENCE AND TRADEOFFS

Exact quotes can still
miss the question.

Grounding and safety

Indexes are read-only during questions. Instruction-like source documents can be quarantined, conflicting evidence can trigger abstention, and the API and investigation tools have no customer actions.

Two live answers quoted the wrong entities even though the citations were exact. Offline hardening rejected those selections and recovered a supported paraphrase. Two false abstentions remained in replay.

Recorded live evaluation
Offline replay and 17-case holdout

What was tested

Local embedding, generation, document lifecycle and scoped investigation runs have separate evidence from synthetic tests. Reviewed implementation CI passed 664 Python tests per version, four Vitest tests, four synthetic Playwright tests and the frontend build.

Fictional responders verified Unix-socket HTTP on Ubuntu WSL2. The application’s private Nginx socket and IPv4-loopback host proxy have offline coverage. Full production container acceptance (5B) and operational hardening (5C) are deferred.

Reviewed implementation CI
Synthetic socket diagnostic

LOCAL SETUP

Run it on your machine.

Use Python 3.12+, Node.js 24, and locally installed Ollama models nomic-embed-text:latest and qwen3:4b. Start Ollama on loopback. The application makes real local model requests during ingestion and questions.

Use .venv\Scripts\python.exe on Windows. In a second terminal, run cd frontend, npm ci, then npm run dev. Open http://127.0.0.1:5173.

Read the full setup and test instructions

Backend setup · from the repository root

python -m venv .venv
.venv/bin/python -m pip install -r requirements.lock
.venv/bin/python -m support_agent.cli corpus
.venv/bin/python -m uvicorn support_agent.api:app \
  --host 127.0.0.1 --port 8000 --workers 1