AI Knowledge Assistant — Local Air-Gapped RAG Pipeline
Air-gapped document intelligence platform with ChromaDB, FastAPI, and local LLM synthesis without cloud leakage.
Project Overview & Objectives
Purpose, Problem & Core Value Proposition
An air-gapped, privacy-first document intelligence platform engineered for enterprises, law firms, and medical clinics that handle highly confidential SOPs, patient charts, and proprietary contracts. By combining a local ChromaDB vector store with locally hosted LLMs (via Ollama) orchestrated by a high-throughput FastAPI backend, it provides grounded document Q&A with verifiable source citations, ensuring zero data packets ever leave the local network.
Purpose & Objective
To provide enterprise-grade document search and conversational intelligence with mathematical guarantee of data privacy through 100% local processing.
Problem Solved
Clinics and corporate legal departments possess sensitive documents that strictly cannot be uploaded to commercial cloud AI vendors (OpenAI, Anthropic) due to HIPAA, DPDP, and confidentiality regulations.
Target Audience
Healthcare clinics, legal firms, accounting practices, and research institutions handling restricted internal documentation.
Key Engineering Highlights
Screenshots & Visual Showcase
Production UI & Page Details
Air-Gapped RAG Document Query Interface
Interactive chat UI with multi-turn grounding, syntax-highlighted code generation, and Word/PDF export.
ChromaDB Document Ingestion & Vector Index
Local document upload pipeline with automatic chunking, embedding generation, and ChromaDB sync.
Source Grounding & Inference Metrics
Real-time token generation timing, data source provenance tracking, and local LLM parameter controls.
Complete Feature Breakdown
Implemented Functionality & Workflows
Every feature listed below represents verified, working code. Categorized by system tier and user workflow.
Privacy & Air-Gapped Architecture
Zero external network calls; complete local sovereignty.
Ingestion & Document Intelligence
Semantic chunking and cited source verification.
User Experience & Export
Fast, clear research experience for knowledge workers.
Technology Stack
Architectural Tools & Libraries
Technologies selected for performance, reliability, and maintainability.
Frontend & UI
Backend & APIs
Database & Storage
Engineering Tools
Technical Implementation
Architecture, Workflows & Decisions
Engineering breakdown covering data pathways, architectural trade-offs, and production optimizations.
System Architecture
Constructed as a decoupled client-server architecture. The frontend is a responsive Next.js web application connecting to a high-concurrency Python FastAPI backend running locally at 127.0.0.1:8000. ChromaDB runs embedded in-process for low-latency similarity search.
Data Communication & State Orchestration
Server-Sent Events (SSE) stream LLM tokens from FastAPI to Next.js in real time with sub-100ms time to first token.
Key Operational Workflows
User uploads private PDF/DOCX via drag-and-drop
FastAPI parses text, applies semantic chunking, and generates vector embeddings locally
User enters a query; system performs cosine similarity search across ChromaDB embeddings
Top-k relevant chunks are injected into the local LLM prompt as context with strict grounding rules
Local LLM streams response to UI with interactive citation pill highlights
Architectural Decisions
- •Selected ChromaDB embedded mode rather than a heavy containerized cluster to keep memory footprint under 2GB on developer laptops
- •Used Ollama as the local LLM runner to leverage hardware acceleration (Metal, CUDA, Vulkan) seamlessly
- •Adopted recursive character chunking with 10% overlap to preserve cross-sentence semantic context
Performance & Security
- Zero egress network traffic ensures 100% compliance with DPDP Act 2023 and HIPAA guidelines
- Retrieval latency under 300ms across 10,000+ indexed document chunks
- Strict prompt injection defense preventing jailbreaks on internal data
Measurable Outcomes
Real Operational Results
Verified performance indicators and business impact. No inflated metrics.
Sub-300ms retrieval latency on consumer GPU hardware
Zero bytes of document data leaked to external cloud vendors
100% local vector indexing with verified source highlights
Want a high-converting website for your business?
Let's talk about your customers, your goals, and what a good enquiry looks like. No obligation on the initial discovery call.