RAG Agent - Local Knowledge Assistant

💬

Chat Interface

Web UI and CLI with real-time streaming responses. Ask questions and get answers with source citations from your knowledge base.

📄

Document Ingestion

Ingest PDF, Word, PowerPoint, Excel, HTML, Markdown, and text files. Automatic format detection and conversion via Docling.

🎤

Audio Transcription

Transcribe MP3, WAV, and M4A files using Whisper ASR. Audio content becomes fully searchable through the RAG pipeline.

🕸️

Web Crawler

Crawl documentation sites and ingest them into your knowledge base. Supports sitemaps, recursive crawling, and llms.txt.

🧠

Semantic Chunking

LLM-driven semantic splitting with rule-based fallback. Configurable chunk sizes for optimal retrieval performance.

🔒

Fully Local

Runs entirely on your machine with Ollama. No API keys required, no data leaves your network. Complete privacy.

Architecture

┌───────────────────────────────────────────────────────────────┐ │ USER INTERFACES │ │ ┌─────────────────────┐ ┌─────────────────────┐ │ │ │ Web Interface │ │ CLI Interface │ │ │ │ (FastAPI + HTML) │ │ (Python async) │ │ │ └──────────┬──────────┘ └──────────┬──────────┘ │ └─────────────┼─────────────────────────┼───────────────┘ │ │ └────────────┬────────────┘ │ ┌─────────────┴─────────────────┐ │ RAG Agent Core │ │ ┌───────────────────────────┐ │ │ │ PydanticAI Agent │ │ │ │ + search_knowledge_base() │ │ │ └───────────────────────────┘ │ └─────────────┬─────────────────┘ │ ┌──────────────────┼──────────────────┐ │ │ │ ┌────────┴────────┐ ┌──────┴───────┐ ┌──────┴───────┐ │ Embeddings │ │ LLM │ │ PostgreSQL │ │ (Ollama/ │ │ (Ollama/ │ │ + PGVector │ │ OpenAI) │ │ OpenAI) │ │ │ └─────────────────┘ └──────────────┘ └──────────────┘

Python 3.10+ PydanticAI Ollama Docling Whisper ASR PostgreSQL PGVector FastAPI crawl4ai Docker

Supported Formats

Format	Extensions	Processing
PDF	.pdf	Docling conversion
Word	.docx, .doc	Docling conversion
PowerPoint	.pptx, .ppt	Docling conversion
Excel	.xlsx, .xls	Docling conversion
HTML	.html, .htm	Docling conversion
Markdown	.md	Direct processing
Text	.txt	Direct processing
Audio	.mp3, .wav, .m4a	Whisper transcription

Quick Start

# 1. Install dependencies
cd rag-agent
uv sync && cp .env.example .env

# 2. Configure .env with DATABASE_URL and Ollama settings

# 3. Set up database
psql $DATABASE_URL < sql/schema.sql

# 4. Pull Ollama models
ollama pull mistral
ollama pull nomic-embed-text

# 5. Start the web interface
uv run python web_app.py    # http://localhost:8000

# Or use the CLI
uv run python cli.py

# 6. Ingest documents
uv run python -m ingestion.ingest --documents documents/

Docker Deployment

# Start all services
docker-compose up -d

# Ingest documents
docker-compose --profile ingestion up ingestion

Docker Hub

Pre-built Docker image available for quick deployment.

# Pull the image
docker pull maruha/rag-agent

# Run with environment variables
docker run -d --name rag-agent \
  --env-file .env \
  -p 8000:8000 \
  maruha/rag-agent

hub.docker.com/r/maruha/rag-agent

LLM Provider Options

Ollama (Local)

Run entirely offline. No API costs.

LLM: mistral, llama3.2, qwen2.5, deepseek-r1

Embeddings: nomic-embed-text, mxbai-embed-large

OpenAI (Cloud)

Higher quality, requires API key.

LLM: gpt-4o-mini, gpt-4o

Embeddings: text-embedding-3-small

Prerequisites

This is a server-side application that requires a local runtime environment to operate.

Requirement	Purpose
Python 3.10+	Runtime (3.11+ recommended)
UV	Package manager
Ollama	Local LLM inference
PostgreSQL + PGVector	Vector knowledge base
Docker (optional)	Containerized deployment