Building a personal AI assistant with RAG — connecting your Obsidian vault to an LLM
General-purpose AI assistants don't know your context. They don't know you're building a payment SaaS for the African market, that you prefer Convex over Supabase for a given use case, or that you already fixed this exact bug last month.
RAG — Retrieval-Augmented Generation — is the fix. Before answering, the assistant searches your knowledge base and injects the relevant passages into its response. It talks with your memory.
This tutorial walks through how I built hub ai — an AI assistant connected to my Obsidian vault, running locally, with specialized personas and smart routing to different LLMs depending on the task.
Prerequisites: Python 3.10+, Docker, an Obsidian vault, a Groq API key (free). Source code:
~/.local/share/ai-assistant/— Python indexer + FastAPI proxy + Open WebUI.
Overall architecture
┌─────────────────────────────────────────────────────────────┐
│ Open WebUI │
│ (browser-based chat interface) │
└─────────────────────┬───────────────────────────────────────┘
│ POST /v1/chat/completions
▼
┌─────────────────────────────────────────────────────────────┐
│ FastAPI Proxy :11435 │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ 1. Embed the query (sentence-transformers) │ │
│ │ 2. Search ChromaDB (top-5 chunks) │ │
│ │ 3. Build the RAG context │ │
│ │ 4. Route to the right LLM based on the persona │ │
│ └─────────────────────────────────────────────────────┘ │
└──────────┬───────────────────────────┬──────────────────────┘
│ │
▼ ▼
┌──────────────────┐ ┌──────────────────────────┐
│ Groq API │ │ Ollama (local) │
│ llama-3.3-70b │ │ mistral:7b │
│ deepseek-r1-70b │ │ (offline fallback) │
└──────────────────┘ └──────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ ChromaDB │
│ 2,045 chunks indexed from ~/Brain/ │
└─────────────────────────────────────────────────────────────┘
Three components: an indexer that reads the vault, a vector database that stores the chunks, and a proxy that orchestrates everything.
1. Core concepts
Embeddings — turning text into vectors
An embedding is a numeric representation of a piece of text. Two semantically close texts have vectors that sit close together in mathematical space.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("paraphrase-multilingual-MiniLM-L12-v2")
v1 = model.encode("Comment indexer un vault Obsidian ?")
v2 = model.encode("How to index an Obsidian vault?")
# v1 and v2 are close together despite the different languagesThe paraphrase-multilingual-MiniLM-L12-v2 model supports 50+ languages in the same vector space. A French vault becomes queryable in English without any translation step.
Documentation: sbert.net — the full list of sentence-transformers models with their benchmarks.
RAG — search before you generate
RAG (Retrieval-Augmented Generation) is a two-step pattern:
- Retrieval — encode the question as a vector, search for the K most similar passages in the database
- Generation — inject those passages into the system prompt, send it to the LLM
[User question]
↓ embed
[Question vector] → ChromaDB → [5 relevant passages]
↓ concat
[Prompt: "Here's some context: {passages}. Question: {question}"]
↓ LLM
[Answer grounded in the vault]
Without RAG, the LLM makes things up. With RAG, it cites sources.
Founding paper: RAG for Knowledge-Intensive NLP Tasks (Lewis et al., 2020) — the paper that formalized the concept.
ChromaDB — a local vector database
ChromaDB is an open-source vector database that runs entirely locally, with no external server.
import chromadb
client = chromadb.PersistentClient(path="./chroma_db")
collection = client.get_or_create_collection(
name="brain",
metadata={"hnsw:space": "cosine"} # cosine similarity
)
# Index
collection.add(
documents=["chunk text"],
embeddings=[[0.1, 0.3, ...]],
metadatas=[{"source": "notes/20260430.md", "type": "note"}],
ids=["chunk-001"]
)
# Search
results = collection.query(
query_embeddings=[question_vector],
n_results=5
)ChromaDB documentation: docs.trychroma.com — getting started, metadata filters, HNSW configuration.
2. The indexer — reading and chunking the vault
The indexer reads every .md file in the vault, splits it into chunks, encodes them, and stores them in ChromaDB.
# brain_indexer.py
import os
import re
from pathlib import Path
from sentence_transformers import SentenceTransformer
import chromadb
BRAIN_DIR = Path.home() / "Brain"
CHROMA_PATH = Path.home() / ".local/share/ai-assistant/chroma_db"
EMBED_MODEL = "paraphrase-multilingual-MiniLM-L12-v2"
CHUNK_SIZE = 400
CHUNK_OVERLAP = 80
model = SentenceTransformer(EMBED_MODEL)
client = chromadb.PersistentClient(path=str(CHROMA_PATH))
collection = client.get_or_create_collection("brain", metadata={"hnsw:space": "cosine"})Chunking by paragraph:
def chunk_text(text: str, size: int = CHUNK_SIZE, overlap: int = CHUNK_OVERLAP) -> list[str]:
paragraphs = re.split(r'\n{2,}', text.strip())
chunks, current = [], ""
for para in paragraphs:
if len(current) + len(para) < size:
current += "\n\n" + para if current else para
else:
if current:
chunks.append(current.strip())
current = para
if current:
chunks.append(current.strip())
return [c for c in chunks if len(c) > 50]Why chunks instead of whole files? LLMs have a limited context window. Searching over 400-token chunks gives better precision than injecting an entire 5,000-word file.
Extracting YAML frontmatter:
import yaml
def parse_frontmatter(content: str) -> tuple[dict, str]:
if content.startswith("---"):
parts = content.split("---", 2)
if len(parts) >= 3:
try:
meta = yaml.safe_load(parts[1]) or {}
return meta, parts[2].strip()
except yaml.YAMLError:
pass
return {}, contentMetadata (type, tags, title) lets you filter searches — for example, searching only within notes of type projet.
Full indexing:
# First-time indexing
python indexer/brain_indexer.py --full
# Incremental update (files modified since the last run)
python indexer/brain_indexer.pyTip: Trigger indexing via
cronor a Git post-commit hook in~/Brain/to keep the index automatically up to date.
3. The FastAPI proxy — the orchestrator
The proxy exposes an OpenAI-compatible API. Open WebUI (or any OpenAI client) sees it as an ordinary LLM, with no idea there's a RAG layer behind it.
# main.py
from fastapi import FastAPI
from openai import AsyncOpenAI
import chromadb
from sentence_transformers import SentenceTransformer
app = FastAPI()
EMBED_MODEL = "paraphrase-multilingual-MiniLM-L12-v2"
embed_model = SentenceTransformer(EMBED_MODEL)
chroma = chromadb.PersistentClient(path=str(CHROMA_PATH))
collection = chroma.get_collection("brain")
groq_client = AsyncOpenAI(
api_key=os.environ["GROQ_API_KEY"],
base_url="https://api.groq.com/openai/v1"
)Retrieval — finding the relevant passages:
def retrieve_context(query: str, n: int = 5) -> str:
query_vec = embed_model.encode(query).tolist()
results = collection.query(query_embeddings=[query_vec], n_results=n)
if not results["documents"] or not results["documents"][0]:
return ""
chunks = []
for doc, meta in zip(results["documents"][0], results["metadatas"][0]):
source = meta.get("source", "")
chunks.append(f"[{source}]\n{doc}")
return "\n\n---\n\n".join(chunks)Injecting into the system prompt:
RAG_SYSTEM_PROMPT = """You are a personal assistant connected to François Aboudou's Brain vault.
Here are relevant excerpts from the knowledge base:
{context}
---
Answer based on these excerpts whenever possible.
If the excerpts aren't relevant, say so clearly."""4. Personas — specialized agents
Instead of a single general-purpose assistant, the proxy exposes several personas as distinct models. Open WebUI lists them in the model selector.
PERSONAS = {
"brain-assistant": {
"model": "llama-3.3-70b-versatile", # via Groq
"system": "General-purpose personal assistant connected to the Brain vault...",
"use_rag": True,
},
"brain-code": {
"model": "llama-3.1-70b-versatile",
"system": "Development expert. Provide precise, typed code with concise explanations...",
"use_rag": True,
},
"brain-think": {
"model": "deepseek-r1-distill-llama-70b", # chained reasoning
"system": "Deep reasoning mode. Analyze complex problems step by step...",
"use_rag": True,
},
"brain-writer": {
"model": "llama-3.3-70b-versatile",
"system": "Writing specialist: articles, devlogs, LinkedIn/Facebook content for African dev audiences...",
"use_rag": True,
},
"brain-fast": {
"model": "mistral:7b", # via local Ollama
"system": "Fast mode. Short, direct answers.",
"use_rag": False,
},
}Intent-based routing — if the user is on brain-assistant (the default persona), the proxy automatically detects intent:
INTENT_ROUTES = [
(["code", "fonction", "bug", "erreur", "typescript", "python"], "brain-code"),
(["pourquoi", "analyse", "stratégie", "compare", "décide"], "brain-think"),
(["écris", "rédige", "article", "linkedin", "post"], "brain-writer"),
]
def _route_by_intent(query: str) -> str:
lower = query.lower()
for keywords, persona in INTENT_ROUTES:
if any(kw in lower for kw in keywords):
return persona
return "brain-assistant"The routing is transparent — the user stays on brain-assistant, and the right model gets called automatically behind the scenes.
Groq offers ultra-fast inference on Llama and Mixtral via API. console.groq.com — a generous free tier for personal use.
5. Groq vs Ollama — which strategy
| Groq | Ollama | |
|---|---|---|
| Models | Llama-3.3-70b, DeepSeek-R1, Mixtral | Mistral, Llama, Gemma... |
| Speed | ~300 tokens/s | ~20-40 tokens/s (local GPU) |
| Cost | Free up to 14k req/day | $0 (local) |
| Offline | No | Yes |
| Privacy | Data sent to Groq | 100% local |
My strategy: Groq for important tasks (code, reasoning, writing), Ollama for quick questions and offline testing.
# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh
ollama pull mistral:7b
# Check
ollama listOllama documentation: ollama.ai/docs — list of available models and GPU configuration.
6. Open WebUI — the interface
Open WebUI is an open-source chat interface compatible with the OpenAI API. It runs in Docker and connects to the proxy.
# docker-compose.yml
services:
proxy:
build: ./proxy
ports: ["11435:11435"]
environment:
- GROQ_API_KEY=${GROQ_API_KEY}
volumes:
- ${HOME}/.local/share/ai-assistant/chroma_db:/app/chroma_db:ro
open-webui:
image: ghcr.io/open-webui/open-webui:main
ports: ["3001:8080"]
environment:
- OPENAI_API_BASE_URL=http://proxy:11435/v1
- OPENAI_API_KEY=brain-local
depends_on: [proxy]
volumes:
- open-webui-data:/app/backend/dataAccess: http://localhost:3001 — a ChatGPT-like interface with a model selector (personas).
# Start it up
hub ai start
# or directly
cd ~/.local/share/ai-assistant && docker compose up -d7. Deployment and updates
File structure:
~/.local/share/ai-assistant/
├── docker-compose.yml
├── proxy/
│ ├── Dockerfile
│ ├── main.py
│ └── requirements.txt
└── indexer/
├── brain_indexer.py
├── requirements.txt
└── venv/
hub ai commands:
hub ai start # docker compose up -d
hub ai stop # docker compose down
hub ai update # reindex the vault
hub ai status # container state + index stats
hub ai logs # proxy logsReindexing after adding notes:
hub ai update
# → python indexer/venv/bin/python indexer/brain_indexer.py
# → 2045 chunks indexed in ~40 seconds8. Questions to test the RAG
These questions check that retrieval is working correctly:
# Retrieval test
"Which projects are currently active in my vault?"
"What have I noted about Moneroo?"
# Persona test
"[brain-code] Write a TypeScript function to validate an African phone number"
"[brain-think] Compare Convex and Supabase for a payment SaaS"
# Language test
"What projects am I working on?" (answer in French, multilingual retrieval)
# Boundary test
"Who won the 2022 World Cup?"
# → should say "I don't have that info in the vault"
9. Tips and common pitfalls
Chunk size affects quality. Too small (< 150 tokens) → context loss. Too large (> 600 tokens) → noise. 400 tokens with 80 of overlap is a good starting point.
Changing the embedding model requires a full reindex. Vectors from all-MiniLM-L6-v2 and paraphrase-multilingual-MiniLM-L12-v2 aren't comparable.
set -euo pipefail at the top of startup scripts. If ChromaDB isn't ready, the proxy should fail cleanly, not start up silently in a broken state.
Structured logging in the proxy.
import logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(message)s"
)
logger = logging.getLogger(__name__)
# In the chat route
logger.info(f"query={query[:60]!r} persona={persona} chunks={len(context)}")Don't commit chroma_db/ to Git. It's a binary directory that changes on every indexing run — heavy and pointless in version control.
echo "chroma_db/" >> ~/.local/share/ai-assistant/.gitignoreFull architecture — recap
Hub CLI (hub ai)
↓
Docker Compose
├── Open WebUI :3001 ← browser interface
└── FastAPI Proxy :11435
├── Embed (multilingual sentence-transformers)
├── Retrieve (ChromaDB cosine similarity)
├── Route (persona + intent detection)
└── Generate
├── Groq API (llama, deepseek — fast, cloud)
└── Ollama (mistral — local, offline)
Indexer (Python)
← ~/Brain/**/*.md
→ 2,045 chunks in ChromaDB
Resources
- SBERT — Sentence Transformers — embedding models, benchmarks, documentation
- ChromaDB — local vector database, getting started
- Groq Console — ultra-fast API, available models
- Ollama — local LLMs, model list
- Open WebUI — open-source chat interface
- FastAPI — async Python framework for APIs
- RAG Survey (arxiv) — state of the art in RAG architectures
→ The Obsidian vault this assistant queries → The CLI that orchestrates all of this from the terminal