How to Run Obsidian Copilot Locally With Ollama for Private Q&A
Running Obsidian Copilot against a local Ollama model means your vault queries never touch a cloud server. You install Ollama, pull a model, point the plugin at localhost:11434, and pick an embedding model for Vault QA — all offline after the initial downloads.
The tradeoff is hardware. Local models are slower and less capable than GPT-4 class APIs, so this is a privacy decision as much as a quality one. Below is what actually runs on 16GB and 32GB machines.
Why go local at all?
Three reasons show up again and again: client confidentiality, working offline on a plane, and predictable cost. A consultant handling contracts cannot paste them into a hosted endpoint without a data agreement. A local model sidesteps that entirely.
The catch: you are managing model files, embedding dimensions, and VRAM. It is not a one-click setup like pasting an OpenAI key.
What do you need installed?
- Install Ollama for Windows, macOS, or Linux and confirm the service is running.
- Pull a chat model: ollama pull llama3.1:8b for a 16GB machine, or qwen2.5:14b if you have 32GB.
- Pull an embedding model: ollama pull nomic-embed-text, which the plugin uses for indexing.
- In Obsidian Copilot settings, choose Ollama as provider and set the base URL to http://localhost:11434.
- Select your chat model and set the embedding model to nomic-embed-text.
Run a test message in the sidebar. If it hangs, Ollama usually is not started, or the model tag is misspelled.
Which model should you pick?
| Model | RAM/VRAM | Best for |
|---|---|---|
| llama3.1:8b | ~6GB | General Q&A on 16GB machines |
| qwen2.5:14b | ~10GB | Better reasoning if you have headroom |
| phi3:mini | ~2GB | Older laptops, fast drafts |
The 8B tier is the sweet spot for vault Q&A. Smaller models miss nuance in long notes; larger ones make your fan spin without a huge accuracy jump for this use case.
How does local indexing differ from cloud?
In cloud mode, embeddings are generated by the provider and the vectors live in your vault folder. Locally, nomic-embed-text runs on your CPU or GPU and produces the same kind of vector store, still stored as a local file.
Re-index after switching embedding models. Vectors from different models are not compatible, and mixing them silently wrecks retrieval quality — you will get confident answers about the wrong notes.
What are the real limitations?
An 8B model will not summarize a 3,000-word research note as cleanly as a frontier API. It forgets context in long threads and sometimes invents a citation to a note that does not exist. Verify anything it claims you wrote.
Ollama also holds RAM between calls, so a 14B model plus Obsidian plus your browser on a 16GB laptop will swap and stutter. Match model size to free memory, not to the biggest number that downloads.
Who is the local setup for?
Anyone with confidential notes, or who works offline, and has at least 16GB of RAM. If you just want sharper answers and do not care about privacy, a hosted key still wins on quality. Local is the privacy-first path, not the best-answers path.
Quick Summary
- Obsidian Copilot works fully offline when paired with Ollama at localhost:11434.
- Pull a chat model plus nomic-embed-text for embeddings; re-index after any embedding model change.
- llama3.1:8b fits a 16GB machine; qwen2.5:14b needs around 10GB of VRAM or RAM.
- Local models are private but weaker than hosted APIs — verify every claim they make about your notes.
