How to Set Up Refact.ai as a Private AI Coding Assistant
Refact.ai is an open-source coding assistant you can run yourself, which matters if your code can’t leave your network. It does the usual autocomplete-and-chat job, but the selling point is that the model runs where you decide.
Direct answer: Refact.ai is an open-source AI coding assistant available as a cloud service or a self-hosted server you run on your own hardware, supporting IDE plugins for VS Code and JetBrains; you can point it at a local or your own hosted model for full privacy.
Why self-host a coding assistant?
Compliance and secrets. If your repo contains regulated data or client code you can’t send to a third-party API, a self-hosted assistant is often the only way to use one at all. Refact.ai also lets you fine-tune on your own codebase, which can improve completions on internal frameworks a general model has never seen.
How do I install it?
- Install Docker on a machine with a GPU if you want local inference (a 24GB card handles mid-size models).
- Pull the Refact.ai server image and start it.
- Create an admin account on first run.
- Install the IDE plugin — VS Code or JetBrains — and point it at your server URL.
- Log in from the IDE with the account you created.
Once connected, completions and chat route through your server, not the cloud.
Which model should I run?
| Hardware | Practical model | Notes |
|---|---|---|
| CPU only | Small 7B (slow) | Fine for testing only |
| 16GB GPU | 7B–8B quantized | Good completions |
| 24GB+ GPU | 14B–32B quantized | Best quality locally |
You can also connect an external API if you’re allowed to send prompts out — a middle ground that keeps the tool on-prem while the model is hosted.
How do I fine-tune it on my codebase?
Refact.ai can train on your repositories so completions pick up your naming and patterns. It’s not instant — you point it at repos, it builds a dataset, and you run training. For a large monorepo with unusual conventions, this is where the tool pulls ahead of generic assistants that keep suggesting idioms your team abandoned.
What are the costs and limits?
The software is open source; your cost is hardware and the time to maintain it. Running inference on a GPU 24/7 isn’t free — a mid-tier cloud GPU is hundreds of dollars a month. Small local models also trail frontier models on hard reasoning, so expect weaker results on complicated refactors even if autocomplete feels fine.
Who is Refact.ai for?
Teams with strict data rules and a GPU to spare, and developers who want autocomplete tuned to their own code. If you can use cloud assistants freely, a managed tool needs less babysitting.
Key Takeaways
- Refact.ai is open source and self-hostable, with VS Code and JetBrains plugins.
- A 24GB GPU handles mid-size quantized models; CPU-only is for testing.
- You can fine-tune on internal repos to match your conventions.
- Total cost is hardware plus maintenance, not a subscription.
- Local models trail frontier models on complex reasoning tasks.
