Running Local LLMs with Andromity

Published: August 08, 2026
Author: Nikunj Dev

Many developers want the power of AI coding agents but are blocked by privacy constraints. Uploading proprietary codebases or private data schemas to external cloud APIs is a non-starter.

In this guide, we show you how to set up Andromity with local LLMs using Ollama for completely private, offline, and cost-free coding assistance.

Prerequisites

To run local models smoothly, we recommend:

Apple Silicon Mac (M1/M2/M3 Pro or Max) or a PC with a dedicated GPU (NVIDIA RTX 3060+ with 12GB+ VRAM).
Ollama installed on your system.

Step 1: Download Ollama and pull a coding model

First, make sure Ollama is installed and running:

BASH

# Pull Qwen 2.5 Coder (excellent for coding tasks)
ollama pull qwen2.5-coder:14b

# Or pull Llama 3.1 Instruct
ollama pull llama3.1:8b

Step 2: Configure Andromity

Create or open your configuration file at ~/.andromity/config.toml and add Ollama as a provider:

TOML

[default]
provider = "ollama"
model    = "ollama/qwen2.5-coder:14b"
mode     = "safe"

[[providers]]
name = "ollama"
type = "ollama"
base_url = "http://localhost:11434"

Step 3: Run private sessions

Open your project folder and run:

BASH

andromity

The TUI will open and connect directly to your local Ollama instance. All reasoning, planning, and file editing happen 100% locally on your own machine. No internet connection required!