Models & Providers

Andromity is built to be provider-agnostic. Under the hood, it uses LiteLLM, allowing it to support virtually every commercial and open-source Large Language Model.

Supported Providers1. Commercial Cloud Providers

Anthropic: Supports the Claude 3.5 Sonnet, Claude 3 Opus, and Claude 3.5 Haiku models.
OpenAI: Supports GPT-4o, GPT-4-turbo, and GPT-3.5-turbo.
Google Gemini: Supports Gemini 1.5 Pro, Gemini 1.5 Flash, and Gemini 2.0.
OpenRouter: Access hundreds of open-source and proprietary models through a single API key.
Groq & NVIDIA NIM: Extremely low-latency hosting of Llama and Mistral open weights models.

2. Local-First / Private Providers

Ollama: Run Qwen 2.5, Llama 3, or Mistral locally on your machine with 100% data privacy and zero API costs.
Llama.cpp: Direct connection to any local GGUF server.

Configuring Providers

API keys are read from environment variables or defined in ~/.andromity/config.toml.

Example configuration:

TOML

[default]
provider = "anthropic"
model    = "claude-sonnet-4-5"

[[providers]]
name = "anthropic"
type = "anthropic"
api_key = "sk-ant-..."

[[providers]]
name = "ollama"
type = "ollama"
base_url = "http://localhost:11434"

Switching Models at Runtime1. In the TUI

Press Ctrl+L or type the /model command:

BASH

/model openai/gpt-4o
/model ollama/qwen2.5:14b

2. From the CLI

BASH

andromity run "fix layout styling" --model google/gemini-1.5-pro

models