Install Node.js (for Claude Code), Python (for LiteLLM), and Ollama. Ensure you have at least 16 GB RAM for the best local experience. Windows users should prefer WSL2 for smoother setup.
Claude Code is the command-line assistant. Ollama runs AI models on your computer. LiteLLM translates Claude Code requests into a format Ollama understands. OpenRouter is an online service that hosts many AI models.
Claude Code → LiteLLM → Ollama → Local Model
Install Ollama, pull a coding model, install LiteLLM, then point Claude Code to LiteLLM using environment variables. After every command, confirm there are no errors before continuing.
ollama pull qwen2.5-coder:14b
pip install litellm
litellm --model ollama/qwen2.5-coder:14b --port 4000
npm install -g @anthropic-ai/claude-code
Set ANTHROPIC_BASE_URL to your LiteLLM server and ANTHROPIC_API_KEY to any non-empty value for local mode. Launch Claude Code inside your project folder.
export ANTHROPIC_BASE_URL=http://localhost:4000
export ANTHROPIC_API_KEY=ollama
Create an account, generate an API key, choose a free coding model, and update the environment variables to use the OpenRouter endpoint. This option is ideal if your computer cannot run large local models.
export ANTHROPIC_BASE_URL=https://openrouter.ai/api/v1
export ANTHROPIC_API_KEY=YOUR_KEY
export CLAUDE_MODEL=meta-llama/llama-3.3-70b-instruct:free
Invalid API key: verify variables. LiteLLM errors: restart the proxy and confirm the model name. 429 errors: you’ve reached a free-tier limit. Slow responses: the model may still be loading.
API: a way for software to communicate. CLI: command-line interface. Model: the AI brain. Endpoint: the server receiving requests. Proxy: software that forwards requests between two systems.