How to Use a Free Local AI in VS Code with Ollama and Continue

In short: install Ollama, run ollama pull qwen2.5-coder:7b, install the Continue extension, and point ~/.continue/config.yaml at Ollama.

Why use a local AI

Some code is private: a bank's code, client data, or code under NDA. A local model keeps it all on your laptop. It also works offline and costs nothing per request. The trade-off: small local models are slower and weaker than big cloud models.

What you need

• Ollama: a free tool that downloads and runs AI models locally. It serves them at http://localhost:11434.
• A coding model: for example qwen2.5-coder:7b for chat, qwen2.5-coder:1.5b for fast autocomplete.
• Continue: an open-source extension for VS Code and JetBrains that talks to Ollama.
• Hardware: at least 8 GB RAM (16 GB+ better) and about 10 GB free disk. A GPU makes it much faster.

How it connects

Install steps

1. Install Ollama. Windows/Mac: download from ollama.com. Linux: curl -fsSL https://ollama.com/install.sh | sh.
2. Check it: ollama --version.
3. Make sure it is running: curl http://localhost:11434 should print Ollama is running. If not, run ollama serve.
4. Pull the models (use pull, not run): ollama pull qwen2.5-coder:7b and ollama pull qwen2.5-coder:1.5b.
5. Check: ollama list shows both, with the exact tags.
6. Install Continue: in VS Code press Ctrl+Shift+X, search Continue, install. Or: code --install-extension Continue.continue.
7. Open the Continue panel from the Activity Bar.

ollama --version
curl http://localhost:11434
# -> Ollama is running
ollama pull qwen2.5-coder:7b
ollama pull qwen2.5-coder:1.5b
ollama list

Configuration: config.yaml

Open Continue's config (gear icon, or the file ~/.continue/config.yaml). The model name must match ollama list exactly.

name: Local Config
version: 0.0.1
schema: v1
models:
  - name: Qwen2.5-Coder 7B
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit
      - apply
  - name: Qwen2.5-Coder 1.5B
    provider: ollama
    model: qwen2.5-coder:1.5b
    roles:
      - autocomplete

Settings

1. After editing, click Reload config in Continue's config selector.
2. Autodetect: instead of listing models, you can set model: AUTODETECT with provider: ollama; Continue lists what you pulled.
3. “Model requires more system memory” error: add defaultCompletionOptions: contextLength: 2048 under the model, or use a smaller model.
4. Agent mode needs tool use. Small local models are weak at it; use chat and edit for best results.
5. Another PC runs Ollama? set apiBase to that machine's address.

First test: a small function

A correct answer looks like this. We tested this version:

def is_palindrome(s: str) -> bool:
    """True if s reads the same both ways, ignoring case and spaces."""
    clean = "".join(s.lower().split())
    return clean == clean[::-1]

from palindrome import is_palindrome

def test_yes():
    assert is_palindrome("Never odd or even")
def test_no():
    assert not is_palindrome("Pune")
def test_empty():
    assert is_palindrome("")

pytest -q
# -> 3 passed

Small models make more mistakes than big ones. Always run the test.

Quick quiz

Save tokens and money

Local models cost no tokens, so use them for practice, boilerplate and private code. Save paid cloud models for hard problems. Keep the context length small for speed.

Save these cards