Setting up Continue.dev with local models in VS Code

On the route through the local LLM guide, you are now at the practical applications: the moment you connect your local infrastructure to your daily working environment. Once you have determined which components you need, it is time to embed AI support directly into your code editor. For a thorough introduction to why this is valuable for developers, see the overview on connecting a local LLM to VS Code.

Many developers want to benefit from intelligent code completions and chat assistance, but run into privacy objections with cloud-based services such as GitHub Copilot or Cursor. Commercially sensitive code must not leave their own infrastructure because of strict compliance rules. By using open-source extensions such as Continue.dev in combination with a locally running background service, you retain full control over your intellectual property. In this guide we walk step by step through configuring this open-source assistant with locally hosted models.

1. What is Continue.dev, and why run it locally?

Continue.dev is an open-source IDE extension for Visual Studio Code and JetBrains environments that acts as an open alternative to proprietary assistants. Where commercial tools use their own cloud infrastructure and APIs, Continue routes all its requests locally to an endpoint of your choosing. That can be a local server running via Ollama or LM Studio.

The advantages of this architecture are threefold. First, there is no data outflow: your code fragments and prompts are processed on your own GPU or CPU. Second, you are entirely independent of third-party subscription models and internet connections; your local assistant works offline too, on the train or on site. Third, it offers full flexibility in model choice. You can switch between specialized code models such as DeepSeek-Coder, CodeLlama or Qwen-Coder, depending on the programming language you are working in.

There are clear weaknesses and limitations to bear in mind as well. Smaller local models (under 8 billion parameters) are generally less capable than hosted giants such as GPT-4 or Claude 3.5 Sonnet. They sometimes miss the deeper context of an entire project and can hallucinate when faced with complex architectures. Local hosting also demands considerable computing power from your system. To understand the underlying system requirements, we recommend reading the guidelines on what hardware you need to run LLMs locally.

2. Requirements and basic installation of the background service

Before you can install the extension in Visual Studio Code, a local inference engine has to be running as a bridge between the model and your editor. The most stable and straightforward way to set this up is via Ollama. If you work on an Apple system, you can consult the installation steps in the manual on installing Ollama and running your first local model on macOS.

Make sure Ollama is running in the background and listening on the default port (`http://localhost:11434`). Next, you need to download at least one suitable model optimized for code tasks. Good choices here are smaller, fast variants that respond quickly to inline completions (tab completes), plus larger variants for the chat interface in the sidebar.

# Download een snel model voor autocomplete
ollama pull qwen2.5-coder:1.5b-base

# Download een krachtig model voor de chat-assistent
ollama pull qwen2.5-coder:7b-instruct

Using a separate model for autocomplete (small and fast, such as 1.5B parameters) and for chat (larger, such as 7B or 14B parameters) is essential to a smooth user experience. If you use a model that is too large for automatic completions while typing, the editor will slow down noticeably.

3. Installing the Continue extension in VS Code

Open Visual Studio Code and navigate to the extensions tab on the left (or use the shortcut `Ctrl+Shift+X` on Windows/Linux and `Cmd+Shift+X` on macOS). Search the marketplace for the package named Continue and click install. After a successful installation, a new icon with a circular logo appears in the left or right menu bar of your editor.

Clicking this icon opens the Continue chat interface in the sidebar. By default the extension will try to connect to a cloud service or start a configuration wizard. To retain full control and keep all data local, we skip the cloud sign-in options and configure the configuration file manually.

Continue's configuration file lives in a hidden folder on your computer, usually at the following location:

• macOS / Linux: ~/.continue/config.json
• Windows: %USERPROFILE%\.continue\config.json

4. Setting up the configuration file (config.json)

The heart of Continue is the JSON configuration file. With it you determine exactly which model is addressed for chat (`models`), which model is used for automatic coding (`tabAutocompleteModel`), and how the systems respond to one another.

Open the file `config.json` in your editor and replace its contents with the following basic configuration. This configuration uses the Qwen2.5-Coder models you have just downloaded via your local Ollama instance:

{
  "models": [
    {
      "title": "Qwen 2.5 Coder 7B Instruct",
      "provider": "ollama",
      "model": "qwen2.5-coder:7b-instruct"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Qwen 2.5 Coder 1.5B",
    "provider": "ollama",
    "model": "qwen2.5-coder:1.5b-base"
  },
  "customCommands": [
    {
      "name": "test",
      "prompt": "Schrijf unit tests voor de geselecteerde code in dezelfde programmeertaal."
    },
    {
      "name": "explain",
      "prompt": "Leg uit wat deze code doet en wijs mogelijke prestatieknelpunten of bugs aan."
    }
  ]
}

Save the file. Continue loads the settings immediately. Should a connection error occur, check whether Ollama is actually running and whether the model names in the JSON file match exactly the names you installed locally via the terminal (`ollama list`).

5. Using the features: chat, inline edit and tab completion

Now that installation and configuration are complete, you can start testing the assistant's various capabilities within your daily workflow. Continue offers three primary modes of interaction:

• Tab completion (autofill): As you type, the small model (1.5B) predicts the next lines of code. You accept a suggestion simply by pressing the Tabkey.
• Inline edit (Ctrl+I / Cmd+I): Select a piece of code and press this shortcut, and a small input field appears in which you can ask to refactor the code, translate it into another language or fix errors.
• Chat sidebar (Ctrl+L / Cmd+L): For general questions, generating new functions from scratch, or searching the codebase, you use the sidebar. Here you can also attach files by typing `@filename` in your prompt.

Within the broader context of AI tooling, it helps to know how this local extension compares to wider market developments and ready-made cloud environments. You will find a broader overview in the comparison on AI coding assistants compared: an overview of IDE tools.

6. Privacy, data isolation and GDPR compliance

The main argument for setting up Continue with local models rather than opting for convenient cloud services is data protection. Because all data flows stay within your own local network or on your own machine, you comply directly with strict legislation such as the General Data Protection Regulation (GDPR). No trade secrets, personal keys (API keys) or intellectual property are shared with external tech giants.

It is important to realize, however, that local security does not automatically make your computer immune to other vulnerabilities. If you work with sensitive customer data or medical records, you must ensure that your local machine does not inadvertently store log files in public folders, and that the extension's debug functions do not write data to telemetry servers. By default Continue is set to minimize telemetry, but you can disable this entirely in the configuration file by adding the option `"allowAnonymousTelemetry": false`.

7. Common pitfalls and optimization tips

Running a code assistant locally brings technical challenges. A common pitfall is memory overload (out-of-memory). If your system has too little VRAM on the graphics card, Ollama will automatically fall back to regular working memory (RAM) and in some cases even to slow SSD swap. This makes tab completion respond more slowly than you type, which defeats the purpose of the assistant.

To solve this you can use quantized models. Choosing a Q4 or Q5 quantization considerably reduces the model's memory footprint while the quality of code generation stays practically the same. Another tip is to limit the context length in your configuration if you notice memory filling up. By opting for targeted instructions rather than loading your entire repository at once, performance stays optimal. Setting such parameters precisely is delicate work, and ultimately determines whether a local setup keeps running stably during intensive programming sessions.

8. Extended evaluation and measurement methods for local code assistants

Measuring the performance of a local code assistant objectively calls for a structured approach. Because local hardware resources are finite, it is wise to measure performance indicators such as tokens per second (t/s) and time-to-first-token structurally while generating large blocks of code. You can check these values by analyzing Ollama's log data in the terminal, where statistics on processing speed per second are shown after every prompt.

When you notice response times climbing above acceptable thresholds — generally, fewer than fifteen tokens per second is undesirable for a fluid chat experience — you can experiment with alternative quantization levels or restrict the context windows. Weighing these performance factors carefully keeps your development environment fast and responsive, even when working on hardware with limited capacity.