Category: news

How the Integration Works

You can now run local AI models directly inside Visual Studio Code — without sending your code to the cloud.

Thanks to the integration between GitHub Copilot Chat and Ollama, developers can switch between cloud-based models and locally running LLMs inside the same interface. This sounds like the ideal setup: private, cost-free, and fully under your control. This isn’t a complete replacement for cloud AI yet but it’s a flexible hybrid approach.

Everything happens inside the GitHub Copilot Chat interface in Visual Studio Code — no extra tools or complex setup required. Once Ollama is running locally, its models become available directly in the Copilot Chat model picker. You can choose between:

• Cloud models (like GPT-4-class models)
• Local models running through Ollama

Switching between them is seamless, so you can use:

• local models for privacy-sensitive tasks
• cloud models for more complex reasoning or better performance

There’s no need for external APIs when using local models — they run entirely on your machine via Ollama.

Prerequisites

To use this setup, you’ll need:

• Ollama (v0.18.3 or newer) running locally
• Visual Studio Code (v1.113 or newer)
• GitHub Copilot Chat extension (v0.41.0 or newer)

You also need to be signed in to GitHub Copilot, but a paid subscription is not required. The free tier supports custom model selection.

What You Can Do With It

Once configured, you can use local models inside VS Code for:

• Code generation
• Explaining code
• Debugging issues
• Context-aware suggestions based on your workspace

Limitations

• Inline autocomplete still relies on cloud-based GitHub Copilot models
• Local models can be noticeably slower, especially on mid-range hardware
• Model quality varies, and smaller local models may struggle with complex tasks

So while local models are useful, they don’t fully replace cloud AI in day-to-day development.

When Local Isn’t Enough — BYOK

If your machine can’t handle local models well, there’s a practical fallback: Bring Your Own Key. This lets you connect external model providers via API keys and use them inside the same Copilot interface — effectively offloading computation to the cloud.

If you want access to multiple models without juggling multiple API keys, services like OpenRouter can simplify this by giving you a single key that works across many different models.

This integration it’s about giving developers more control. In practice, local models can feel fast for short responses since there’s no network latency. However, for longer outputs or more complex tasks, cloud models still tend to be faster due to higher token generation speeds and more powerful hardware so that’s the major issue.

You can keep sensitive code local when it matters, avoid unnecessary costs, and still rely on powerful cloud models when/if needed, so,

It depends what you are working on, medium coding tasks, long complex tasks and if you are dealing with Privacy-critical work.

Your real experience will depends on:

Hardware

• CPU
• GPU

Model size

• 7B → fast
• 30B → slower
• 70B → slow (impossible locally)

Context size

• Bigger prompt = slower generation
• Tools/agents/ = MUCH slower

Cold start

• Loading model = seconds delay

https://medium.com/@s.bogeska/vs-code-ollama-integration-287302f7a594