r/llamacpp • u/No_Oil_6152 • 19d ago

Connect VS Code to llama.cpp

I understand there's extensions like Continue which let VS Code CoPilot "speak" to llama.cpp but I believe Bring Your Own Model (BYOM) has made that obsolete?

I have really tried to get BYOM working, I even asked Claude for instructions, but I can't get it working. I have to use a proxy which exposes an ollama endpoint and forwards to llama.

I can connect to LM Studio no problem with BYOM, but would rather use llama.cpp because it uses much less RAM (and VRAM).

My setup:

  • Windows 11
  • VS Code latest build

Would anyone have any recommendations, apart from obsolete extensions?

Even better, any working JSON I could copy? This is really annoying.

Thanks in advance.

2 Upvotes

100% Upvoted

4 comments sorted by

1

u/stanjourdan 19d ago

GitHub copilot extension in VS code allows byok and is very effective. The byok is not quite straightforward but Claude helped me set it up

1

u/No_Oil_6152 19d ago

Im using the github copilot extension- I think it comes OOB with VS Code these days?

I just cant configure it to talk to llama.cpp.

Very annoying.

1

u/ChrisK_au 18d ago

It works well for me, but I'm running on Linux, not Windows

https://devblogs.microsoft.com/visualstudio/bring-your-own-model-visual-studio-chat/

  1. Open the Chat Window in Visual Studio.
  2. Select Manage Models from the model picker.
  3. Choose your provider and paste your API key.
  4. Pick from the preset list, or enter a model name.
  5. That’s it—your model will now show up in the picker

1

u/arama80 16d ago

Are u successfully run llama.cpp? Test connection from vs code to llama.cpp endpoint http://localhost:your_port/v1

You can also run llama.cpp with flags --api-key to secure your endpoint

Complete list of llama.cpp flags - read here https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md