Anyone using the code specific models, – how are you prompting them? Are you using any integration into vim emacs or other truly open source and offline text editor/IDE; not electron or proton based? I’ve compiled VS code before, but it is basically useless in that form, and the binary version sends network traffic like crazy.
I’ve downloaded the 13B codellama from huggingface, passed it my NVIDIA 2070 via cuda, and have interfaced either through the terminal or lmstudio.
Usually my prompts include the specific code block and a wordy explanation about what I’m trying to do.
It’s okay, but it’s not as accurate as chatgpt, and tends to repeat itself a lot more.
For editor integration, i just opted for codeium in neovim. It’s a pretty good alternative to copilot imho.
Hugging face have an llm plug-in for code completion in neovim btw!
Oh nice! Got a link for anyone that comes across this? Save me and others a search plz?
EDIT: NM. Got it. Gonna give it a try later.
If you use ollama you can try to use the fork that I am using. This is my config to make it work: https://github.com/Amzd/nvim.config/blob/main/lua/plugins/llm.lua
Nice. Thanks. I’ll save this post in case I use ollama in the future. Right now I use a codellama model and a mythomax model, but am not running them via a localhost server, just outputted in the terminal or LMStudio.
This looks interesting though. Thanks!
I don’t like to sound like a broken clock, but all the llama models have restrictions on their use that mean they aren’t open source.
And they don’t provide the source… So it’s neither open nor source. I get why and how Meta tries to make themselves look better. And I’m grateful for having access to such models. But I think words have meanings and journalists should do better than repeat that phrasing and help watering down the meaning of ‘open source’. (Which technically doesn’t mean free or without restrictions, but is often used synonymously.)
Don’t they provide the source for the code to actually run the model? Otherwise how are people loading it up and running it? Are they shipping executables along with model weights?
