Introduction

Context: HuggingFace has revamped an older plugin to introduce https://github.com/huggingface/llm.nvim plugin for Neovim (see corresponding plugin for VSCode).

Why Neovim: I love VSCode and I use it when I'm focused on a specific project. However, when I'm jumping in and out of multiple code repositories and mostly searching/reading, I prefer the combination of Neovim + LazyVim + LazyGit.

Why LazyVim: I use LazyVim as the configuration for Neovim, since it works out-of-the-box and drastically reduces my yak shaving.

Why CodeLlama: Llama 2 has become the new defacto LLM to get started with. Given that CodeLlama is a specialized model of Llama2, I was eager to try it for autocompletion inside Neovim.

Why HuggingFace plugin: It is easy to get started with using HuggingFace servers for model inference, but you have the option to run your own server.

How-to

If you are using Neovim and not using LazyVim, jump to steps in https://github.com/huggingface/llm.nvim#package-manager instead.

Here is my quick how-to to set up llm.nvim with LazyVim:

Step 1 - Configure LazyVim

Create ~/.config/nvim/lua/plugins/llm.lua with the following code:

-- https://github.com/huggingface/llm.nvim#readme
return {
  {
    "huggingface/llm.nvim",
    opts = {
      -- https://github.com/huggingface/llm.nvim#models
      tokens_to_clear = { "<EOT>" },
      fim = {
        enabled = true,
        prefix = "<PRE> ",
        middle = " <MID>",
        suffix = " <SUF>",
      },
      model = "codellama/CodeLlama-13b-hf",
      context_window = 4096,
      tokenizer = {
        repository = "codellama/CodeLlama-13b-hf",
      },
      accept_keymap = "<c-g>",
      dismiss_keymap = "<c-x>",
    },
  },
}

Content of ~/.config/nvim/lua/plugins/llm.lua

Use ctrl-g to accept the completion and ctrl-x to dismiss the completion (because I could not get <tab> to be accepted by LazyVim, which is the suggestion in the llm.nvim documentation).

Step 2 - Login to HuggingFace

First, generate a new token at https://huggingface.co/settings/tokens (use the name huggingface/llm.nvim )

Second, login in the terminal:

# https://huggingface.co/docs/huggingface_hub/quick-start
pip install -U huggingface_hub

# https://huggingface.co/docs/huggingface_hub/quick-start#login
huggingface-cli login

Step 3 - Done!

Open Neovim, it's ready for llm.nvim & CodeLlama autocompletion.

Update:

~~Because of rate limiting and pricing options, decided to stick with~~ ~~GitHub Copilot~~ ~~for now instead, which~~ ~~LazyVim supports out of the box~~.