TL;DR

I built a neovim plugin that runs jest when a test file is saved and displays the results directly next to the test in neovim.

If you want to see the code and try it out for yourself, you can go to my jest.nvim repo.

Introduction

Switching to neovim has been one of the best changes I've made to my workflow.

Writing code in neovim is more efficient. Since it's a modal editor, you rarely have to leave the keyboard while you're working. It's also more fun in my experience.

Having fun when you code is essential if you want to be doing this long term.

Recently, I've been learning lua.

And what is lua and how does it relate to neovim?

Lua is a powerful, efficient, lightweight, embeddable scripting language.

My main experience with lua has been with setting up my neovim configuration in my dotfiles. Dotfiles are a set of files that define your configuration for different tools. For example, I have dotfiles for neovim, tmux, zsh and others.

With dotfiles, you can quickly set up your personalized environment on any machine.

Hacking away at my neovim configuration has been interesting, but after a while I wanted to do something more. I thought it would be a good idea to try building a neovim plugin. It's still lua and it should provide me with some value in my daily workflow.

It's a win win!

So what kind of plugin should I build?

I frequently work on frontend code and rely on jest as my test framework. Normally I have one tmux pane showing neovim and another with jest running in watch mode. Every time I save a file, jest reruns the test and I get immediate feedback.

There's nothing wrong with that workflow.

The feedback loop is tight. The only tests that run are the ones I care about, so things are fast. And it's easy to parse through the results.

That said, I thought I could make my workflow better by using neovim.

I could execute the tests on file save and report that information directly in the editor. If a test failed I could provide additional information next to the relevant test. This ties the error output to the test case. I also don't need another tmux pane open while I'm writing tests.

Overall it's a more enjoyable workflow.

Here's what I learned...

Key Concepts

For this to work, I had to learn about a few different neovim apis to accomplish the following:

• Create a neovim auto command to start a process
• Use a file api to find the correct directory to start the tests
• Run a job (i.e. jest) from inside of neovim
• Take the results and create diagnostic information to display in neovim
• Use deferred functions to show a loader while jest is running

Autocmds

An autocmd is exactly what it sounds like.

It's a way to execute a command (i.e. run some code) automatically when some event happens. For example, I could format a file when I save (think prettier) or lazy load a plugin when I enter (open) a file. Autocmds make it easy to set up this kind of logic with very little code.

In this case, I want to execute jest tests when I write (save) a file.

This is what that looks like:

vim.api.nvim_create_autocmd("BufWritePost", {
  group = vim.api.nvim_create_augroup("group_name", {clear = true}),
  pattern = "*.spec.js",
  callback = function()
    -- Code to run when we save the file
  end
})

There's not that much code above, but there's a lot happening.

The nvim_create_autocmd takes an event type as the first argument. In this case it's the BufWritePost event which neovim fires after you save a file. The second argument is a table which has a group, pattern and callback.

Let's take a look at each in a bit more detail.

group

A group is a way to categorize related autocmds.

You'll notice that an optional clear = true configuration is passed as a second argument to the group. This tells neovim to clear out any existing autocmds if we try to create this group again at some later point.

This can be useful for a couple of reasons:

Rapid Development

It's common to work on a neovim file that has autocmds.

In that case, you'll want to test the changes by sourcing (re-loading) the file. If you source the file without closing neovim, duplicate autocmds are created. To avoid that, you can set clear = true so that neovim clears out the old autocmds as you're developing.

Lazy Loading Groups

If you lazy load a group when you open a file with a given file type (e.g. *.js) then you might run into this issue.

In that case, that group is applied every time you open anything with that file type. I actually ran into this exact problem in my neovim configuration. I lazy loaded an autocmd group that formatted JavaScript files (i.e. ran prettier). Every time I opened a new JavaScript file, neovim sourced the file, added the group, and created a duplicate autocmd.

This led to a huge performance drop after opening a few files. At some point, neovim was trying to call prettier 20+ times.

Here's the code that was causing the problem. That line sources another file which sets up the autocmd to format my JavaScript files. The syntax might be confusing because it's vimscript instead of lua, but the relevant part is the autocmd!. That line does the same thing as clear = true from above.

The fix was easy, but if you're not familiar with autocmds then it can be painful trying to debug what's happening.

pattern

The pattern determines which files the autocmd should apply to. In this case, I only want to run jest if I save a test file. It doesn't make sense to run it for other files.

The pattern above is set to run against *.spec.js files (a common naming convention for jest tests).

callback

This is the code that we run when neovim triggers the autocmd. Eventually this will have the logic to execute the jest test.

File API

Great!

With the autocmd set up, now we need to actually run jest when the command is executed.

Before we get to that though, there are a couple of problems. The first is that jest is typically installed as a project dependency, so it's not guaranteed to be available globally. In other words, I can't run jest <my_test> directly from the command line.

To get around this, what we can do is find the root directory for the current open file. For JavaScript projects, it's a safe bet that the root directory is the closest parent directory that has a package.json file. You could also look for a .git folder if you wanted to be more generic and had version control set up.

For now let's use package.json.

To get this to work with a clean implementation, I actually had to upgrade my neovim instance to the nightly build instead of the stable build. The reason for that is because the neovim team is working on a new file api that can be used to do exactly what we need.

This is how you would use the new API:

local find_root_dir = function()
  local full_path = vim.fn.expand("%:p")
  return vim.fs.dirname(vim.fs.find({"package.json"} {path = full_path, upward = true})[1])
end

That's it.

• vim.fn.expand is getting the full path for the current open file
• vim.fs.find is searching up the file tree from the current file for a directory with a package.json file
• vim.fs.dirname is returning the path to the directory that was found (i.e. the root directory)

Running a job

We have our autocmd.

We have our root directory.

Now time for the fun part. All we need to do is execute jest from this root directory and then capture the results.

Here's how to do that:

-- Another command you could run: "npm t --"
local command = "./node_modules/jest/bin/jest.js"

-- We get this from the previous step
local root_dir = ""

local append_data = function()
-- Handle jest output
end

vim.fn.jobstart(command, {
  cwd = root_dir,
  stdout_buffered = true,
  on_stdout = append_data,
  on_stderr = append_data
})

Again. We just need a little bit of code to make things work.

You can call vim.fn.jobstart with the command you want to run. Then pass a table with the directory (cwd) to execute the command from and handlers for any output of the job.

Build and Display Diagnostics

Before we dig into this section, let's talk about diagnostics.

Here's what the neovim documentation on diagnostics has to say about it:

Nvim provides a framework for displaying errors or warnings from external tools, otherwise known as "diagnostics". These diagnostics can come from a variety of sources, such as linters or LSP servers. The diagnostic framework is an extension to existing error handling functionality such as the quickfix list.

In other words, diagnostics give us a way to report on the results of our jest tests. We can keep track of the status of a test (pass or fail) and show diagnostic information for specific lines. We'll use this to show a ✓ or a ✕ on the line next to a given test's title.

In the case of errors, we'll go a step further and provide information about why a test failed. We're also going to run jest with the --json and --testLocationInResults flags so that we can easily consume the output.

Great! So here's what that looks like:

local ns = vim.api.nvim_create_namespace("namespace_name")
local bufnr = vim.api.nvim_get_current_buf()

local append_data = function(_, data)
  local parsed_data = parse_json(data)
  for _, result in ipairs(parsed_data) do
    local result_icon = result.passed and "✓" or "✕"
    local line = result.test.line
    vim.api.nvim_buf_set_extmark(bufnr, ns, line, 0, {virt_text = {result_icon}})

    if not result.passed then
      local diagnostic_structure = {
        bufnr = bufnr,
        lnum = line,
        end_lnum = line,
        col = 0,
        message = result.message,
        source = "jest.nvim diagnostics"
      }

      vim.diagnostic.set(ns, bufnr, diagnostics, {virtual_text = false})
    end
  end
end

There's quite a bit going on this time.

The code handles creating diagnostic structures and attaching them to a neovim buffer. Before we talk about the relevant neovim APIs, let's discuss buffers and namespaces.

buffers

Buffers are in-memory text representations of a file.

When you open a file in (neo)vim, the contents of the file are stored in a buffer. Any edits made update the buffer without affecting the actual file. The file isn't affected until you write (save) the file.

Neovim uses a unique number to represent each buffer. The buffer number is constant for the life of the neovim session.

namespaces

Namespaces are a way to group buffer highlights and virtual text.

Both buffer highlights and virtual text are visual markings that aren't really a part of your file. For example, your editor might add an underline to a variable to signify it's unused. The editor might go even further and add virtual text on the line to explain the problem.

Neither of these are really part of your file.

Namespaces group this extra information and provide APIs to work with the items in the group. You can do things like clear everything within a namespace. By grouping this information, you can work with each item in the group as a collective.

With buffer numbers and namespaces we can start creating diagnostic information.

The relevant APIs we're going to look at are the following:

vim.api.nvim_buf_set_extmark

This is a low level API to create an extended mark.

Extended marks are used to represent buffer annotations. These annotations track text changes within a buffer. In this case, we're marking each line that has a test title with a success or failure icon.

We also make sure to turn on virtual text for the extended mark so that you can see the icon in the editor.

vim.diagnostic.set

This is a standard API to create diagnostic information.

Internally this is going to use the extmark system as well. So why use the diagnostics API instead of extmarks? The benefit is that you can hook into the other diagnostic APIs. For example, you can use the diagnostic goto_next command to jump to the next diagnostic message in a file.

And we're done with the core functionality.

At this point, you can save a test file, run jest and report on the results.

This is great! But, I wanted to take it a step further and provide some visual indicator that jest is actually running. That way you know something is happening when you save.

Deferred Functions and Loaders

A visual indicator is critical to know that some action is happening in the background.

I decided to keep things simple and show the word Jest with a little dot loader. It uses a recursive function with a deferred wrapper to make it run every half a second.

Here's a simplified version of that code:

function update()
  if done then return end

  vim.api.nvim_buf_set_lines(bufnr, 0, -1, false, {"Jest: " .. current_frame})

 current_frame = get_next_frame()

  vim.defer_fn(function()
      update()
  end, 500)
end

This code does the following:

1. Check if jest is done processing and return early
2. Get the current loader frame and update a loader buffer
3. Set the loader frame to the next one in the list
4. Recursively call itself using vim.defer_fn to cause a 500 ms delay

Conclusion

I built this jest neovim plugin to make my testing workflow better.

I didn't end up writing that much code to get things working. That said, there were quite a few concepts I needed to wrap my head around to understand everything. So far the plugin has made testing things locally a bit faster and more enjoyable.

Disclaimer

In the spirit of full transparency, I didn't come up with the idea for this my self. I walked through this youtube video by TJ DeVries who wrote something similar for go.

I'd recommend giving it a watch.