treesitter missing tests in summary/status

[conermurphy]

First of all I appreciate this is a bit of a niche configuration but wanted to raise an issue to see if anyone had any advice for fixing it as I tried looking into it but was unsuccessful.

In a project I work in, we have a series of tests that test different API endpoints and in these test suites we have multiple describe blocks, one of which is returned from a function that conditionally runs tests (see example below). The issue is that as far as I can tell treesitter will only parse the text in the file itself, it won't evaluate any functions in the file to see what they return and whether that can be a test or not which leads to tests being missed in the discovery window / test statuses in the gutter.

Example implementation below:

// foo.test.js
import {functionReturnsTests} from './test-generator.js';

describe('Foo Test', () => {
   describe('Describe One', () => {
        it('runs some test', () => {});
    });
    
    functionReturnsTests({runTestOne: true, runTestTwo: false});
}

// test-generator.js

function testOne() {
   return  it('runs a test', () => {});
}

function testTwo() {
   return  it('runs a second test', () => {});
}

export function functionReturnsTests({runTestOne, runTestTwo}) {
   return describe('Describe Two', () => {
       if (runTestOne) testOne();
       if (runTestTwo) testTwo();
   });
}

In this example I would expect the discovery panel to look like the below.

- foo.test.js
   - Foo Test
       -  Describe One
          - runs some test
       - Describe Two
          - runs a test

But at the moment I'm only seeing the below.

- foo.test.js
   - Foo Test
       -  Describe One
          - runs some test

As mentioned I did some digging into this to try see what the issue could be and found that the tests are being run by neotest as when I checked the output files in the var directory I could see the results of the tests from the imported function. These tests are also correctly found by the jest CLI which is why I imagine they run correctly through neotest so it appears the issue is just in the project summary / discovery as well as in the pass/fail/in-progress symbols in the gutter.

It's worth noting that these tests are correctly identified and ran with status symbols in VSCode with the Jest extension so I believe this issue is related to neotest.

Someone more experienced with Neovim/treesitter might have a better idea of how to fix this but my original thinking was to somehow build the tests file out into a single file that contains all of the imported code and then run treesitter against that to correctly identify the imported code but I imagine this could get quite time consuming / costly when run at scale which isn't ideal.

[MisanthropicBit]

@conermurphy Thanks for submitting an issue 🙂

FWIW, the main branch now partially supports this if you run the whole file and just the namespace (describe block) but as treesitter is used internally to scan files and is the backbone of how neotest works, I can't generate the correct summary tree unless the file or namespace is run first (currently not happening on the main branch). I believe this is because the two external tests are not classified as parametric/runtime tests in the current implementation (this generally works by having jest list all tests and generating new tree nodes as required based on that info).

I could instead generate new nodes for all results that aren't necessarily parametric/runtime tests if they don't exist in the tree. That should generate them.

The issue is that as far as I can tell treesitter will only parse the text in the file itself, it won't evaluate any functions in the file to see what they return and whether that can be a test or not which leads to tests being missed in the discovery window / test statuses in the gutter.

That is correct. Treesitter also cannot evaluate anything as it is "just" a parser.

It's worth noting that these tests are correctly identified and ran with status symbols in VSCode with the Jest extension so I believe this issue is related to neotest.

Are you saying that you run the equivalent of "nearest test" in vscode or just that it lists the tests in some test overview? I'm not a vscode user and haven't tried that plugin but it seems like it could work, so I would greatly appreciate if you had time to dig into how they make it work if you can.

my original thinking was to somehow build the tests file out into a single file that contains all of the imported code and then run treesitter against that to correctly identify the imported code

That won't be possible since it would require either blindly stitching the files together (prone to break) or evaluating the code with some interpreter.

[MisanthropicBit]

Closed due to inactivity. Feel free to reopen.