tag tests that require network

[teto]

I am trying to run the busted tests in nixpkgs.
I managed to run the test from a devShell (unluckily I first started with busted -l spec/ which didn't work for me ).
I had to sandbox the call with NVIM_APPNAME otherwise the rocks.nvim from ~/.local/share/nvim was getting loaded instead of my checkout.

$ NVIM_APPNAME=toto busted spec
Running healthchecks...●●●●●●●●●●●●●●●                                                   a5.1-nRecursive import detected: /tmp/nix-shell.BxRVdZ/toto.teto/QEnD1a/1/rocks.toml●●●●●●●●●●●●
Recursive import detected: /tmp/nix-shell.BxRVdZ/toto.teto/QEnD1a/6/rocks.tomlkkk●●●●●●●●
foo pinned to version 1.0.0
foo unpinned●
foo pinned to version 1.0.0
foo unpinned●
foo pinned to version 1.0.0
foo unpinned●●●
40 successes / 0 failures / 0 errors / 0 pending : 126.272105 seconds

I find the 2 minutes long. Is that expected ? I wonder how much of that was due to the plenary timeout.

Anyway, as mentioned previously, I am trying to run the tests in nixpkgs, which now fail due to sandboxing and no internet.
With busted, it's possible to provide customized test targets in .busted that match filenames or tags it("Test with #internet tag"). I have identified spec/operations/install_update_spec.lua for now but the feedback loop is long (vim.env.PLENARY_TEST_TIMEOUT = 60000 * 5) . Shouldn't we reduce it and/or stop hardcoding it in the test so it can be customized before running the test ?

[mrcjkb]

I don't think PLENARY_TEST_TIMEOUT is the culprit here. That just tells nvim-nio when to timeout tests that are run in an async context. But they finish as soon as they complete. Our integration tests that use luarocks are just slow because luarocks is slow.

In rocks (the rust package), we have a ROCKS_SKIP_IMPURE_TESTS env variable that we set to true in the nix build environment. Perhaps the simplest solution would be to do that here too.

[teto]

In rocks (the rust package), we have a ROCKS_SKIP_IMPURE_TESTS env variable that we set to true in the nix build environment.

good to know !

Perhaps the simplest solution would be to do that here too.

So to check for an environment variable at the start of test and abort otherwise ?

I think busted can be configured to run different sets of tests via either--exclude filters or tags which is an option that is better when files can mix impure and pure tests.

[teto]

I can run the tests successfully if I remove 6 out of the 7 files in spec/operations, just keeping parser_spec.lua

      rm spec/operations/install_update_spec.lua spec/operations/sync_spec.lua \
        spec/operations/helpers_spec.lua spec/adapter_spec.lua \
        spec/operations/pin_spec.lua spec/operations/bulk_install_spec.lua

don't think PLENARY_TEST_TIMEOUT is the culprit here

when you are in the sandbox, you reach the timeout since network fails, which made the tests extra longer.

[teto]

btw do we run the busted tests in CI?

[mrcjkb]

btw do we run the busted tests in CI?

Yes, but with --option sandbox false.