Skip to content

Initial load of multiple home units is slow with lazy component loading #4381

@fendor

Description

@fendor
Collaborator

With a default hie.yaml of

cradle:
  cabal:

and "sessionLoading": "multiComponent", the initial load can be incredibly slow, as HLS starts asking for the GhcSession for each module that's currently open in the editor one by one. cabal needs to reconfigure the project each time the set of units to load changes, which can take a non-trivial amount of time on big projects (for example HLS itself).

We get logs such as these:

...
2024-08-16T08:59:17.665247Z | Debug | cabal exec -v0 -- ghc --print-libdir
2024-08-16T08:59:19.499261Z | Debug | cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2024-08-16T08:59:19.500156Z | Info | cabal --builddir=/home/hugin/.cache/hie-bios/dist-hls-036ebb56a4cc135651b7510a90f3136d v2-repl --with-compiler /home/hugin/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /home/hugin/.cache/hie-bios/ghc-pkg-99f55999495b863a61d4f15d989eaab4 --keep-temp-files --enable-multi-repl /home/hugin/Documents/haskell/hls/ghcide/test/exe/HighlightTests.hs
...
2024-08-16T08:59:21.104638Z | Debug | Resolving dependencies...
2024-08-16T08:59:25.984647Z | Debug | Build profile: -w ghc-9.6.5 -O1
2024-08-16T08:59:25.984774Z | Debug | In order, the following will be built (use -v for more details):
2024-08-16T08:59:25.984835Z | Debug |  - hls-plugin-api-2.9.0.1 (lib) (configuration changed)
2024-08-16T08:59:25.984921Z | Debug |  - ghcide-2.9.0.1 (lib) (configuration changed)
2024-08-16T08:59:25.984989Z | Debug |  - hls-test-utils-2.9.0.1 (lib) (configuration changed)
2024-08-16T08:59:25.985049Z | Debug |  - ghcide-2.9.0.1 (exe:ghcide) (configuration changed)
2024-08-16T08:59:25.985105Z | Debug |  - haskell-language-server-2.9.0.1 (interactive) (test:ghcide-tests) (configuration changed)
...
2024-08-16T09:01:06.465845Z | Info | Load cabal cradle using all files (multi-components)
2024-08-16T09:01:08.269128Z | Debug | cabal exec -v0 -- ghc --print-libdir
2024-08-16T09:01:10.157674Z | Debug | cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2024-08-16T09:01:10.158281Z | Info | cabal --builddir=/home/hugin/.cache/hie-bios/dist-hls-036ebb56a4cc135651b7510a90f3136d v2-repl --with-compiler /home/hugin/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /home/hugin/.cache/hie-bios/ghc-pkg-99f55999495b863a61d4f15d989eaab4 --keep-temp-files --enable-multi-repl /home/hugin/Documents/haskell/hls/ghcide/src/Development/IDE/Plugin/TypeLenses.hs /home/hugin/Documents/haskell/hls/ghcide/test/exe/HighlightTests.hs
...
2024-08-16T09:01:11.794655Z | Debug | Resolving dependencies...
2024-08-16T09:01:16.890247Z | Debug | Build profile: -w ghc-9.6.5 -O1
2024-08-16T09:01:16.890352Z | Debug | In order, the following will be built (use -v for more details):
2024-08-16T09:01:16.890393Z | Debug |  - ghcide-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:01:16.890695Z | Debug |  - hls-test-utils-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:01:16.890832Z | Debug |  - ghcide-2.9.0.1 (interactive) (exe:ghcide) (configuration changed)
2024-08-16T09:01:16.890906Z | Debug |  - haskell-language-server-2.9.0.1 (interactive) (test:ghcide-tests) (configuration changed)
...
2024-08-16T09:02:09.749233Z | Debug | cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2024-08-16T09:02:09.749803Z | Info | cabal --builddir=/home/hugin/.cache/hie-bios/dist-hls-036ebb56a4cc135651b7510a90f3136d v2-repl --with-compiler /home/hugin/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /home/hugin/.cache/hie-bios/ghc-pkg-99f55999495b863a61d4f15d989eaab4 --keep-temp-files --enable-multi-repl /home/hugin/Documents/haskell/hls/hls-plugin-api/src/Ide/Types.hs /home/hugin/Documents/haskell/hls/ghcide/src/Development/IDE/Plugin/TypeLenses.hs /home/hugin/Documents/haskell/hls/ghcide/test/exe/HighlightTests.hs
...
2024-08-16T09:02:16.081585Z | Debug | Build profile: -w ghc-9.6.5 -O1
2024-08-16T09:02:16.081704Z | Debug | In order, the following will be built (use -v for more details):
2024-08-16T09:02:16.081758Z | Debug |  - hls-plugin-api-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:02:16.082405Z | Debug |  - ghcide-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:02:16.082488Z | Debug |  - hls-test-utils-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:02:16.082542Z | Debug |  - ghcide-2.9.0.1 (interactive) (exe:ghcide) (configuration changed)
2024-08-16T09:02:16.082591Z | Debug |  - haskell-language-server-2.9.0.1 (interactive) (test:ghcide-tests) (configuration changed)
...
2024-08-16T08:59:19.500156Z | Info | cabal --builddir=/home/hugin/.cache/hie-bios/dist-hls-036ebb56a4cc135651b7510a90f3136d v2-repl --with-compiler /home/hugin/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /home/hugin/.cache/hie-bios/ghc-pkg-99f55999495b863a61d4f15d989eaab4 --keep-temp-files --enable-multi-repl /home/hugin/Documents/haskell/hls/ghcide/test/exe/HighlightTests.hs
...
2024-08-16T08:59:21.104638Z | Debug | Resolving dependencies...
2024-08-16T08:59:25.984647Z | Debug | Build profile: -w ghc-9.6.5 -O1
2024-08-16T08:59:25.984774Z | Debug | In order, the following will be built (use -v for more details):
2024-08-16T08:59:25.984835Z | Debug |  - hls-plugin-api-2.9.0.1 (lib) (configuration changed)
2024-08-16T08:59:25.984921Z | Debug |  - ghcide-2.9.0.1 (lib) (configuration changed)
2024-08-16T08:59:25.984989Z | Debug |  - hls-test-utils-2.9.0.1 (lib) (configuration changed)
2024-08-16T08:59:25.985049Z | Debug |  - ghcide-2.9.0.1 (exe:ghcide) (configuration changed)
2024-08-16T08:59:25.985105Z | Debug |  - haskell-language-server-2.9.0.1 (interactive) (test:ghcide-tests) (configuration changed)
...
2024-08-16T09:01:06.465845Z | Info | Load cabal cradle using all files (multi-components)
2024-08-16T09:01:08.269128Z | Debug | cabal exec -v0 -- ghc --print-libdir
2024-08-16T09:01:10.157674Z | Debug | cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2024-08-16T09:01:10.158281Z | Info | cabal --builddir=/home/hugin/.cache/hie-bios/dist-hls-036ebb56a4cc135651b7510a90f3136d v2-repl --with-compiler /home/hugin/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /home/hugin/.cache/hie-bios/ghc-pkg-99f55999495b863a61d4f15d989eaab4 --keep-temp-files --enable-multi-repl /home/hugin/Documents/haskell/hls/ghcide/src/Development/IDE/Plugin/TypeLenses.hs /home/hugin/Documents/haskell/hls/ghcide/test/exe/HighlightTests.hs
2024-08-16T09:01:11.794655Z | Debug | Resolving dependencies...
2024-08-16T09:01:16.890247Z | Debug | Build profile: -w ghc-9.6.5 -O1
2024-08-16T09:01:16.890352Z | Debug | In order, the following will be built (use -v for more details):
2024-08-16T09:01:16.890393Z | Debug |  - ghcide-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:01:16.890695Z | Debug |  - hls-test-utils-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:01:16.890832Z | Debug |  - ghcide-2.9.0.1 (interactive) (exe:ghcide) (configuration changed)
2024-08-16T09:01:16.890906Z | Debug |  - haskell-language-server-2.9.0.1 (interactive) (test:ghcide-tests) (configuration changed)
...
2024-08-16T09:02:09.749233Z | Debug | cabal exec -v0 -- ghc -package-env=- -ignore-dot-ghci -e Control.Monad.join (Control.Monad.fmap System.IO.putStr System.Environment.getExecutablePath)
2024-08-16T09:02:09.749803Z | Info | cabal --builddir=/home/hugin/.cache/hie-bios/dist-hls-036ebb56a4cc135651b7510a90f3136d v2-repl --with-compiler /home/hugin/.cache/hie-bios/wrapper-b54f81dea4c0e6d1626911c526bc4e36 --with-hc-pkg /home/hugin/.cache/hie-bios/ghc-pkg-99f55999495b863a61d4f15d989eaab4 --keep-temp-files --enable-multi-repl /home/hugin/Documents/haskell/hls/hls-plugin-api/src/Ide/Types.hs /home/hugin/Documents/haskell/hls/ghcide/src/Development/IDE/Plugin/TypeLenses.hs /home/hugin/Documents/haskell/hls/ghcide/test/exe/HighlightTests.hs
...
2024-08-16T09:02:12.266032Z | Debug | eval: fp /home/hugin/Documents/haskell/hls/ghcide/src/Development/IDE/Plugin/TypeLenses.hs
2024-08-16T09:02:12.266264Z | Debug | Finished: codeLens.config Took: 0.00s
2024-08-16T09:02:16.081585Z | Debug | Build profile: -w ghc-9.6.5 -O1
2024-08-16T09:02:16.081704Z | Debug | In order, the following will be built (use -v for more details):
2024-08-16T09:02:16.081758Z | Debug |  - hls-plugin-api-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:02:16.082405Z | Debug |  - ghcide-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:02:16.082488Z | Debug |  - hls-test-utils-2.9.0.1 (interactive) (lib) (configuration changed)
2024-08-16T09:02:16.082542Z | Debug |  - ghcide-2.9.0.1 (interactive) (exe:ghcide) (configuration changed)
2024-08-16T09:02:16.082591Z | Debug |  - haskell-language-server-2.9.0.1 (interactive) (test:ghcide-tests) (configuration changed)
...

Note, how this takes three minutes.

Potential solution

Load all modules that are currently open immediately if loadStyle == multiComponent.

Activity

soulomoon

soulomoon commented on Aug 24, 2024

@soulomoon
Collaborator

I suppose, we need a batch loader in hie-bois ? So we can generate deps for all the foi files at once.

self-assigned this
on Oct 24, 2024
soulomoon

soulomoon commented on Oct 28, 2024

@soulomoon
Collaborator

After implementing a demo for batch load mode and play with it, I’ve encountered a significant challenge: it’s difficult to effectively distribute errors when consulting cradles, as we’re batch loading a large number of new files. 🤔

fendor

fendor commented on Oct 29, 2024

@fendor
CollaboratorAuthor

Yeah, that makes sense... can't we just display the errors in all files that we failed to load?

soulomoon

soulomoon commented on Oct 29, 2024

@soulomoon
Collaborator

can't we just display the errors in all files that we failed to load?

Yes we can, but the cons is a bit high.
Say we opened x1 .... xn. x1 is the one can't be loaded. But we can load x2 ... xn perfectly.
In batch mode, we are left with x1 ... xn all in loading errors, which is really annoying.

Pertential workaround would be to load these files one by one when batch load failed.
Trade off with load times.

added a commit that references this issue on Nov 2, 2024
58b8b68
hellwolf

hellwolf commented on Feb 3, 2025

@hellwolf

Load all modules that are currently open immediately if loadStyle == multiComponent.

Sorry, it is not clear to me what exactly I should do here?

soulomoon

soulomoon commented on Feb 4, 2025

@soulomoon
Collaborator

Load all modules that are currently open immediately if loadStyle == multiComponent.

Sorry, it is not clear to me what exactly I should do here?

That is to add the “sessionLoading”: “multiComponents” in the configuration and if cabal version support it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

component: ghcidemulti-componentIssues relating to multi-component supporttype: bugSomething isn't right: doesn't work as intended, documentation is missing/outdated, etc..

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    Participants

    @hellwolf@fendor@soulomoon

    Issue actions

      Initial load of multiple home units is slow with lazy component loading · Issue #4381 · haskell/haskell-language-server