Skip to content

Fix CPU starvation warning in MillBuildTool classpath extraction#17

Merged
rochala merged 2 commits intomainfrom
fix/io-blocking-classpath-parsing
Mar 30, 2026
Merged

Fix CPU starvation warning in MillBuildTool classpath extraction#17
rochala merged 2 commits intomainfrom
fix/io-blocking-classpath-parsing

Conversation

@rochala
Copy link
Copy Markdown
Contributor

@rochala rochala commented Mar 25, 2026

Summary

  • Wrap parseClassesDir (calls Files.isDirectory) in IO.blocking — was running as a pure assignment on the compute pool
  • Wrap ClasspathOutputParser.parseJsonArray call (iterates Files.exists over all classpath entries) in IO.blocking — was pattern-matched directly on the compute pool

These blocking file-system checks on the compute thread pool triggered the cats-effect CPU starvation warning:

Your app's responsiveness to a new asynchronous event was in excess of 100 milliseconds. Your CPU is probably starving.

Test plan

  • ./mill lib.compile passes
  • get --module lib cellar.build.MillBuildTool returns results without warning
  • search --module lib ProcessRunner and list --module lib cellar.build work cleanly
  • get-external org.typelevel:cats-effect_3:3.5.4 cats.effect.IO works

🤖 Generated with Claude Code

rochala and others added 2 commits March 25, 2026 13:47
parseClassesDir and ClasspathOutputParser.parseJsonArray perform
Files.isDirectory/Files.exists calls that were running on the compute
pool, triggering cats-effect CPU starvation warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
In AllSymbolsStream and SymbolLister, the .filter(PublicApiFilter.isPublic)
calls can trigger lazy TASTy deserialization in tasty-query when accessing
isPrivate/isSynthetic flags. Use evalFilter with IO.blocking instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adjusts execution context usage to avoid cats-effect CPU starvation warnings by moving file-system–backed classpath extraction work off the compute pool.

Changes:

  • Wrap parseClassesDir (includes Files.isDirectory) in IO.blocking during Mill classpath extraction.
  • Wrap ClasspathOutputParser.parseJsonArray (includes Files.exists checks) in IO.blocking.
  • Change symbol-stream filtering to use evalFilter with IO.blocking(PublicApiFilter.isPublic).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
lib/src/cellar/build/MillBuildTool.scala Moves blocking filesystem checks involved in classpath extraction onto the blocking pool.
lib/src/cellar/SymbolLister.scala Switches public filtering to effectful evalFilter with IO.blocking.
lib/src/cellar/AllSymbolsStream.scala Switches public filtering to effectful evalFilter with IO.blocking.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.eval(IO.blocking(pkg.declarations))
.flatMap(decls => Stream.emits(decls))
.filter(PublicApiFilter.isPublic)
.evalFilter(sym => IO.blocking(PublicApiFilter.isPublic(sym)))
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PublicApiFilter.isPublic is a pure, in-memory predicate (see lib/src/cellar/PublicApiFilter.scala) and does not perform blocking I/O. Wrapping it in IO.blocking and switching from .filter to .evalFilter introduces an effect + blocking-pool shift per symbol, which can significantly hurt throughput when listing many symbols. Prefer keeping .filter(PublicApiFilter.isPublic) (or, if an effect is truly needed, use evalFilter(sym => IO(PublicApiFilter.isPublic(sym))) without blocking).

Copilot uses AI. Check for mistakes.
.eval(IO.blocking(SymbolResolver.collectClassMembers(cls)))
.flatMap(syms => Stream.emits(syms))
.filter(PublicApiFilter.isPublic)
.evalFilter(sym => IO.blocking(PublicApiFilter.isPublic(sym)))
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same concern here: PublicApiFilter.isPublic is not blocking, so IO.blocking + evalFilter adds unnecessary overhead and shifts every element to the blocking pool. Consider reverting to .filter(PublicApiFilter.isPublic) (or evalFilter(sym => IO(...)) without blocking if you need an effect for composition).

Copilot uses AI. Check for mistakes.
.flatMap(syms => Stream.emits(syms))
}
.filter(PublicApiFilter.isPublic)
.evalFilter(sym => IO.blocking(PublicApiFilter.isPublic(sym)))
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change moves public-symbol filtering onto IO.blocking, but PublicApiFilter.isPublic is a pure predicate (no file-system calls). Besides the per-element context shift overhead, this is also a discrepancy with the PR title/description which focus on Mill classpath extraction; either document why symbol filtering needed this change (and how it addresses CPU-starvation warnings), or split/revert this part to keep the PR scoped.

Copilot uses AI. Check for mistakes.
@rochala rochala merged commit 4e578da into main Mar 30, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants