Generate plugin spec

[bentsherman]

This PR adds an entrypoint for generating a "plugin index file", which contains a list of plugin definitions (config scopes, functions, etc).

It should be used by the Nextflow Gradle plugin to generate the index file when building/releasing a plugin, and it should be consumed by the plugin registry for use by the language server, registry UI, etc.

---

Note

Introduces JSON plugin spec generation (config scopes, functions, factories, operators) with a CLI entrypoint, adds schema hooks to config AST, and renames the plugin id/version model from PluginSpec to PluginRef across code and tests.

• ADR docs:
  • Add adrs/2025-09-22-plugin-spec.md describing the plugin spec; include ADR template.
• Plugin spec generation:
  • New builders: nextflow.plugin.spec.PluginSpec, FunctionSpec, ConfigSpec to emit JSON definitions for ConfigScope, ConfigOption, ConfigPlaceholderScope, Function, Factory, Operator.
  • Add PluginSpecWriter entrypoint to write spec JSON at build-time.
  • Tests: PluginSpecTest.groovy validating emitted spec structure.
• Core/runtime integration:
  • ConfigNode now carries SchemaNode.Scope (getter/setter) for schema-aware config processing.
• Refactor: Plugin reference model:
  • Rename PluginSpec (id/version pair) to PluginRef across nf-commons (e.g., DefaultPlugins, HttpPluginRepository, PluginUpdater, PluginsFacade, PrefetchUpdateRepository), diagrams, and tests.

^{Written by Cursor Bugbot for commit 8b772f4. This will update automatically on new commits. Configure here.}

[netlify]

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	8b772f4
🔍 Latest deploy log	https://app.netlify.com/projects/nextflow-docs-staging/deploys/68dc1faad77a010008f0f2c3

[pditommaso]

Does not look to have dependency with the nextflow runtime, does it? could make sense to keep it in its own module or even keep it in the gradle plugin source? it would be preferable to have the gradle plugin to no depend on the main nextflow runtime

[bentsherman]

The PluginIndexWriter does depend pretty heavily on runtime classes, mainly around plugin extensions and config schema. This is why I placed it in a separate configuration, so that the main build doesn't depend on the specific Nextflow version

Long-term, I think we'll want to have a module in Nextflow that defines a versioned "runtime API" that is stable across Nextflow versions. For now I'm just trying to make it work

[bentsherman]

• Add ADR with description of plugin schema (JSON)
• Add unit test

[pditommaso]

Would not make sense to create a JSON schema for the definition spec? it could be used to have more a "formal" definition and above all to validate it in the upload

[bentsherman]

Okay I have written a JSON schema for plugin "specs"

We need to store this schema somewhere on GitHub so that a plugin spec can refer to it, for example:

{
    "$schema": "https://raw.githubusercontent.com/nextflow-io/nextflow/master/plugin_schema.json",
    "definitions": [
        // ...
    ]
}

We should probably keep it in a separate repo since it is not tied to the Nextflow version. We could extend https://github.com/nextflow-io/schema-spec to include both input/output schemas and plugin schemas

[pditommaso]

Nice. I'm fine both with a separate repo or keep in nextflow one, in cause the shema should have its own version.

[pditommaso]

@claude review

[claude]

Claude finished @pditommaso's task —— View job

---

Code Review Complete

I've completed a comprehensive review of the plugin spec generation PR. Here are my findings:

Overall Assessment

This is a well-designed and implemented feature that addresses a real need for external systems (language server, registry UI) to understand plugin capabilities without loading them in the JVM. The implementation is clean and follows good practices.

Strengths

✅ Clean Architecture: The separation of concerns between PluginSpec, ConfigSpec, and FunctionSpec is well done
✅ Comprehensive Coverage: Handles config scopes, functions, operators, and factories
✅ Good Documentation: The ADR clearly explains the rationale and design decisions
✅ JSON Schema Reference: Uses versioned schema URL for validation
✅ Test Coverage: Good unit tests with realistic examples
✅ Refactoring Quality: The PluginSpec → PluginRef rename improves naming clarity

Issues to Address

🐛 Critical: Generic Type Detection Bug

Location: ConfigSpec.groovy:93 and FunctionSpec.groovy:61

if( cn.getGenericsTypes() != null ) {

Problem: This condition can be true even for non-generic types that have an empty generics array. The original codebase uses cn.isUsingGenerics() which is more accurate.

Fix: Change to:

if( cn.isUsingGenerics() ) {

🐛 Minor: Missing Child Names in Nested Scopes

Location: ConfigSpec.groovy:77-78

final children = node.children().collect { name, child ->
    fromNode(child, name)  // name is passed correctly
}

Actually, looking at this more carefully, this appears to be working correctly - the name parameter is being passed to fromNode which forwards it appropriately.

📝 ADR Filename Convention

The ADR filename uses the requested YYYY-MM-DD format, which is good. However, consider using YYYYMMDD as suggested in the review comments for consistency.

Minor Observations

1. Schema URL: The schema references https://raw.githubusercontent.com/nextflow-io/schemas/main/plugin/schema.json - ensure this schema exists or will be created
2. Error Handling: Consider adding validation for malformed extension points in PluginSpec.build()
3. Null Safety: The description handling looks robust with proper null checks

Test Quality

The test coverage is excellent with a realistic example that covers:

• Config scopes with nested options
• Multiple function types (Factory, Function, Operator)
• Parameter and return type extraction
• Description handling

Refactoring Assessment

The PluginSpec → PluginRef refactoring is well-executed:

• Maintains backward compatibility in the public API
• Updates all references consistently
• Improves semantic clarity (PluginRef better represents an ID+version pair)
• Test coverage ensures no functionality is broken

Recommendation

Approve with minor fix required: This PR significantly enhances the plugin ecosystem's extensibility. The only blocking issue is the generic type detection bug, which should be fixed before merging.

---

[bentsherman]

@pditommaso ready to merge

[pditommaso]

Cursor does not agree! 😆