Skip to content

Conversation

tadasant
Copy link
Member

@tadasant tadasant commented Sep 29, 2025

Co-authored with Claude Code, but I've closely reviewed/modified/vetted all changes; ready for final review.

Summary

This PR adds LocalTransport and RemoteTransport separation with URL template variable support for remote servers, enabling e.g. multi-tenant deployments with configurable endpoints.

Key additions:

  • LocalTransport (for Packages) and RemoteTransport (for Remotes)
  • URL template variables for remote transports (e.g., {tenant_id} in URLs)
  • Pattern validation ensuring URLs are URL-like while allowing templates
  • Full test coverage via Go validator tests and documentation examples

Problem

The current schema describes URL templating with {curly_braces} for transports but provides no mechanism for Remote contexts to define what those variables should resolve to, making the feature unusable for remote servers.

Referenced issue: #394

  • Users need multi-tenant remote servers with different endpoints per deployment
  • Each tenant/region has its own URL (e.g., us-cell1.example.com, emea-cell1.example.com)
  • Current schema doesn't support parameterized remote URLs

Solution

Schema Architecture

LocalTransport (for Package context):

LocalTransport = StdioTransport | StreamableHttpTransport | SseTransport
  • Used in Package context (non-breaking rename of existing behavior)
  • Variables in {curly_braces} reference parent Package arguments/environment variables
  • Supports all three transport types including stdio

RemoteTransport (for Remote context):

RemoteTransport = (StreamableHttpTransport | SseTransport) + { variables: object }
  • Extends StreamableHttp/Sse via allOf composition - no duplication!
  • Adds variables object for URL templating
  • Only supports streamable-http and sse (stdio not valid for remotes)

Key Changes

1. Schema Updates (openapi.yaml, server.schema.json):

  • Added LocalTransport union type for Package context
  • Added RemoteTransport with allOf composition extending base transports
  • Updated Package to reference LocalTransport
  • Updated remotes to reference RemoteTransport
  • Added URL pattern validation: ^https?://[^\\s]+$ to both StreamableHttp and SSE
  • Removed format: uri from SSE (was blocking template variables)

2. Go Types (pkg/model/types.go):

  • Base Transport struct unchanged for Package context
  • RemoteTransport struct for remotes with Variables field
  • Both work with existing validation infrastructure

3. Validators (internal/validators/):

  • validateRemoteTransport() validates Remote-specific constraints
  • collectRemoteTransportVariables() extracts available variables
  • IsValidTemplatedURL() validates template variables reference defined variables
  • IsValidRemoteURL() handles template variable substitution before localhost check
  • 6 new test cases for remote transport variable validation

4. Documentation Examples (generic-server-json.md):

  • Remote Server with URL Templating (StreamableHttp with {tenant_id})
  • SSE Remote with URL Templating (SSE with {tenant_id})
  • Local Server with URL Templating (Package with {port})
  • All examples validate via existing validate-examples tool

Example: Remote Transport with Variables

{
  "remotes": [
    {
      "type": "streamable-http",
      "url": "https://api.example.com/mcp/{tenant_id}/{region}",
      "variables": {
        "tenant_id": {
          "description": "Tenant identifier",
          "isRequired": true,
          "choices": ["us-cell1", "emea-cell1", "apac-cell1"]
        },
        "region": {
          "description": "Deployment region",
          "isRequired": true
        }
      }
    }
  ]
}

Clients configure the variables, and {tenant_id} and {region} get replaced to connect to tenant-specific endpoints like https://api.example.com/mcp/us-cell1/east.

Architecture Details

Context-based variable resolution:

  • Package + LocalTransport: {port} references --port argument or PORT env var from parent Package
  • Remote + RemoteTransport: {tenant_id} references variables.tenant_id defined in the transport itself
  • Code reuse: StreamableHttp/Sse definitions shared via allOf, not duplicated
  • Validation: Template variables validated against available variables for each context

Follow-on Work

Follow-on work will be done to adopt the feedback in #570 (comment) regarding variable naming/prioritzation/scoping conventions, but should not have any impact on the shape being introduced here (just validations).

"url": {
"type": "string",
"description": "URL template for the streamable-http transport. Variables in {curly_braces} reference argument valueHints, argument names, or environment variable names. After variable substitution, this should produce a valid URI.",
"description": "URL template for the streamable-http transport. Variables in {curly_braces} reference variable names from the 'variables' object. If variables are not provided, {curly_braces} should be treated as literal text. After variable substitution, this should produce a valid URI.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea with the references initially here is that a server that is launched with stdio and has, for example, a --port argument could automatically reference the value of that argument in the transport. I hypothesize the vast majority of locally-launched servers are going to have an address detereministic from their command line inputs and (from a client PoV) asking the user to fill in the port in the url separately from a port in the package_arguments is just confusing.

Copy link
Contributor

@connor4312 connor4312 Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also,

All existing server.json files remain valid

this is not the case -- this change is breaking for any packages that were already published with the original reference behavior (not sure if there are any, this was only in a recent addition)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, right. Thanks for the reminder that was the original reason we didn't include a separate field here.

The learning we've had since is that this doesn't work for the usage of StreamableHttpTransport and SseTransport within Remote.

Will think more on how we can properly combine (or separate out) the use cases.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

definitely -- I suggest perhaps combining the variables with an allOf in just the Remote case and adding some description in the property.

Copy link
Member Author

@tadasant tadasant Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also re -

this is not the case -- this change is breaking for any packages that were already published with the original reference behavior (not sure if there are any, this was only in a recent addition)

That's my bad - I had forgotten this original use case inside Package and thus thought the reference to arguments/envvars was just a typo. This is all clearer now that you've reminded me :) Thanks again

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feeling pretty good about the direction now: https://github.com/modelcontextprotocol/registry/pull/570/files

Not ready to land this, but intent for me here was to explore whether we can add this functionality without breaking changes. I think the answer is yes, so might punt this for a few days.

@tadasant tadasant force-pushed the tadasant/support-remotes-in-json branch from 73c2f75 to 622a257 Compare September 29, 2025 17:02
@BobDickinson
Copy link
Contributor

I somehow missed this PR and created #576 (which should get closed if this PR is adopted). FWIW, I've reviewed this PR and am a fan of the approach (including adding variables to the remote transports for URI substitution).

@domdomegg
Copy link
Member

I'm happy with this approach 👍. Might just need a rebase and minor code cleanup

@BobDickinson
Copy link
Contributor

BobDickinson commented Oct 3, 2025

OK, I really like this PR and don't want to slow it down. That being said, in going through existing registry entries with an eye toward making them take full advantage of the configuration support offered, I ran across a case that might impact this.

The general use case for this feature is configurable ports for the package transport url (so we configure the port we want the server to run on, and then have the transport config pick up that same port from the runtime config). But what about the case where the port is not an argument or env var, but is another part of the config? For example (from a published server):

Current

{
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-09-16/server.schema.json",
  "name": "io.github.SamYuan1990/i18n-agent-action",
  "description": "An i18n github action for language translate",
  "version": "mcp",
  "packages": [
    {
      "registryType": "oci",
      "registryBaseUrl": "https://ghcr.io",
      "identifier": "SamYuan1990/i18n-agent-action",
      "version": "mcp",
      "runtimeHint": "docker",
      "transport": {
        "type": "sse",
        "url": "https://example.com:8080/sse"
      },
      "runtimeArguments": [
        {
          "description": "Port mapping from host to container",
          "value": "8080:8080",
          "type": "named",
          "name": "-p"
        }
      ]
    }
  ]
}

If we wanted to parameterize the port so we can change it and have it be reflected in the docker command and the transport url, we would do something like this:

Parameterized

{
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-09-16/server.schema.json",
  "name": "io.github.SamYuan1990/i18n-agent-action",
  "description": "An i18n github action for language translate",
  "version": "mcp",
  "packages": [
    {
      "registryType": "oci",
      "registryBaseUrl": "https://ghcr.io",
      "identifier": "SamYuan1990/i18n-agent-action",
      "version": "mcp",
      "runtimeHint": "docker",
      "transport": {
        "type": "sse",
        "url": "https://example.com:{host_port}/sse"
      },
      "runtimeArguments": [
        {
          "description": "Port mapping from host to container",
          "value": "{host_port}:8080",
          "type": "named",
          "name": "-p",
          "variables": {
            "host_port": {
              "description": "The host (local) port",
              "isRequired": true,
              "format": "number",
              "default": "8080"
            }
          }
        }
      ]
    }
  ]
}

For docker we have to parameterize (templatize) the port (-p) argument with a variable to make the host port configurable. With the current resolution logic, there would be no way to map the token in the url to that (because there is no corresponding arg or env var representing just the host port).

I would argue that the mapping logic is already a little "spicy" since you could theoretically have runtime args, package args, and env vars with conflicting names (so you already need to specify the resolution logic / priority for deterministic resolution). My recommendation is that the guidance for token substitution be amended to say that after checking the runtime args, package args, and env vars (in that order) that you then fall back to checking for a matching variable under each of those things (in the same order). That would support the exact case above and keep the syntax clean.

I also considered a syntax like {p.host_port} indicating look in the first matching config element (arg/env) named "p", then use it's variable called "host_port". It's more specific, but I'm not sure that's worth the complexity for what it solves (directing to a specific variable when there might be multiple config elements with the same variable name - and since the variable names are only internal and are under the publisher's control, it's their own fault if they're ambiguous IMO).

@connor4312
Copy link
Contributor

Good callout.

I also considered a syntax like {p.host_port} indicating look in the first matching config element (arg/env) named "p", then use it's variable called "host_port".

I think I'd prefer this to the blind sub-variable matching. Though it would actually be {-p.host_port} in your example as -p is the name of the argument.

@BobDickinson
Copy link
Contributor

I could certainly live with the directed (dot) references.

I can't explain exactly why, but {-p.host_port} and even {--port} hurt my eyes, even though they are technically correct. It's complicated by the fact that many publishers aren't using the -- in the argument name (even though the docs are very clear about it), making the tokens look prettier, but causing the generated args to be wrong. I have a lax mode of config generation that works around this (both in token matching and config generation), but I'm not happy about it. I have a validator proposal coming soon (with schema validation and a linter to catch logic errors / misconfigurations, including things like missing dashes in arg names).

But back to the point: I'd advocate that we specify that it's OK to omit the leading dashes for named arg matching in the url token just to make it not hurt my eyes, but maybe that's not a good enough reason ;)

@BobDickinson
Copy link
Contributor

IMO, both StreamableHttpTransport and SseTransport url should have:

  "pattern": "^https?://[^\\s]+$"

So as to at least encourage url-like values (versus free form strings)

Currently StreamableHttpTransport has no pattern or format, and SseTransport has:

  "format": "uri"

which will prevent tokens in SSE (the intent to allow tokens in SSE seems clear, so I assume this is just an oversight).

@tadasant
Copy link
Member Author

tadasant commented Oct 9, 2025

Sorry I've been slow to follow up here - planning to get into it tomorrow

@tadasant
Copy link
Member Author

@BobDickinson @connor4312 regarding the last few comments -- am I correct in reading that those changes don't affect the shape of "adding template variable support in remote transports", and are both additional changes we could do as follow on PRs?

If so I will probably skip on those and let's do them in follow ups, just so I can land this. Sorry I've been slow here.

#565)

This pull request makes a minor update to the documentation for
installing the MCP Publisher in GitHub Actions workflows. The change
updates the installation command to use version 1.1.0 of the MCP
Publisher instead of version 1.0.0.

The `mcp-publisher` version in GitHub Actions has to be updated so it
can publish the MCP server to the newest version of MCP Registry.

Documentation update only. No test needed.

No

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update

<!-- Go over all the following points, and put an `x` in all the boxes
that apply. -->
- [x] I have read the [MCP
Documentation](https://modelcontextprotocol.io)
- [x] My code follows the repository's style guidelines
- [x] New and existing tests pass locally
- [ ] I have added appropriate error handling
- [x] I have added or updated documentation as needed

None
@tadasant tadasant force-pushed the tadasant/support-remotes-in-json branch from 0a4ae44 to 4335162 Compare October 13, 2025 15:54
@connor4312
Copy link
Contributor

am I correct in reading that those changes don't affect the shape of "adding template variable support in remote transports", and are both additional changes we could do as follow on PRs?

Correct

…es support

- Add LocalTransport type (StdioTransport | StreamableHttpTransport | SseTransport) for Package context
- Add RemoteTransport type extending StreamableHttp/Sse with variables field for Remote context
- Update Package transport to reference LocalTransport
- Update remotes to reference RemoteTransport
- Update transport URL descriptions to clarify variable resolution by context
- Remove unintended production_servers.json file

This enables URL templating for remote servers with dedicated variable definitions,
supporting multi-tenant deployments with configurable endpoints.
- Remove 'Deprecated Server Example' section that conflicts with main's intent
- Update expectedServerJSONCount from 15 to 14 (12 original + 2 new URL templating examples)
@BobDickinson
Copy link
Contributor

I'd like to see the fix to the SSE format (which is a current bug that prevents SSE transports using templates, and fails on currently published server entries that would otherwise pass).

The more flexible template resolution could be a separate PR.

tadasant and others added 3 commits October 13, 2025 09:20
- Add 6 new test cases for remote transport variable validation:
  - Valid single variable in URL
  - Valid multiple variables in URL (hostname + path)
  - Invalid undefined variable in URL
  - Invalid missing variable definition
  - Valid SSE transport with variables
  - Valid variables defined but not used in URL
- Fix IsValidRemoteURL to replace template variables before localhost check
- Update IsValidTemplatedURL comment to reflect remote variable support

All tests pass. Closes test coverage gap for the new remote transport
variables feature.
Add pattern validation to ensure transport URLs are URL-like values
while still supporting template variables for multi-tenant deployments.

Changes:
- Add `pattern: "^https?://[^\\s]+$"` to both StreamableHttpTransport
  and SseTransport URL fields in openapi.yaml
- Remove `format: uri` from SseTransport URL field which was blocking
  template variables like {tenant_id} in URLs
- Regenerate server.schema.json from openapi.yaml
- Add comprehensive test coverage in schema_pattern_test.go that
  verifies:
  - Static URLs pass validation
  - Template variables in URLs pass validation (critical for remotes)
  - Free-form strings without http(s):// are rejected
  - URLs with spaces are rejected

This fixes an issue where StreamableHttpTransport had no URL validation
and SseTransport's format: uri was preventing valid template variable
usage. The new pattern allows templates while still enforcing basic URL
structure.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
…ample

Instead of adding a new test pattern (schema_pattern_test.go), add a
simple SSE transport example with template variables to the documentation
which gets validated by the existing validate-examples tool.

Changes:
- Add SSE remote transport example with {tenant_id} template variable
  to generic-server-json.md
- Update expectedServerJSONCount from 14 to 15 in validate-examples tool
- Remove internal/validators/schema_pattern_test.go (new test pattern)

This follows the existing testing patterns where schema validation
happens through documentation examples rather than dedicated unit tests.
The new SSE example proves that template variables work correctly with
the pattern validation added in the previous commit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@tadasant tadasant changed the title feat: Add template variable support for remote transport URLs feat: Add LocalTransport/RemoteTransport with URL template variables Oct 13, 2025
tadasant and others added 2 commits October 13, 2025 10:16
Instead of duplicating the variables definition in each branch of anyOf,
use allOf at the top level with anyOf nested inside. This makes the
schema more maintainable and follows better composition practices.

Before:
  RemoteTransport = (StreamableHttp + variables) | (SSE + variables)

After:
  RemoteTransport = (StreamableHttp | SSE) + variables

Changes:
- Change RemoteTransport from anyOf with repeated allOf to allOf with anyOf
- Variables definition now appears only once at the top level
- Semantically equivalent but more DRY and maintainable
- All validation tests still pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
…dURL

The allowTemplates boolean parameter was never used - both call sites
always passed true. This was dead code left over from an earlier design
where we considered disallowing templates in certain contexts.

Since we now support templates in both Package and Remote contexts, the
parameter serves no purpose.

Changes:
- Remove allowTemplates parameter from IsValidTemplatedURL signature
- Update both call sites (package and remote validation)
- Remove related dead code checking !allowTemplates
- Simplify function logic

All tests still pass - functionality unchanged.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@tadasant
Copy link
Member Author

I'd like to see the fix to the SSE format (which is a current bug that prevents SSE transports using templates, and fails on currently published server entries that would otherwise pass).

The more flexible template resolution could be a separate PR.

Fair - included this.

Ready for final review @BobDickinson @connor4312 @modelcontextprotocol/registry-wg

@tadasant tadasant requested a review from a team October 13, 2025 17:22
@tadasant tadasant marked this pull request as ready for review October 13, 2025 17:22
@tadasant
Copy link
Member Author

Follow up ticket for the case @BobDickinson raised a few comments ago: #656

"name": "--port",
"description": "Port for the server to listen on",
"default": "3000",
"valueHint": "port"
Copy link
Contributor

@connor4312 connor4312 Oct 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

valueHint is only valid on positional arguments, not named arguments (otherwise we would have nothing to key off of except an unstable index)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed, thanks! Also added a ticket #662

tadasant and others added 3 commits October 14, 2025 05:37
When validating URLs with template variables in the port position
(e.g., http://localhost:{--port}/mcp), the validator was incorrectly
replacing the template variable with the string "placeholder", which
caused URL parsing to fail since ports must be numeric.

This commit fixes the issue by:
- Detecting when a template variable appears in a port position
  (after a colon and before a slash or end of string)
- Replacing port-position template variables with a numeric placeholder
  "8080" instead of "placeholder"
- Adding a new error constant ErrInvalidPackageTransportURL to
  distinguish package transport errors from remote transport errors
- Updating validatePackageTransport to use the correct error constant

The fix allows named arguments like "--port" to be properly referenced
in package transport URLs, enabling examples like:
```json
{
  "transport": {
    "type": "streamable-http",
    "url": "http://localhost:{--port}/mcp"
  },
  "packageArguments": [
    {
      "type": "named",
      "name": "--port",
      "default": "3000"
    }
  ]
}
```

Fixes validation of Example 15 in generic-server-json.md (line 732)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Copy link
Contributor

@connor4312 connor4312 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spec side lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants