fix(ai): follow RFC 9728 protected-resource metadata in MCP OAuth discovery

[yurekami]

Fixes #766.

The defect

discover() probed only two documents, both at the MCP endpoint's own origin:

/** Try the protected-resource and auth-server well-known docs at the URL's origin. */
async function discover(url: string): Promise<AuthServerMetadata> {
	const origin = new URL(url).origin;
	const candidates = [
		`${origin}/.well-known/oauth-authorization-server`,
		`${origin}/.well-known/openid-configuration`,
	];

The doc comment names protected-resource metadata; oauth-protected-resource appears nowhere in the tree at b9a4461. RFC 9728 makes that document the resource's own statement of which authorization server to use, and it routinely names a different host, so /mcp login was impossible for those servers.

The change

1. Fetch the protected-resource document first: path-suffixed (/.well-known/oauth-protected-resource/mcp/v1), then bare root. Both are needed — see ironclad below.
2. Resolve each advertised issuer with the RFC 8414 path-insertion form (https://host/.well-known/oauth-authorization-server/oidc) and the OpenID Connect Discovery path-append form.
3. Fall back to today's two origin-rooted probes, unchanged, for servers publishing no such document.
4. Split fetchJson's transport failure from its JSON-parse failure.

refreshToken() inherits this through its existing discover() call; no call site changed. Point 4 is not cosmetic: Ashby returns HTTP 200 with an HTML sign-in page at /.well-known/oauth-authorization-server, so today's failure is a bare SyntaxError: Unexpected token '<' with no indication of which document was bad.

Evidence

I ran the real createMcpOAuthProvider().login() against production servers with fetch restricted to GET /.well-known/ requests, so discovery ran end to end and the DCR/token requests were blocked. Same harness on both sides.

Before, at b9a4461:

Endpoint	origin oauth-authorization-server	origin openid-configuration	Result
https://ai.todoist.net/mcp	404	404	fails
https://mcp.ashbyhq.com/mcp/v1	200 (HTML)	200 (HTML)	fails, SyntaxError
https://mcp.box.com	401	401	fails
https://drivemcp.googleapis.com/mcp/v1	404	404	fails
https://mcp.na1.ironcladapp.com/mcp	404	404	fails
https://mcp.shippo.com	403	403	fails
https://mcp.linear.app/mcp	200	—	resolves

After:

Endpoint	Protected-resource document	Authorization server resolved
https://ai.todoist.net/mcp	…/oauth-protected-resource/mcp	https://todoist.com (DCR available)
https://mcp.ashbyhq.com/mcp/v1	…/oauth-protected-resource/mcp/v1	https://mcp-auth.ashbyhq.com/oidc (DCR available)
https://mcp.box.com	…/oauth-protected-resource	https://api.box.com/ (no DCR)
https://drivemcp.googleapis.com/mcp/v1	…/oauth-protected-resource/mcp/v1	https://accounts.google.com/ (no DCR)
https://mcp.na1.ironcladapp.com/mcp	suffixed 404, then bare root 200	https://ironcladapp.com (no DCR)
https://mcp.shippo.com	…/oauth-protected-resource	https://goshippo.com (DCR available)
https://mcp.linear.app/mcp	…/oauth-protected-resource/mcp	https://mcp.linear.app (unchanged)

Ashby, Box and Google exercise the three edge cases the helpers exist for: an issuer carrying a path, an issuer advertised with a trailing slash, and a path-suffixed document where the bare root 404s. Ironclad is the reverse — its suffixed document 404s and only the bare root answers.

Three of these discover successfully but advertise no registration_endpoint, so they still need a pre-registered client id. That is the separate gap the reporter notes; this PR does not address it.

Tests

Five cases added to packages/ai/test/mcp-oauth.test.ts (10 pass, 5 pre-existing unchanged). I mutation-checked them — each mutant is caught by exactly one test, and each passes at HEAD only when it should:

Mutation	Caught by
drop the bare-root protected-resource fallback	falls back to the bare protected-resource document…
probe the endpoint origin before the advertised issuer	resolves the authorization server from protected-resource metadata
drop the RFC 8414 path-insertion form	resolves an issuer that carries a path
revert the fetchJson non-JSON split	reports a non-JSON metadata document distinctly…

The precedence test serves a valid authorization-server document at the endpoint origin too, so it pins that the protected-resource document wins rather than merely that the origin probe failed.

npm run check is clean.

Verification

• Quoted code and the oauth-protected-resource absence checked against main @ b9a4461.
• Before/after table produced by running this branch and b9a4461 through the same harness against live servers.
• npm run check; packages/ai/test/mcp-oauth.test.ts 10/10.
• No login completed end to end. I have accounts on none of these services, so the harness stops at client registration. What is verified is that discovery resolves an authorization server advertising authorization_endpoint, token_endpoint, and S256 among its code_challenge_methods_supported — not that the subsequent DCR + PKCE exchange succeeds.
• The "17 of 36" figure is the reporter's measurement, not mine. I verified the seven endpoints above.

Deliberately out of scope

• WWW-Authenticate resource_metadata (step 1 of the proposal). Needs an extra unauthenticated probe and reaches into mcp-manager.ts. Every server above resolves without it.
• Recording the resolved issuer on the credential. Changes the persisted McpCredentials shape.
• Validating that the authorization-server document's issuer matches the advertised one (RFC 9728 SHOULD). Not added because a strict comparison would reject servers that work: Box advertises https://api.box.com/ and its document reports https://api.box.com, and Google advertises https://accounts.google.com/ against a reported https://accounts.google.com. Happy to add it with normalization if you want it here rather than separately.

One note on the fan-out: every advertised issuer is tried in order rather than only authorization_servers[0], with no cap. Say the word if you would rather bound it.

Note

Fix MCP OAuth discovery to follow RFC 9728 protected-resource metadata for cross-host authorization servers

• The discover function in packages/ai/src/mcp/oauth.ts now fetches RFC 9728 protected-resource metadata from the MCP endpoint first, reads listed authorization_servers issuers, and builds candidate discovery URLs using both RFC 8414 and OIDC discovery patterns before falling back to the endpoint's own origin.
• Adds support for issuers with path components, path-suffixed well-known URLs, and deduplication of candidate URLs; error messages now enumerate each attempted URL with its failure reason.
• fetchJson is updated to throw a distinct error when a well-known endpoint returns non-JSON content, and includes response body text for non-OK responses.
• New tests cover cross-host authorization servers, path-based issuers, fallback from path-suffixed to bare protected-resource documents, and non-JSON metadata error reporting.
• Behavioral Change: discovery now prefers the endpoint's protected-resource metadata over the endpoint origin, which changes resolution order for existing MCP endpoints that serve a protected-resource document.

^{Macroscope summarized b1c38eb.}

[sethkarten]

Thank you for the report and proposed work. This root cause is now covered by maintainer-owned stacked PR #1164, authored independently from upstream/main.

We did not inspect or reuse this PR's diff, branch, commits, implementation code, or tests; its public description/comments were used only as a bug report. To keep one review surface, this PR is superseded by #1164 and is being closed.

The complete review stack is #1158–#1165. It is being left unmerged for human review after CI and review-bot findings are cleared.