docs(internet-identity): alternative origins cap is 100, not 10 - #373
Merged
Conversation
Internet Identity raised MAX_ALTERNATIVE_ORIGINS from 10 to 100 in dfinity/internet-identity#4261 (commit fbdb75f8), first shipped in release-2026-08-21. The skill still stated the old cap of 10. Also records what happens past the cap: the check in validateDerivationOrigin.ts returns invalid for the whole document before the membership test runs, so exceeding the limit takes down every alternative origin rather than truncating the list. Adds an adversarial eval (40 white-label domains — over the old cap, under the new one). Baseline scores 0/3 and invents a sharding workaround for a limit that no longer exists.
Skill Validation ReportValidating skill: /home/runner/work/icskills/icskills/skills/internet-identityStructure
Frontmatter
Tokens
Markdown
Tokens
Content Analysis
Contamination Analysis
Result: 2 warnings Project Checks |
The skill documents current behavior, not a changelog. A developer cannot be running II release-2026-08-19, so the old cap and the release boundary describe a state nobody can reach. Keeps the current number and the over-cap consequence. Eval 19 still passes 3/3 without the historical note.
The previous case counted one fact three times. "Confirms ~40 domains fit" was derived from the prompt's own "about 40", not from the skill, and both it and "does NOT claim 10" are entailed by stating the cap is 100 — so 0/3 vs 3/3 overstated the delta. Rewrites it to probe the cap and the over-cap behavior. The honest result is 3/3 with the skill and 2/3 without: baseline already knows exceeding the cap invalidates the entire list, and only gets the number wrong.
raymondk
approved these changes
Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The skill stated a maximum of 10 alternative origins. Internet Identity's cap is 100.
Verification
Checked against the II source rather than the reported number alone:
MAX_ALTERNATIVE_ORIGINS = 100insrc/frontend/src/lib/utils/validateDerivationOrigin.tsdocs/ii-spec.mdxagrees — "No more than 100…", JSON schema"maxItems": 100release-2026-08-21{result: "invalid"}for the whole document before the membership test — so all alternative origins fail, not just the extrasOne claim was drafted and dropped: "entries must be unique." The spec's JSON schema carries
uniqueItems: true, butvalidateDerivationOrigin.tsnever enforces it — it only runsincludes()on the normalized array. Stating it as a rule would be guidance the runtime does not back.Nothing to file upstream — the II docs already say 100.
Changes
skills/internet-identity/SKILL.md— 10 → 100, plus the "rejects the entire list" consequence.evaluations/internet-identity.json— new adversarial case 19 covering the cap and the over-cap behavior.The skill records the current cap and what happens past it, and no version history. A developer cannot be running an II release older than mainnet's, so the previous cap describes a state nobody can reach. The provenance above is here for review, not in the skill.
Evals
Case 19 is new, so it is run with baseline. The other 18 cases were not re-run — none of them assert the origin count.
Eval 19 — WITH skill 3/3 · WITHOUT skill 2/3
Read that delta honestly: the number is the only thing that discriminates. Baseline already knows that exceeding the cap invalidates the whole list — it just applies that correct behavior to the wrong limit. The over-cap sentence stays in the skill because it is accurate and cheap, not because it was measured as differentiating.
An earlier version of this case reported 0/3 → 3/3, which flattered the change. It counted one fact three times: "confirms ~40 domains fit" was derived from the prompt's own "about 40" rather than from the skill, and both it and "does NOT claim 10" are entailed by stating the cap is 100. Rewritten to probe two distinct facts, and the weaker number above is the real one.
npm run validatepasses — 30 skills, warnings only.