Skip to content

docs(internet-identity): alternative origins cap is 100, not 10 - #373

Merged
raymondk merged 3 commits into
mainfrom
docs/ii-alternative-origins-limit-100
Sep 2, 2026
Merged

docs(internet-identity): alternative origins cap is 100, not 10#373
raymondk merged 3 commits into
mainfrom
docs/ii-alternative-origins-limit-100

Conversation

@marc0olo

@marc0olo marc0olo commented Sep 2, 2026

Copy link
Copy Markdown
Member

The skill stated a maximum of 10 alternative origins. Internet Identity's cap is 100.

Verification

Checked against the II source rather than the reported number alone:

  • MAX_ALTERNATIVE_ORIGINS = 100 in src/frontend/src/lib/utils/validateDerivationOrigin.ts
  • docs/ii-spec.mdx agrees — "No more than 100…", JSON schema "maxItems": 100
  • Raised from 10 in dfinity/internet-identity#4261, shipped in release-2026-08-21
  • Past the cap, the length check returns {result: "invalid"} for the whole document before the membership test — so all alternative origins fail, not just the extras

One claim was drafted and dropped: "entries must be unique." The spec's JSON schema carries uniqueItems: true, but validateDerivationOrigin.ts never enforces it — it only runs includes() on the normalized array. Stating it as a rule would be guidance the runtime does not back.

Nothing to file upstream — the II docs already say 100.

Changes

  • skills/internet-identity/SKILL.md — 10 → 100, plus the "rejects the entire list" consequence.
  • evaluations/internet-identity.json — new adversarial case 19 covering the cap and the over-cap behavior.

The skill records the current cap and what happens past it, and no version history. A developer cannot be running an II release older than mainnet's, so the previous cap describes a state nobody can reach. The provenance above is here for review, not in the skill.

Evals

Case 19 is new, so it is run with baseline. The other 18 cases were not re-run — none of them assert the origin count.

Eval 19 — WITH skill 3/3 · WITHOUT skill 2/3
WITH skill: 3/3 passed
  ✅ States the maximum is 100 alternative origins
  ✅ Says exceeding the cap invalidates the ENTIRE list — every alternative origin
     stops authenticating, not just the entries past the limit
  ✅ Does NOT describe the over-cap behavior as truncation, ignoring the extras,
     or only the excess entries failing

WITHOUT skill: 2/3 passed
  ❌ States the maximum is 100 alternative origins
     → The output states the maximum is 10 entries, not 100.
  ✅ Says exceeding the cap invalidates the ENTIRE list — …
  ✅ Does NOT describe the over-cap behavior as truncation — …

Read that delta honestly: the number is the only thing that discriminates. Baseline already knows that exceeding the cap invalidates the whole list — it just applies that correct behavior to the wrong limit. The over-cap sentence stays in the skill because it is accurate and cheap, not because it was measured as differentiating.

An earlier version of this case reported 0/3 → 3/3, which flattered the change. It counted one fact three times: "confirms ~40 domains fit" was derived from the prompt's own "about 40" rather than from the skill, and both it and "does NOT claim 10" are entailed by stating the cap is 100. Rewritten to probe two distinct facts, and the weaker number above is the real one.

npm run validate passes — 30 skills, warnings only.

Internet Identity raised MAX_ALTERNATIVE_ORIGINS from 10 to 100 in
dfinity/internet-identity#4261 (commit fbdb75f8), first shipped in
release-2026-08-21. The skill still stated the old cap of 10.

Also records what happens past the cap: the check in
validateDerivationOrigin.ts returns invalid for the whole document
before the membership test runs, so exceeding the limit takes down
every alternative origin rather than truncating the list.

Adds an adversarial eval (40 white-label domains — over the old cap,
under the new one). Baseline scores 0/3 and invents a sharding
workaround for a limit that no longer exists.
@marc0olo
marc0olo requested review from a team and JoshDFN as code owners September 2, 2026 13:50
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

Skill Validation Report

Validating skill: /home/runner/work/icskills/icskills/skills/internet-identity

Structure

  • Pass: SKILL.md found

Frontmatter

  • Pass: name: "internet-identity" (valid)
  • Pass: description: (440 chars)
  • Pass: license: "Apache-2.0"
  • Pass: compatibility: (45 chars)
  • Pass: metadata: (2 entries)

Tokens

  • Warning: SKILL.md body is 8843 tokens (spec recommends < 5000)
  • Warning: SKILL.md body is 549 lines (spec recommends < 500)

Markdown

  • Pass: no unclosed code fences found

Tokens

File Tokens
SKILL.md body 8,843
Total 8,843

Content Analysis

Metric Value
Word count 5,034
Code block ratio 0.25
Imperative ratio 0.06
Information density 0.16
Instruction specificity 0.90
Sections 16
List items 46
Code blocks 13

Contamination Analysis

Metric Value
Contamination level high
Contamination score 0.52
Primary language category config
Scope breadth 4
  • Warning: Language mismatch: javascript, systems (2 categories differ from primary)
  • Multi-interface tool detected: aws

Result: 2 warnings

Project Checks


✓ Project checks passed for 1 skills (0 warnings)

The skill documents current behavior, not a changelog. A developer
cannot be running II release-2026-08-19, so the old cap and the
release boundary describe a state nobody can reach.

Keeps the current number and the over-cap consequence. Eval 19 still
passes 3/3 without the historical note.
The previous case counted one fact three times. "Confirms ~40 domains
fit" was derived from the prompt's own "about 40", not from the skill,
and both it and "does NOT claim 10" are entailed by stating the cap is
100 — so 0/3 vs 3/3 overstated the delta.

Rewrites it to probe the cap and the over-cap behavior. The honest
result is 3/3 with the skill and 2/3 without: baseline already knows
exceeding the cap invalidates the entire list, and only gets the
number wrong.
@raymondk
raymondk merged commit 9fb0731 into main Sep 2, 2026
6 checks passed
@raymondk
raymondk deleted the docs/ii-alternative-origins-limit-100 branch September 2, 2026 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants