Test Plan: Migrate Customizations to Agent Host Harnesses

[pwang347]

Refs: #330539, #330772

• anyOS Logan Ramos (@lramos15)
• anyOS Henning Dieterichs (@hediet)
• anyOS Connor Peet (@connor4312)

Complexity: 4

Author: Paul (@pwang347)

Create Issue

---

Context: VS Code stores some personal agents, instructions, and prompt files in the profile's User Data directory. These files roam between devices through Settings Sync, but agent-host harnesses such as Copilot CLI do not read this location.

What Changed: The Agent Customizations editor now offers two independent migration experiences for agent-host harnesses:

• Migrate Prompt Files converts workspace and user prompt files into skills.
• Migrate User Data Customizations moves User Data agents and instructions into harness-specific folders without changing their contents.

Use a disposable profile or back up your User Data customizations before testing cases that delete the originals.

Setup

1. Enable:
   • chat.customizations.promptMigration.enabled
   • chat.customizations.userDataMigration.enabled
2. Ensure Copilot CLI sessions are available.
3. Create the following customizations:
   • A workspace .prompt.md file.
   • A user .prompt.md file.
   • A user .agent.md file in VS Code User Data.
   • A user .instructions.md file in VS Code User Data.
4. Run Open Customizations and select the Copilot CLI harness.

Test Cases

1. Migration experiences are independent

1. Disable both migration settings.
2. Verify neither migration card nor sidebar shortcut appears.
3. Enable only chat.customizations.promptMigration.enabled.
4. Verify only Migrate Prompt Files appears.
5. Disable it and enable only chat.customizations.userDataMigration.enabled.
6. Verify only Migrate User Data Customizations appears.
7. Enable both settings and verify both migrations appear with correct counts.

2. Prompt files migrate to skills

1. Open Migrate Prompt Files.
2. Verify prompt files are grouped into Workspace and User.
3. Verify search, individual selection, group selection, and collapsing groups work.
4. Open a candidate and verify its source file is shown.
5. Migrate selected files.
6. If multiple destination roots are available, select one and verify it is reused for the remaining files.
7. Verify:
   • A valid SKILL.md is created in the selected harness skill folder.
   • The migrated skill contains disable-model-invocation: true.
   • Supported prompt metadata and body content are retained.
   • Original prompt files are deleted when the delete checkbox is selected.
8. Repeat with the delete checkbox cleared and verify both original and migrated files remain.

3. User Data agents and instructions migrate without conversion

1. Open Migrate User Data Customizations.
2. Verify:
   • The Settings Sync warning is visible.
   • The learn-more link appears below the warning.
   • Files are grouped into Agents and Instructions.
3. Migrate an agent and instruction with deletion enabled.
4. Verify:
   • Each file moves to the appropriate harness-specific user folder.
   • Filename and file contents are unchanged.
   • The originals are removed from User Data.
   • The migrated files appear in the corresponding Customizations sections.
5. Repeat with deletion disabled and verify the originals remain in User Data.

4. Keyboard and accessibility behavior

1. Navigate the migration page using only the keyboard.
2. Verify the checkbox, file Open action, and Delete action are separate focus targets.
3. Activate Open with both Enter and Space.
4. Verify the opened file matches the selected row.
5. With a screen reader, verify the Open action announces both the customization name and its path, allowing same-named files to be distinguished.
6. On the overview, verify each migration card exposes only its Migrate... action as an interactive target and does not announce nested buttons.