openhands-automation skill steers agents toward prompt presets even for deterministic tasks

[enyst]

Desired Behavior

The openhands-automation skill should give the agent an even-handed view of prompt presets versus deterministic custom scripts. Rule 0 already says a task that needs no reasoning should be a no-LLM script, but the rest of the skill text pulls toward the prompt preset:

• Several prompt-preset examples model deterministic tasks (an hourly health check, running tests on push, a weekly temp-file cleanup). An agent reads rule 0 and then watches the skill break it.
• references/custom-automation.md opens with a warning not to use the reference unless the user explicitly asked for a custom automation. That contradicts rule 0 and the Reference Files section, which tell the agent to consult it in order to recommend the custom path.
• The skill has fourteen prompt-preset examples and no complete no-LLM example in the body, so the two paths are not equally concrete.
• Custom scripts need an extra "only after the user agrees" step that presets do not, even though the creation process already asks "ready to deploy?" for everything.

The skill should model agent-shaped tasks in its prompt examples, drop the one-sided warning, include one complete no-LLM worked example next to the preset examples, and apply the same consent step to every path.

Acceptance Criteria

• No prompt-preset example in SKILL.md models a task that rule 0 lists as always deterministic.
• references/custom-automation.md no longer opens with a "do not use unless explicitly requested" warning, and lists deterministic tasks as a reason to use the custom path.
• SKILL.md contains one complete no-LLM example (script, tarball, upload, create) next to the prompt-preset examples.
• The agent behavior rules present prompt preset, plugin preset, and custom script side by side and apply the same confirmation step to each.
• skills/index.js is regenerated and uv run python scripts/sync_extensions.py --check plus the catalog tests pass.

Additional Context

Observed while reviewing what the agent actually receives from this skill. The imbalance is in the weight of the document, not in its stated rules.

---

OpenHands AI triage

The following comments and acceptance criteria were added by the OpenHands AI agent.

Triage

The issue is in scope and unambiguous: skills/openhands-automation/SKILL.md states in rule 0 that deterministic tasks (health-check pings, scheduled reminders, templated posts, rotating from a fixed list) must not use an LLM preset, while the surrounding skill text models those same tasks as prompt-preset examples and makes the custom path carry an extra consent step plus a "do not use unless explicitly requested" gate on references/custom-automation.md. The fix is a content change confined to skills/openhands-automation/ and the generated skills/index.js; no API, schema, or preset behavior changes.

Bounded scope: rebalance the skill text and examples, and keep derived artifacts in sync. Explicit non-goals: changing the automation API, presets, or automations/catalog; the separate one-sided warning in plugins/openhands/commands/create.md, which is a different artifact and is left to its own issue; and adding new deterministic presets. An open PR (#686) already implements this exact scope, which confirms the design intent is settled.

Acceptance Criteria

• No prompt-preset or plugin-preset example in skills/openhands-automation/SKILL.md models a task that rule 0 lists as always deterministic. Specifically, the hourly health-check prompt tip, the "Run tests on push to main" event example, and the "Weekly Cleanup" cron example are replaced with genuinely agent-shaped tasks that require reasoning or judgment (for example error triage, reproduction, or dependency review), and none of the remaining preset examples matches a rule 0 instant-recognition pattern.
• skills/openhands-automation/references/custom-automation.md no longer opens with the "Do NOT use this reference unless the user has explicitly requested a custom automation" warning, and its "when to use" list names deterministic / no-LLM tasks as a first-class reason to choose the custom path.
• skills/openhands-automation/SKILL.md contains one complete no-LLM worked example next to the prompt-preset examples, covering all four steps end to end: the main.py script (including the get_secret and fire_callback helpers), tarball creation, the POST /api/automation/v1/uploads upload, and the POST /api/automation/v1 create call with entrypoint set to python3 main.py. The new example script passes python3 -m py_compile, and its success and failure paths are exercised (against a reachable or deliberately unreachable URL) with the documented exit behavior.
• The agent behavior rules in SKILL.md present prompt preset, plugin preset, and custom script side by side in one list and apply the same confirmation step to all three. No path requires separate consent that the others do not, and the creation process still requires user confirmation before deployment for every path.
• The statement is preserved that determinism, not the user's naming of a "custom script", is what selects the no-LLM path, and references/custom-automation.md remains the single reference for it (no parallel source of truth is introduced).
• skills/openhands-automation/README.md is updated so it no longer presents the prompt preset as the recommended default or frames custom automation as only for advanced users, consistent with the evenhanded SKILL.md text (required by the repository convention that a skill behavior change updates its README).
• skills/index.js is regenerated from the updated SKILL.md and node scripts/build-skills-catalog.mjs produces no further diff (the test_index_is_up_to_date check passes).
• uv run python scripts/sync_extensions.py --check passes, and tests/test_skills_catalog.py, tests/test_catalogs.py, and tests/test_sync_extensions.py pass.
• Existing automations and presets are unaffected: the preset endpoints, request fields, and generated skills/index.js schema (name, description, triggers, content, category) are unchanged apart from the updated content string for openhands-automation, and DEFAULT_ENABLED_SKILL_NAMES is unchanged.