Skip to content

FEAT: add code_attack_framed technique (closes #2088) - #2721

Merged
Richard Lundeen (richlundeen) merged 3 commits into
microsoft:mainfrom
u7k4rs6:feat/2088-code-attack-system-prompt
Sep 22, 2026
Merged

Richard Lundeen (richlundeen) merged 3 commits into
microsoft:mainfrom
u7k4rs6:feat/2088-code-attack-system-prompt

Conversation

@u7k4rs6

@u7k4rs6 Utkarsh Bahuguna (u7k4rs6) commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Description

Follow-up to #1960. Closes #2088.

code_attack (added in #1960) implements the CodeAttack code-completion method from Ren et al., CodeAttack, Findings of ACL 2024 (arXiv:2403.07865). The paper and reference implementation send the generated code template as a user message; they do not define a separate code-completion system prompt.

This PR adds code_attack_framed, an optional PyRIT variant that combines the existing paper-aligned converter with additional system framing. The existing code_attack technique is unchanged.

Because the framing is a PyRIT-specific experimental condition rather than part of the reference method, the new variant is registered in the opt-in extra catalog. It does not expand RapidResponse's default light technique set.

No new abstraction was needed

#2088 asked whether this should be a technique variant, or whether a generic way to attach an objective-target system prompt should land first. That mechanism already exists through AttackTechniqueSeedGroup.from_system_prompt, so no bespoke attack class is required.

Targets without editable history can use the existing capability-normalization path to fold prepended framing into the target-facing user turn.

Changes

File Change
pyrit/datasets/executors/code_attack.yaml restores the historical PyRIT framing seed and clarifies its provenance
pyrit/setup/initializers/techniques/extra.py adds the opt-in code_attack_framed factory
tests/unit/setup/techniques/test_extra_techniques.py covers factory shape, converter parity, seed merging, persisted framing, and capability adaptation
tests/unit/setup/test_technique_initializer.py adds the name to EXTRA_TECHNIQUE_NAMES

The converter config is duplicated between code_attack and code_attack_framed, as required by the declarative technique catalog convention.

Tests and documentation

  • 98 passed across the affected setup technique and initializer tests.
  • The complete pre-commit suite passed formatting, lint, type checking, async naming, YAML, notebook, documentation, migration, and secret checks. The repository-wide check-added-large-files hook hit a Windows git check-attr subprocess error; its checker passed directly on the added test file.
  • No documentation source changes are required. The attack-technique catalog is generated from the registered factories.

Clarify that the system prompt is an optional PyRIT framing variant rather than part of the CodeAttack paper or reference implementation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@richlundeen
Richard Lundeen (richlundeen) added this pull request to the merge queue Sep 22, 2026
Merged via the queue into microsoft:main with commit 58f4883 Sep 22, 2026
49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a code_attack technique variant that applies the code-completion system prompt

2 participants