FEAT: add code_attack_framed technique (closes #2088) - #2721
Merged
Richard Lundeen (richlundeen) merged 3 commits intoSep 22, 2026
Merged
Richard Lundeen (richlundeen) merged 3 commits into
Richard Lundeen (richlundeen) merged 3 commits into
Conversation
Utkarsh Bahuguna (u7k4rs6)
force-pushed
the
feat/2088-code-attack-system-prompt
branch
from
September 21, 2026 21:27
0ec8c65 to
7aa8af9
Compare
Clarify that the system prompt is an optional PyRIT framing variant rather than part of the CodeAttack paper or reference implementation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Richard Lundeen (richlundeen)
approved these changes
Sep 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Follow-up to #1960. Closes #2088.
code_attack(added in #1960) implements the CodeAttack code-completion method from Ren et al., CodeAttack, Findings of ACL 2024 (arXiv:2403.07865). The paper and reference implementation send the generated code template as a user message; they do not define a separate code-completion system prompt.This PR adds
code_attack_framed, an optional PyRIT variant that combines the existing paper-aligned converter with additional system framing. The existingcode_attacktechnique is unchanged.Because the framing is a PyRIT-specific experimental condition rather than part of the reference method, the new variant is registered in the opt-in
extracatalog. It does not expand RapidResponse's defaultlighttechnique set.No new abstraction was needed
#2088 asked whether this should be a technique variant, or whether a generic way to attach an objective-target system prompt should land first. That mechanism already exists through
AttackTechniqueSeedGroup.from_system_prompt, so no bespoke attack class is required.Targets without editable history can use the existing capability-normalization path to fold prepended framing into the target-facing user turn.
Changes
pyrit/datasets/executors/code_attack.yamlpyrit/setup/initializers/techniques/extra.pycode_attack_framedfactorytests/unit/setup/techniques/test_extra_techniques.pytests/unit/setup/test_technique_initializer.pyEXTRA_TECHNIQUE_NAMESThe converter config is duplicated between
code_attackandcode_attack_framed, as required by the declarative technique catalog convention.Tests and documentation
98 passedacross the affected setup technique and initializer tests.check-added-large-fileshook hit a Windowsgit check-attrsubprocess error; its checker passed directly on the added test file.