FEAT: Add VigenereConverter - #2333
Conversation
@microsoft-github-policy-service agree |
The .py side of 1_text_to_text_converters was updated but the paired .ipynb was not; jupytext keeps these in lockstep. Adds the import, the conversion call, its output, and the markdown mention. Also refreshes the generated modality reference table in 0_converters.ipynb, which now picks up VigenereConverter (and AcrosticConverter, which was already missing on main). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4e67eb0e-8f79-4d4f-8023-e3f0da61cab7
| name: vigenere cipher | ||
| description: | | ||
| A Vigenere cipher description for prepending. | ||
| authors: |
| - Chitta Baral | ||
| groups: | ||
| - Arizona State University | ||
| source: https://arxiv.org/abs/2402.10601 |
There was a problem hiding this comment.
Can you double-check this attribution? Three separate things look off, and I think the fix is to drop the authors/groups block entirely.
1. The names and the link point at different papers.
The author list here (Handa, Chirmule, Gajera, Baral) is exactly v1 of arXiv 2402.10601, "Jailbreaking Proprietary Large Language Models using Word Substitution Cipher." But source is the versionless abs URL, which currently resolves to v5, "When 'Competency' in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Complex Ciphers." Their author lists differ:
authors (per arXiv citation_author metadata) |
|
|---|---|
| v1 | Handa, Chirmule, Gajera, Baral |
| v5 (what the link resolves to today) | Handa, Zhang, Saeidi, Kumbhar, Uddin, RRV, Baral |
Only Handa and Baral are on both. So the YAML contradicts its own citation as written — and because the URL is unpinned, that mismatch will keep drifting with each new version posted.
2. Neither version is actually about the Vigenère cipher. v1's technique is a word substitution cipher; v5 is about custom/layered encryptions (ACE/LACE). Vigenère isn't the contribution of either, so citing this paper as the origin of a Vigenère description is a stretch.
3. It doesn't match the convention it's modeled on. In caesar_description.yaml / atbash_description.yaml the authors/groups fields exist because the prose is copied verbatim from CipherChat — I diffed them against prompts_and_demonstrations.py and they're identical modulo line wrapping (e.g. "You are an expert on The Atbash Cipher. We will communicate in Atbash Cipher. Do not be a translator."). Those fields are marking borrowed text, not "related prior work."
The Vigenère prose below reads as newly written for this PR — which is fine and arguably better! But then authors/groups credit the writing to people who didn't write it.
Suggested fix (both fields are optional on SeedPrompt, defaulting to [], so they can just be removed):
name: vigenere cipher
description: |
A Vigenere cipher description for prepending.
source: https://arxiv.org/abs/2402.10601v1Pinning to v1 keeps the citation as prior art on cipher-based jailbreaks without silently drifting to a different paper and author list later. If the prose was in fact adapted from one of these papers, ignore all this — just pin the version and match the author list to it.
…onverter' into feature/vigenere-converter
…iation test _try_instantiate_converter fills required str params with a generic placeholder, which VigenereConverter rejects because its key must be alphabetic. Add an override like the other converters with validated required params. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4e67eb0e-8f79-4d4f-8023-e3f0da61cab7

Description
Adds
VigenereConverter, a classical polyalphabetic cipher converter that generalizesCaesarConverterusing a repeating keyword instead of a fixed shift. Closes #2304.Follows the existing
CaesarConverter/AtbashConverterstructure:__init__(self, *, key: str, append_description: bool = False), validates the key is non-empty and ASCII alphabetic_build_identifier()returns aComponentIdentifierwithkeyas a paramconvert_asyncencodes the prompt, with the sameappend_descriptionbehavior renderingvigenere_description.yamlvigenere_description.yamlcites Handa et al. (arXiv:2402.10601), the paper specifically covering word substitution cipher jailbreaking, rather than the CipherChat paper cited by Caesar/Atbash, since that paper doesn't cover Vigenère.Per the discussion on #2304, this does not touch
pyrit/scenario/scenarios/garak/encoding.py(garak has no corresponding probe) orFoundryTechnique(will be picked up when Foundry's converter set is refreshed separately).Tests and Documentation
New
tests/unit/converter/test_vigenere_converter.py, 13 tests covering basic encoding, case preservation, key case insensitivity, non-alphabetic passthrough (including non-ASCII alphabetic characters, which was a real bug caught during development, see note below), wraparound,append_description, and invalid key handling.Added
VigenereConverterto the existing parametrized fixtures intests/unit/converter/test_converter.pyalongside Caesar/Atbash.Verified against
tests/unit/registry/test_converter_registry.py, confirms the converter is discovered, correctly classified as non-LLM-based, and buildable through the registry.pyrit/converter/__init__.pyupdated with the import and__all__export.doc/code/converters/1_text_to_text_converters.pyupdated with a demo line. Ran withjupytext --execute --to notebook doc/code/converters/1_text_to_text_converters.py, all cells including the new one execute cleanly.Implementation note: the initial version used
str.isalpha()to detect letters, which returnsTruefor non-ASCII characters (accented letters, etc.) not present in the cipher alphabet, causing a crash. Fixed by checking ASCII letter membership explicitly. Caesar and Atbash avoid this because they usestr.translate(), which passes through unmapped characters silently.