feature: lightweight probe defaults #1116

leondz · 2025-02-26T21:49:41Z

Resolves #1032

Cap probes at a certain number of requests for standard version, full version can be present but inactive
Make lightweight probes the default, moving larger probes out to -Full versions
Add a config value to set a suggested cap on number of prompts per default probe
Migrate many probes to use random shuffling + this cap when reducing probe count (shuffling preferred because a. we're not a benchmark, we're about discovery; b. getting variance is good, we don't want to overfit to subsets of the test cases)
Add fixers for renames

(NB Getting some black churn)

Verification

garak --list_probes reveals no -Mini or -80 (etc) probe names
garak --list_probes shows all -Full probes are inactive
garak -m test -g 1 shows no probe having over 256 prompts (config.run.soft_probe_prompt_cap) - calculations for 256 available on request
altering config.run.soft_probe_prompt_cap changes #prompts for affected probes

… class

…ull versions with special names

…able

garak/resources/fixer/__init__.py

garak/resources/fixer/20250224_lightweight_probe_defaults.py

erickgalinkin

Largely looks good. I think this is good to merge but would like to resolve the discussion @jmartin-tech brought up and my minor quibble with the DanInTheWildMini docstring.

erickgalinkin · 2025-02-28T13:33:11Z

garak/configs/fast.yaml

@@ -7,7 +7,7 @@ run:
  generations: 5

 plugins:
-  probe_spec: continuation,dan,encoding.InjectBase64,encoding.InjectHex,goodside,av_spam_scanning,leakreplay,lmrc,malwaregen.SubFunctions,malwaregen.TopLevel,packagehallucination,realtoxicityprompts.RTPIdentity_Attack,realtoxicityprompts.RTPProfanity,realtoxicityprompts.RTPSexually_Explicit,realtoxicityprompts.RTPThreat,snowball,xss
+  probe_spec: ansiescape.AnsiRaw,continuation,dan,encoding.InjectBase64,encoding.InjectHex,goodside,av_spam_scanning,leakreplay,lmrc,malwaregen.SubFunctions,malwaregen.TopLevel,packagehallucination,realtoxicityprompts.RTPIdentity_Attack,realtoxicityprompts.RTPProfanity,realtoxicityprompts.RTPSexually_Explicit,realtoxicityprompts.RTPThreat,snowball,xss


Do we still want av_spam_scanning in the default fast config? It's largely useless for model-only evaluation.

Good question. garak's mostly been used for scanning models so far, but this isn't an intended constraint on use. I appreciate the opportunity to have a "we should be scanning systems" talk whenever this module comes up. On the other hand, it takes time to have that talk.

garak/probes/continuation.py

garak/probes/dan.py

garak/probes/latentinjection.py

garak/resources/fixer/__init__.py

garak/resources/garak.core.yaml

jmartin-tech

This PR assumes config_root is always the global module instance _config. An enhancement to enable _config.run items to be distributed in a consistent way for implementers of Configurable is planned and will be needed for this PR.

garak/probes/continuation.py

Signed-off-by: Jeffrey Martin <[email protected]>

* plugins define what run and system params to support * parallel requests is configurable for genertors instantiated by plugins Signed-off-by: Jeffrey Martin <[email protected]>

Signed-off-by: Jeffrey Martin <[email protected]>

leondz

lgtm

garak/configurable.py

garak/generators/base.py

garak/generators/ggml.py

garak/generators/octo.py

garak/probes/fileformats.py

jmartin-tech

One minor improvement might be to consolidate _prune_data() as that code in that method looks to be duplicated in a few classes.

garak/resources/fixer/_plugin.py

garak/probes/fileformats.py

garak/resources/fixer/__init__.py

jmartin-tech · 2025-03-04T17:32:04Z

A separate note, this PR suggests that the idea of tracking renames with aliases in the plugin may be something we could remove for now. Hence I don't think we need to add them here.

Signed-off-by: Leon Derczynski <[email protected]>

erickgalinkin

LGTM. Only thought: Do we want to add a warning that these probes have been renamed? If we detect the newly updated names (specifically, lots of folks like DanInTheWildMini) in the config, inform them that we've done the rename? I suppose this is something that can be captured in the release notes, but just trying to minimize support requests.

leondz · 2025-03-07T15:24:57Z

Do we want to add a warning that these probes have been renamed?

Kinda. We don't want to say nothing! I'm leaning towards only putting this in the release notes seeing as we're 0.x - the other question, if we do more than that, would be for how long we hold the message around for. Which is more maintenance.

* defensive coding for _plugin rename util * use `raw` string for regex * add tests for minimal and regex paths Signed-off-by: Jeffrey Martin <[email protected]>

jmartin-tech · 2025-03-07T22:01:00Z

A note from testing, likely worth a new issue. The fixer migrations output the yaml with sorted keys, making comparison less than idea.

leondz · 2025-03-10T13:11:17Z

A note from testing, likely worth a new issue. The fixer migrations output the yaml with sorted keys, making comparison less than idea.

Oh, that's.. cute. Yeah, I can see the advantage to preserving input YAML order on some levels. On the other hand, if we're dealing with items under plugins, run, system etc - maybe this is a fine convention to adopt?

leondz · 2025-03-11T17:00:45Z

prune refactor sent, + test added. I think that may be three LGTMs..

leondz added 16 commits February 24, 2025 11:32

stabilise and make explicit order of multiple Migrations in one fixer…

640d304

… class

update FigStep names

e7c7db5

rename and fixers for snowball

a85c257

add config entry for soft cap on how many prompts per probe

4370779

rename promptinject probes & bind to soft probe prompt cap

169d481

migrate past tense probe names

2e40865

resolve merge

829f97f

rename probes to have lightweight versions as defaults and extended/f…

ea5bed8

…ull versions with special names

shrink LatentInjectionFactSnippetEiffel to soft cap, w/ shuffle

79105cb

rename FalseAssertion, Glitch, use soft cap

3aa6677

fix rename

3b3e786

get order of operations right: set max_prompts after _config is avail…

88411ab

…able

lightweight defaults for latent injection probes

3f03bc2

use random shuffle + prune for lightweight slur continuation

0375e52

move to using shuffling & prompt cap to produce lightweight probes

02d202a

access config_root not _config

7310811

leondz added the probes Content & activity of LLM probes label Feb 26, 2025

leondz added this to the 25.02 Efficiency milestone Feb 26, 2025

leondz requested review from jmartin-tech and erickgalinkin February 26, 2025 21:49

leondz changed the title ~~lightweight probe defaults~~ feature: lightweight probe defaults Feb 26, 2025

leondz marked this pull request as draft February 26, 2025 21:59

jmartin-tech reviewed Feb 26, 2025

View reviewed changes

garak/resources/fixer/__init__.py Show resolved Hide resolved

leondz added 3 commits February 27, 2025 16:25

fixer class sorting should.. work

e28f8c0

update test cases to fit current state of class names

f45938b

constrain class replacement to final position in plugin name

0163a4c

jmartin-tech reviewed Feb 27, 2025

View reviewed changes

garak/resources/fixer/20250224_lightweight_probe_defaults.py Outdated Show resolved Hide resolved

place migrations involving ordered ops into single classes. much tidier

f5d168a

erickgalinkin approved these changes Feb 28, 2025

View reviewed changes

jmartin-tech requested changes Feb 28, 2025

View reviewed changes

garak/probes/continuation.py Outdated Show resolved Hide resolved

jmartin-tech and others added 5 commits March 3, 2025 08:46

change test to use renamed class

44aebe1

Signed-off-by: Jeffrey Martin <[email protected]>

consolidate system and run distribution in Configurable

2e45b31

* plugins define what run and system params to support * parallel requests is configurable for genertors instantiated by plugins Signed-off-by: Jeffrey Martin <[email protected]>

manual garak/resources/plugin_cache.json update

e5762c7

Signed-off-by: Jeffrey Martin <[email protected]>

ensure config is loaded when running test module in isolation

7b95a12

rm unused import, prune redundant param

f642a45

leondz commented Mar 4, 2025

View reviewed changes

garak/configurable.py Show resolved Hide resolved

garak/generators/base.py Show resolved Hide resolved

garak/generators/ggml.py Show resolved Hide resolved

garak/generators/octo.py Show resolved Hide resolved

garak/probes/fileformats.py Outdated Show resolved Hide resolved

leondz marked this pull request as ready for review March 4, 2025 10:14

rm generations set in fileformats probe

9561700

jmartin-tech approved these changes Mar 4, 2025

View reviewed changes

garak/resources/fixer/_plugin.py Outdated Show resolved Hide resolved

garak/probes/fileformats.py Outdated Show resolved Hide resolved

garak/resources/fixer/__init__.py Show resolved Hide resolved

Merge branch 'main' into feature/lightweight_probe_defaults

257bf88

Signed-off-by: Leon Derczynski <[email protected]>

erickgalinkin approved these changes Mar 7, 2025

View reviewed changes

update DanInTheWild doc

650c18e

allow minimal config passed to fixer

662f905

* defensive coding for _plugin rename util * use `raw` string for regex * add tests for minimal and regex paths Signed-off-by: Jeffrey Martin <[email protected]>

refactor _prune method in probes.base

83c89e2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature: lightweight probe defaults #1116

feature: lightweight probe defaults #1116

leondz commented Feb 26, 2025 •

edited

Loading

erickgalinkin left a comment

erickgalinkin Feb 28, 2025

leondz Mar 4, 2025

jmartin-tech left a comment

leondz left a comment

jmartin-tech left a comment

jmartin-tech commented Mar 4, 2025

erickgalinkin left a comment

leondz commented Mar 7, 2025

jmartin-tech commented Mar 7, 2025

leondz commented Mar 10, 2025

leondz commented Mar 11, 2025

feature: lightweight probe defaults #1116

Are you sure you want to change the base?

feature: lightweight probe defaults #1116

Conversation

leondz commented Feb 26, 2025 • edited Loading

Verification

erickgalinkin left a comment

Choose a reason for hiding this comment

erickgalinkin Feb 28, 2025

Choose a reason for hiding this comment

leondz Mar 4, 2025

Choose a reason for hiding this comment

jmartin-tech left a comment

Choose a reason for hiding this comment

leondz left a comment

Choose a reason for hiding this comment

jmartin-tech left a comment

Choose a reason for hiding this comment

jmartin-tech commented Mar 4, 2025

erickgalinkin left a comment

Choose a reason for hiding this comment

leondz commented Mar 7, 2025

jmartin-tech commented Mar 7, 2025

leondz commented Mar 10, 2025

leondz commented Mar 11, 2025

leondz commented Feb 26, 2025 •

edited

Loading