v0.9.0.14

[leondz]

What's Changed

New plugins

• BEAST Attack Implementation by @erickgalinkin in BEAST Attack Implementation #728
• add probe for "grandma" attack by @leondz in add probe for "grandma" attack #740
• generator: lorem ipsum by @leondz in generator: lorem ipsum #776

New features

• Feature: configurable plugins by @jmartin-tech in Feature: configurable plugins #711
• Add JSON Path support for RestGenerator by @leondz in Add JSON Path support for RestGenerator #651
• Make banner line test dynamic by @jmartin-tech in Make banner line test dynamic #670
• Tutorial for contributing a generator by @leondz in Tutorial for contributing a generator #688
• add 'complete' message to report at end of run by @leondz in add 'complete' message to report at end of run #735
• support chat modality in attempts by @leondz in support chat modality in attempts #644
• add grandma substance synth probe, update docs and var names by @leondz in add grandma substance synth probe, update docs and var names #763
• update NVCF to support versioning, custom payload keys by @leondz in update NVCF to support versioning, custom payload keys #772
• WINDOWS & OSX SUPPORT: test by default on linux, osx, win by @leondz in test by default on linux, osx, win #793
• enable nested setting of plugin configs by @leondz in enable nested setting of plugin configs #773
• always log nvcf payload by @leondz in always log nvcf payload #778
• update ecoji for windows support by @jmartin-tech in update ecoji for windows support #787
• remove extraneous default params for nims that expect conservative pa… by @leondz in remove extraneous default params for nims that expect conservative pa… #749
• require probes to match input modality of the generator by @jmartin-tech in require probes to match input modality of the generator #738
• Improve cloze data for leakreplay probe by @leondz in Improve cloze data for leakreplay probe #708
• add CONTRIBUTING.md, update contributing guide in docs by @leondz in add CONTRIBUTING.md, update contributing guide in docs #706
• add DEFAULT_PARAMS to base types that are configurables by @leondz in add DEFAULT_PARAMS to base types that are configurables #771
• atkgen - add configurability, allow config of which adversarial generator to use by @leondz in atkgen - add configurability, allow config of which adversarial generator to use #777
• enhancement: enable lazy setting of nested dicts by @leondz in enhancement: enable lazy setting of nested dicts #775
• convert mutation_generator to use load_plugin by @jmartin-tech in convert mutation_generator to use load_plugin #795
• Add support for gpt-4o, gpt-4o-mini by @erickgalinkin in Add support for gpt-4o, gpt-4o-mini #798
• bump transformers ver to min reqd for llama 3.1 by @leondz in bump transformers ver to min reqd for llama 3.1 #800
• add mitigation strings, add test for string-based mitigation skipping by @leondz in add mitigation strings, add test for string-based mitigation skipping #762
• speedup report generation by using class attribs instead of instantiating by @leondz in speedup report generation by using class attribs instead of instantiating #707
• cached plugin enum by @jmartin-tech in cached plugin enum #768

Tuning & fixes

• add some actionable opinions about linting by @leondz in add some actionable opinions about linting #668
• handle target model returning None in atkgen by @leondz in handle target model returning None in atkgen #678
• bump & fix black version by @leondz in bump & fix black version #686
• Bump/black 24.4.2 -- and apply formatting by @leondz in Bump/black 24.4.2 -- and apply formatting #687
• convert module-level constant names to uppercase by @leondz in convert module-level constant names to uppercase #685
• Guard cli run with invalid or incomplete config by @jmartin-tech in Guard cli run with invalid or incomplete config #677
• Catch generic exception from transformers in huggingface by @jmartin-tech in Catch generic exception from transformers in huggingface #690
• stablize openai parallel by @jmartin-tech in stablize openai parallel #689
• rest generator response must be a list by @jmartin-tech in rest generator response must be a list #693
• Set generator _call_model() and _generate() type hints; amend functions by @leondz in Set generator _call_model() and _generate() type hints; amend functions #694
• typo in racial slur prompt by @leondz in typo in racial slur prompt #697
• Generator streamlining, docs by @leondz in Generator streamlining, docs #682
• Split NVCF generator into completion chat by @leondz in Split NVCF generator into completion chat #696
• Update the quality taxonomy by @leondz in Update the quality taxonomy #703
• generator & detector structure validation by @leondz in generator & detector structure validation #702
• doc & inheritance fixes to DanInTheWildMini by @leondz in doc & inheritance fixes to DanInTheWildMini #712
• Remove support for openai api v0 by @leondz in Remove support for openai api v0 #718
• restore req_template_json_object lost in refactor by @jmartin-tech in restore req_template_json_object lost in refactor #729
• utilize GarakException in cli by @jmartin-tech in utilize GarakException in cli #730
• ensure report is printed in utf-8 for Windows by @jmartin-tech in ensure report is printed in utf-8 for Windows #732
• Linting - 20240612 by @leondz in Linting - 20240612 #733
• add garak preprint paper by @leondz in add garak preprint paper #739
• only set generations if still default by @jmartin-tech in only set generations if still default #741
• return value from batch_decode is already a list by @jmartin-tech in return value from batch_decode is already a list #743
• Handle openai 400 errors more gracefully by @leondz in Handle openai 400 errors more gracefully #744
• Refactor huggingface config support by @jmartin-tech in Refactor huggingface config support #742
• rename class level probe doc_uri by @jmartin-tech in rename class level probe doc_uri #759
• retry openaicompatible requests if invalid content received by @leondz in retry openaicompatible requests if invalid content received #761
• add colour theme config by @leondz in add colour theme config #769
• reduce zalgo-ness by @leondz in reduce zalgo-ness #770
• move REST generator exception into garak.exceptions by @leondz in move REST generator exception into garak.exceptions #779
• defensive coding: allow python generators more places by @leondz in defensive coding: allow python generators more places #782
• stop encoding.payloads accumulating each load by @leondz in stop encoding.payloads accumulating each load #780
• correct sorting for set json in plugin_cache.json by @jmartin-tech in correct sorting for set json in plugin_cache.json #794
• corrections to config format for TAP internal generators by @jmartin-tech in corrections to config format for TAP internal generators #796
• add missing comma, extra test case in probes.test.Test by @leondz in add missing comma, extra test case in probes.test.Test #797
• Bugfix/leakreplay output setting by @leondz in Bugfix/leakreplay output setting #790

Full Changelog: v0.9.0.13...v0.9.0.14

---

This discussion was created from the release v0.9.0.14.