v2.2.0: Easy to add any HF Transformers style model! DeBERTa added.

Making it easy to add a Hugging Face-style Transformers model

We refactored jiant to make it easier to add a Transformers-style model to the library. Please see the guide to add a model for more details. We added DeBERTa V2 as part of these changes.

Breaking Changes

The simple API now uses hf_pretrained_model_name_or_path instead of model_type as an argument. hf_pretrained_model_name_or_path is used as an input to Hugging Face's Auto Classes.

Features

de5437a Merge easy_add_model feature branch (#1309)
56ceae5 Updating notebooks, removing model_type (#1270)
723786a Switch export_model to use AutoModel and AutoTokenizer (#1260)
84f2f5a Adding Acceptability judgment and SentEval tasks (#1271)
f796e5a improve robustness of the simple runscript (#1307)

Tests

4d0f6a9 Add test matrix (#1308)

Bugfixes

ee65662 Update README.md
65888b4 Benchmark script fixes (#1301)
b4b5de0 Assert spans <= max_seq_len (#1302)
5ba72f7 axg->axb fix (#1300)
235f646 MNLI diagnostic example bug (#1294)

Maintenance

4ab0c08 Bump lxml from 4.6.2 to 4.6.3 (#1304)
741ab09 Documentation + cleanup (#1282)
dbbb4e6 export_model tweak (#1277)

Add downloaders for ARCT, MCTest, MCTACO, MuTual, and QuAIL

Tasks

e9d6c68 Adding download code for ARCT, MCTest, MCTACO, MuTual, and QuAIL (#1258)

Examples

a88956a Add edge probing notebook (#1261)

Bugfixes

d5e3b2e Prevents computing loss in multigpu when not available (#1257)
9cfe644 Truncate MCQ inputs from the start and not end (#1256)
33c4a26 Bump lxml from 4.5.1 to 4.6.2 (#1263)

Tests

9a45712 Add test for export_model (#1259)

Add ROPES and RACE tasks

Tasks

1ab34a4 Adding ROPES, RACE tasks (#1234)

Bugfixes

ce62495 Add simple save model test + fix (#1227)
e5fbea4 fix create examples wsc (#1247)
605d794 committing quail fix (#1249)
192d6b5 Namespace default cache dir by model_type. (#1246)

SQuAD Tokenization Fix, Load (only) Encoder Weights

Bugfixes

838cdd2 SQuAD tokenization update (#1232)
5329c7e Winogrande Task Property (#1229)
50b0116 Further fix for encoder_only (#1242)
1f66050 Allow force_overwrite to override "done" condition (#1241)
59438ed change checkpoint save default (#1233)
e2e85c9 guids_fix (#1225)
74c6ba0 fix notebooks path (#1231)

Features

18c41fc Load only encoder weights (#1240)

Documentation

711c6c8 Introduction filename correction (#1239)
f340d04 minor typo fix (#1238)

WinoGrande, FEVER, QuAIL, MCTest, MCTACO

Added Tasks

e4f1c4b Winogrande (#1203)
e7eefc6 Fever NLI task and data downloader (#1215)
cb601cf Quail (#1208)
c00360f MCTest and MCTACO (#1197)
76e2826 Mcscript Task Property (#1219)

Documentation

9892766 Add docs for adding tasks to data downloader (#1221)

Bugfixes

cb7ee4a Fix save-last behavior (#1220)

Cleanup

0cc8cbb use task_name instead of task.name (#1224)

PIQA, MRQA, NewsQA, Quoref, MCScript, ARCT

Added Tasks

442a2b0 - piqa (#1216) (William Huang)
c535e78 - Natural Questions (MRQA), NewsQA, Quoref (#1207) (Jason Phang)
d1b14c1 - mcscript (#1152) (William Huang)
da7550d - Adding arc_easy, arc_challenge, mutual, mutual_plus (#1206) (yzpang)
f4bca4e - add arct task doc documentation (#1154) (jeswan)
b23c0f7 - arct (#1151) (William Huang)
58beb8f - anli download (#1147) (Jason Phang)

Features

0b3dff5 - Adding ability to resume run in Simple API (#1205) (Jason Phang)

Notebooks

b81254b - Fix git clone in example notebooks (#1155) (jeswan)

Bugfixes

aa4d111 - Bugfix for single-task configurator (#1143) (Jason Phang)
14fac1c - Fix colab link in README (#1142) (Jonathan Chang)
02bb070 - setup.py fix (#1141) (Jason Phang)
7d1cc29 - Adding SingleTaskConfigurator, some cleanup (#1135) (Jason Phang)

Maintenance

bump torch>=1.5.0. bump transformers==3.1.0. notebook installation switched to local pip install. (#1218) (jeswan)
b20f30a - resolve_is_lower_case fix (#1204) (Jason Phang)
5724fee - Adjust case for span prediction (#1201) (Jason Phang)
c3387a3 - nlp to datasets (#1137) (Jason Phang)
04bbb39 - update issue numbers from jiant-dev to jiant transfer (#1196) (jeswan)
392976c - Task tweaks (#1149) (Jason Phang)
82ed396 - use hidden_size (#1148) (Jason Phang)

v2.0.0

Highlighted changes:

• jiant 2.0 is a complete rewrite of jiant, built natively on transformers and nlp/datasets
• Support for 50+ natural language understanding task, including the GLUE, SuperGLUE and XTREME benchmarks
• Support for BERT, RoBERTa, ALBERT, XLM-R, etc
• Includes data-downloading, tokenize-and-caching, training-and-evaluation code, and benchmark submission code for relevant tasks

v1.3.2

Highlighted changes:

New Tasks

• Masked Language Modeling (for RoBERTa and ALBERT) (#1030) (@pruksmhc and @phu-pmh)
• Sentence Order Prediction (for ALBERT) (#1061) (@pruksmhc and @phu-pmh)

Minor changes and fixes

• Fixed target training data fraction bug where target training data fraction was not reflected in logging and scheduler (#1071) (@HaokunLiu)
• Fixed target train data fraction overwriting pretrain data fraction bug (#1070) (@pyeres)
• Added CONTRIBUTING.md (#1036, #1038) (@pyeres)

Dependency changes

• transformers 2.3.0 → transformers 2.6.0 (#1059) (@zphang)

v1.3.1

Minor changes and fixes

• Fixed QAMR and QASRL tasks (#1019) (@pyeres)
• Changed tasks names using underscores to use hyphens (#1016) (@HaokunLiu)
• Fixed cola inference script (#1023) (@zphang)
• Re-ordered GPT-style inputs for consistency with GPT paper (#1031) (@HaokunLiu)
• Fixed edge probing and Senteval tasks (#1025) (@pruksmhc)

v1.3.0

Highlighted changes:

New Tasks

• QA-SRL (#716) (@zphang)
• QAMR (#932) (@zphang)
• Abductive NLI (aNLI) (#922) (@zphang)
• SocialIQA (#924) (@pruksmhc)
• SentEval Probing (#926) (@pruksmhc)
• SciTail (#943) (@phu-pmh)
• CommonsenseQA (#942) (@HaokunLiu)
• HellaSwag (#942) (@HaokunLiu)
• Acceptability probing (#949) (@HaokunLiu)
• Adversarial NLI (#966) (@pyeres)
• Two-class MNLI variant (#976) (@sleepinyourhat)
• WinoGrande (#996) (@HaokunLiu)

New Models

• ALBERT (#990) (@HaokunLiu)

New Features

• Faster retokenization (#935) (@pruksmhc)
• Gradient accumulation option (#980) (@pyeres)
• Full/stable data-parallel multi-GPU support (#992) (@pruksmhc)

Minor changes and fixes

• Fixed bug in restoring checkpoints in multi-GPU mode (#928) (@pruksmhc)
• Fixed bugs in RoBERTa retokenization (#982) (@HaokunLiu) and ids (#959) (@njjiang)
• Fixed load_target_train_checkpoint with mixing setting (#960) (@pruksmhc)
• Fixed bug in CCG loss function that artificially reduced accuracy (#948) (@HaokunLiu)
• Fixed label parsing for QQP (#956) (@zphang)
• Updated CoLA inference script (#931) (@zphang)

Dependency changes

• PyTorch 1.0.0 → 1.1.0 (#965) (@pyeres)
• Numpy 1.14.5 → 1.15.0 (#965) (@pyeres)
• pytorch-transformers 1.2.0 → transformers 2.3.0 (#990) (@HaokunLiu)