Merge branch 'main' of https://github.com/NathanGodey/AdvancedNLP

NathanGodey · Nov 8, 2024 · ee9a96a · ee9a96a
2 parents 619da75 + e0161f1
commit ee9a96a
Show file tree

Hide file tree

Showing 13 changed files with 8 additions and 8 deletions.
diff --git a/README.md b/README.md
@@ -6,7 +6,7 @@
 
 1. Recap on Deep Learning & basic NLP ([slides](https://github.com/NathanGodey/AdvancedNLP/raw/main/slides/pdf/course1_recap.pdf) / [lab session](https://colab.research.google.com/drive/1_QzQBdP289benS8Uo3yPQmtXoM-f80-n?usp=sharing))
 2. Tokenization ([slides](https://github.com/NathanGodey/AdvancedNLP/raw/main/slides/pdf/course2_tokenization.pdf) / [lab session](https://colab.research.google.com/drive/1xEKz_1LcnkfcEenukIGCrk-Nf_5Hb19s?usp=sharing))
-3. Language Modeling ([slides](https://github.com/NathanGodey/AdvancedNLP/raw/main/slides/pdf/course3_lm.pdf) / lab session)
+3. Language Modeling ([slides](https://github.com/NathanGodey/AdvancedNLP/raw/main/slides/pdf/course3_lm.pdf) / [lab session](https://colab.research.google.com/drive/1QmVOWC1oB206PmOBn8j0EF54laSh3BBd?usp=sharing))
 4. NLP without 2048 GPUs ([slides](https://github.com/NathanGodey/AdvancedNLP/raw/main/slides/pdf/course4_efficiency.pdf) / lab session)
 5. Language Models at Inference Time ([slides](https://raw.githubusercontent.com/NathanGodey/AdvancedNLP/main/slides/pdf/course5_inference.pdf) / lab session)
 6. Handling the Risks of Language Models ([slides](https://github.com/NathanGodey/AdvancedNLP/raw/main/slides/pdf/course6_risks.pdf) / lab session)

diff --git a/imgs/course3/T5_lm.svg b/imgs/course3/T5_lm.svg
diff --git a/markdown/course3_lm.md b/markdown/course3_lm.md
@@ -521,10 +521,10 @@ $$
 - A useful evaluation metric: ***Perplexity***
 - Defined as:
 $$
-ppl(T_{\theta}; w_1...w_n) = \sqrt[n]{\frac{1}{P_{\theta}(w_1...w_n)}}
+ppl(T_{\theta}; w_1...w_n) = \exp \left( -\frac{1}{n}\sum_{t=1}^{n}\log P_{\theta}(w_t | w_{<t}) \right)
 $$
 
-- Other metrics: accuracy, f1-score, ...
+- Other metrics: accuracy, MAUVE, ...
 
 ---
 ### Zero-shot evaluation

diff --git a/slides/course3_lm.html b/slides/course3_lm.html
diff --git a/slides/course5_inference.html b/slides/course5_inference.html
diff --git a/slides/pdf/course1_recap.pdf b/slides/pdf/course1_recap.pdf
diff --git a/slides/pdf/course2_tokenization.pdf b/slides/pdf/course2_tokenization.pdf
diff --git a/slides/pdf/course3_lm.pdf b/slides/pdf/course3_lm.pdf
diff --git a/slides/pdf/course4_efficiency.pdf b/slides/pdf/course4_efficiency.pdf
diff --git a/slides/pdf/course5_inference.pdf b/slides/pdf/course5_inference.pdf
diff --git a/slides/pdf/course6_risks.pdf b/slides/pdf/course6_risks.pdf
diff --git a/slides/pdf/course7_advanced.pdf b/slides/pdf/course7_advanced.pdf
diff --git a/slides/pdf/course8_specific.pdf b/slides/pdf/course8_specific.pdf