Skip to content

Commit

Permalink
Added Chapter 11 (Training LLMs)
Browse files Browse the repository at this point in the history
  • Loading branch information
MikeySaw committed May 15, 2024
1 parent 76bc59d commit f41920d
Show file tree
Hide file tree
Showing 3 changed files with 29 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: "Chapter 11.01: LLMs: Parameters, Data, Hardware, Scaling"
weight: 1101
---

In this chapter you will learn how to calculate the number of parameters in the Transformer, understand Transformer computation and memory load, learn about Flash Attentions and understand Scaling Laws and Chinchilla.

### Lecture Slides

{{< pdfjs file="https://github.com/slds-lmu/lecture_dl4nlp/blob/main/slides/chapter11-training-llms/111-compute_scaling_chinchilla.pdf" >}}
14 changes: 14 additions & 0 deletions content/chapters/11_training_llms/11_02_x_optimize.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
title: "Chapter 11.02: LLM Optimization"
weight: 1102
---

In this Chapter we discuss ways to optimize the performance of Large Language Models (LLMs) with methods such as Prompt engineering or methods beyond that.

### Lecture Slides

{{< pdfjs file="https://github.com/slds-lmu/lecture_dl4nlp/blob/main/slides/chapter11-training-llms/112-slides-x-optimize.pdf" >}}

### Additional Resources

- [Video by OpenAI about LLM Optimization](https://www.youtube.com/watch?v=ahnGLM-RC1Y)
5 changes: 5 additions & 0 deletions content/chapters/11_training_llms/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
title: "Chapter 11: Training Large Language Models"
---

In this chapter we cover multiple concepts that deal with training LLMs. You will learn about Transformer computation and scaling laws. In the second chapter we discuss how we can optimize LLM performance.

0 comments on commit f41920d

Please sign in to comment.