Skip to content

Commit 498445d

Browse files
authored
Change mermaid diagram to image (#3430)
1 parent d6a723c commit 498445d

File tree

2 files changed

+1
-24
lines changed

2 files changed

+1
-24
lines changed

sdk/python/foundation-models/system/distillation/README.md

Lines changed: 1 addition & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -32,30 +32,7 @@ During distillation, a smaller LLM "student" learns from a larger LLM "teacher".
3232

3333
The process of model distillation is a two stage process as seen below.
3434

35-
```mermaid
36-
sequenceDiagram
37-
participant TM as Teacher Model
38-
participant D as Dataset (Training + Validation)
39-
participant SM as Student Model
40-
41-
42-
D->>TM: Generate Training Responses
43-
activate TM
44-
TM-->>D: Responses
45-
deactivate TM
46-
47-
activate D
48-
D->>TM: Generate Validation Responses
49-
deactivate D
50-
activate TM
51-
TM-->>D: Responses
52-
deactivate TM
53-
54-
note over D: Datasets Complete
55-
56-
D->>+SM: Finetune
57-
58-
```
35+
![Model Distillation](../docs/images/model_distillation.png)
5936

6037
1. The first stage is the synthetic data generation step. In this step, using a training dataset, the teacher model is asked to generate responses for the training data. If there is a validation dataset, the teacher model also generates responses for that dataset as well.
6138
2. The second stage is finetuning. Once the synthetic data is collected, the student model is then finetuned off of the training and validation data created from the teacher model. This transfers the knowledge from the teacher model to the student model.
49.4 KB
Loading

0 commit comments

Comments
 (0)