Skip to content

Commit

Permalink
fix: KL loss formula in course 7
Browse files Browse the repository at this point in the history
  • Loading branch information
Madjakul committed Dec 6, 2024
1 parent 048465b commit 8a4a42b
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions markdown/course7_advanced.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ Example:

### Part-of-Speech Tagging (POS)

We wish to predict an output vector $\textbf{y} = (y_{1}, y_{1}, ..., y_{L})$, of random variables, given an observed characteristic vector $\textbf{x} = (x_{1}, x_{2}, ..., x_{L})$
We wish to predict an output vector $\textbf{y} = (y_{1}, y_{2}, ..., y_{L})$, of random variables, given an observed characteristic vector $\textbf{x} = (x_{1}, x_{2}, ..., x_{L})$

$\textbf{y}$ takes it value from a list of $N$ possible values.

Expand Down Expand Up @@ -258,7 +258,7 @@ $$\mathcal{L}_{CE} = - \frac{1}{N} \sum_{n'=1}^{N}y^{(n)}.log(f(\textbf{x}, \the

$$\mathcal{L}_{BCE} = - y^{(n)}.log(f(\textbf{x}, \theta)^{(n)}) + (1 - y^{(n)}).(1 - f(\textbf{x}, \theta)^{(n)})$$

$$\mathcal{L}_{KL} = - \frac{1}{N} \sum_{n'=1}^{N}y^{(n)}.log(\frac{y^{(n)}}{f(\textbf{x}, \theta)^{(n)}})$$
$$\mathcal{L}_{KL} = \frac{1}{N} \sum_{n'=1}^{N}y^{(n)}.log(\frac{y^{(n)}}{f(\textbf{x}, \theta)^{(n)}})$$

---

Expand Down

0 comments on commit 8a4a42b

Please sign in to comment.