Skip to content

Commit

Permalink
Update numerical-stability-and-init.md
Browse files Browse the repository at this point in the history
  • Loading branch information
astonzhang authored Jan 9, 2023
1 parent ac0e460 commit efd2940
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,7 @@ $$
通常,Xavier初始化从均值为零,方差
$\sigma^2 = \frac{2}{n_\mathrm{in} + n_\mathrm{out}}$
的高斯分布中采样权重。
我们也可以利用Xavier的直觉来选择从均匀分布中抽取权重时的方差
我们也可以将其改为选择从均匀分布中抽取权重时的方差
注意均匀分布$U(-a, a)$的方差为$\frac{a^2}{3}$。
将$\frac{a^2}{3}$代入到$\sigma^2$的条件中,将得到初始化值域:

Expand Down

0 comments on commit efd2940

Please sign in to comment.