Skip to content

Fix Information Gain Calculation in Decision Tree Algorithm #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ElmekaouiHaitham wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Fix Information Gain Calculation in Decision Tree Algorithm #2

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...d Learning Algorithms/Week 4 - Decision Trees/w4-ch2-Decision-trees-learning.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ $$H(0.5)- \left(\frac{4}{10}0.81 + \frac{6}{10}0.92\right) = 0.12$$
- Finally, $p_1$ of _root node_ as $p_1^{root}$.
- And, our formula will be:

$$\text{Information Gain } = H(p_1^{root})\left(w^{left}p_1^{left} + w^{right}p_1^{right}\right)$$
$$\text{Information Gain } = H(p_1^{root})\left(w^{left}H(p_1^{left}) + w^{right}H(p_1^{right})\right)$$

> So, _Information gain_ is then defined as the _entropy_ of $p_1^{root}$, so what's the _entropy_ at the root node, minus that _weighted entropy_ calculation that we had on the previous slide, minus $w^{left}$ those were 5/10 in the example, times the _entropy_ applied to $p_1^{left}$, that's _entropy_ on the left sub-branch, plus $w^{right}$ the fraction of examples that went to the right branch, times _entropy_ of $p_1^{right}$. ~ _Andrew Ng_

Expand Down