Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mixture of Linear Experts does not appear to use EM #1113

Open
dfd opened this issue Jul 10, 2023 · 1 comment
Open

Mixture of Linear Experts does not appear to use EM #1113

dfd opened this issue Jul 10, 2023 · 1 comment

Comments

@dfd
Copy link

dfd commented Jul 10, 2023

https://github.com/probml/pyprobml/blob/master/notebooks/book1/13/mixexpDemoOneToMany.ipynb

The random initialization of parameters and weights occurs within the E-Step (instead of before it), so it's not really making use of the fit parameters from the M-Step in each subsequent iteration.

      #E-step :
      np.random.seed(iteration)
      Wy = 0.1*np.random.randn(D, K)
      bias = 0.3*np.random.randn(D, K)
      mixweights = np.random.rand(1, K)
      normmw =  np.linalg.norm(mixweights)
      mixweights = mixweights/normmw
      sigma2 = 0.1*np.random.randn(1, K)
@berkott
Copy link

berkott commented Aug 30, 2023

I observed this same issue today. Running with max_iter = n and iteration = 0 gives the same result as max_iter = n and iteration = n-1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants