CompatHelper: bump compat for ExponentialFamilyManifolds to 3, (keep existing compat)

[github-actions]

This pull request changes the compat entry for the ExponentialFamilyManifolds package from 2.0.0 to 2.0.0, 3.
This keeps the compat entries for earlier versions.

Note: I have not tested your package with this new compat entry.
It is your responsibility to make sure that your package tests pass before you merge this pull request.

[bvdmitri]

@Nimrais is there any updates on this?

[Nimrais]

@bvdmitri I am super busy right now with another task. My hands will become free in a week or two. So if there is any chance to separate this work among more than one dev, I can describe what should be done to fix this.

The main problem is that we did tricks in the ExponentialFamilyManifolds.jl (EFM) to cancel out some mistakes here. Now these mistakes are fixed, but we our tricks right does not work. Basically, our gradients were slightly off, and there were many actual problems with that (like any Conjugate Gradients algorithms underperformed). There is a super easy fix which is to just replace our manual computation with ManifoldDiff, but I do not want to go this way because it's slow, and generally our gradients have a closed form - they just need to be corrected according to the last changes in EFM. However, this work is hard in the sense that someone needs to sit with a pen and correct the gradients.

What I would really like to do is the following: to have a benchmark test that our performance is not degraded, implement a fix that will pass all unit tests, and then optimize it with hand-crafted gradients. The benchmarking part is just Julia coding, so I think anyone can do it.

[bvdmitri]

@Nimrais I’m not sure I’m following — I’m not aware of any mistakes we made by cross-covering with two packages.
From what I previously understood, the changes in EFM were mainly a slight refactoring to better “align with math,” rather than a real change in functionality or a bug fix. AFAIR, we didn’t even touch most of the tests in EFM — we actually added more.
So, from a user perspective (or from the EFP perspective), the new version didn’t change much and should supposedly work the same?

I can understand if slight changes in EFM require slight changes in EFP, but I don’t see how that would justify a complete overhaul of all hand-written gradients — unless I’m misunderstanding your point.

our gradients have a closed form - they just need to be corrected according to the last changes in EFM

Are you referring to the gradients implemented in ExponentialFamily.jl?
I don’t quite get how the validity of our hand-written gradients could be affected by the internals of another library. As far as I know, those gradients were written for exponential family members in their natural parametrization — not specifically for the EFM library.
Also, those gradients aren’t even part of the projection package.

Basically, our gradients were slightly off

Where exactly?

(like any Conjugate Gradients algorithms underperformed)

Could you clarify what you mean here?

[Nimrais]

I’m not sure I’m following — I’m not aware of any mistakes we made by cross-covering with two packages.
From what I previously understood, the changes in EFM were mainly a slight refactoring to better “align with math,” rather than a real change in functionality or a bug fix. AFAIR, we didn’t even touch most of the tests in EFM — we actually added more.
So, from a user perspective (or from the EFP perspective), the new version didn’t change much and should supposedly work the same?

The main assumption in this package was that we can treat a point on the manifolds from EFM always as a natural parameter, right now this assumption is broken. We have quite some code written based on this assumption. For example, this line https://github.com/ReactiveBayes/ExponentialFamilyProjection.jl/blob/main/src/manopt/projection_objective.jl#L36.

Are you referring to the gradients implemented in ExponentialFamily.jl?
I don’t quite get how the validity of our hand-written gradients could be affected by the internals of another library. As far as I know, those gradients were written for exponential family members in their natural parametrization — not specifically for the EFM library.
Also, those gradients aren’t even part of the projection package.

I am not talking about the gradients in Exponential Family, which are correct. I am talking about the compute_gradient! function. Right now this function is not producing the correct gradient with respect to p. I am referring to this line: https://github.com/ReactiveBayes/ExponentialFamilyProjection.jl/blob/main/src/manopt/projection_objective.jl#L37. Right now it's not an identity operation, so we need to account for it in the gradient with respect to p.

Could you clarify what you mean here?

Before when I tried to use Conjugate Gradient with EFM manifolds it always diverged now it works.

[Nimrais]

P.S.
moreover if https://github.com/ReactiveBayes/ExponentialFamilyProjection.jl/blob/main/src/manopt/projection_objective.jl#L37 is the identity, the package is working correctly - for example, for Bernoulli tests are passing.

[bvdmitri]

@Nimrais ping

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (008b307) to head (95df7de).

Additional details and impacted files

@@            Coverage Diff            @@
##              main       #55   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            7         8    +1     
  Lines          278       324   +46     
=========================================
+ Hits           278       324   +46     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.