[WIP] Relaxation Runge-Kutta #2227

Theozeud · 2024-05-27T20:20:13Z

Start of the implementation of relaxation Runge-Kutta methods (see #1029)

Instead of writing things with a callback as suggested in the issue, I prefer to propose, I think, a more suitable solution by decoupling perform_step! into several steps one of which will allow the user to do modifications on the computations of the method. I think the problem with the callback is when it occurs : in the loopfooter function where things that have already modified have been registered as dtnew, t, tprev and others things related to tstops, which could be changed by the relaxation methods.

For the moment, to test the new structure, I copy the Tsit5 method with another name Tsit5_for_relaxation. I will certainly change some parts of the code and its structure. I am not sure yet if the idea is good, but I think it is a good starting point.

To do

From the relaxation method point of view :

Implement a copy of Tsit5 with the new structure explained above,
Test if the code run without the relaxation,
Implement a structure to perform a relaxation and look for (a) good package(s) and method(s) to do the needed optimization step,
Implement FSAL-R,
See if it is possible to implement easily R-FSAL,
Look for suitable problems for the relaxation method,
Make the relaxation method running,
Make the relaxation method working,
Make benchmark.

From an interface point of view :

See which variables are really needed to be added to the structure integrator,
See if the splitting in three steps of the perform_step! function is a good choice or if it is better to do only two,
Investigate all things needed to meticulously take care of time as matching tstops ect,
Find out what need to be changed in DiffEqBase.jl if we want to add a solver option (because it looks like something needs to be changed)

Modif file : - integrators\type.jl + solve.jl: add u_propose, u_changed, dt_has_changed, dt_changed + initialisation - OrdinaryDiffEq.jl : add file and method - add cache - add a new file for performstep - add alg_order in alg.utils

…Eq.jl into relaxation

ChrisRackauckas · 2024-05-29T07:50:00Z

Alerting @ranocha who may be interested in keeping on top of this.

Theozeud · 2024-05-30T19:42:26Z

Two comments on the current implementation :

The new structure seems to work at least without any modification of the calculation by a user but there is a small bug with interpolation that I do not understand for the moment
I divided perform_step ! into three steps:
- computation!
- modify_step !
- finalize_step !
The first is to do the mathematical side of the method, the second is to allow the user to make modifications to the calculation and the solver to track it, and the last is to do all the work related to the DiffEq solver.
But it is not easy to divide the tasks between computation ! and finalize_step ! In particular, we need to reload some objects to do this, so I am not sure it is really necessary, except perhaps to recalculate fsal or things like that if u changes.

ranocha

Interesting development. If you want to achieve best performance with step size control, you can have a look at our recent preprint https://arxiv.org/abs/2311.14050

Feel free to ask me if you have any specific questions while working on this

JoshuaLampert · 2024-05-31T13:50:00Z

Regarding the step

Implement a structure to perform a relaxation and look for a good package/method to do the needed optimization step

in your TODO list above, I have an implementation of the relaxation step (there it is implemented as a callback) in https://github.com/JoshuaLampert/DispersiveShallowWater.jl/blob/a7eb1307169d1278f0c4e1afeae55b3e9a8cea64/src/callbacks_step/relaxation.jl, which might or might not help you.

Theozeud · 2024-05-31T14:00:16Z

Interesting development. If you want to achieve best performance with step size control, you can have a look at our recent preprint https://arxiv.org/abs/2311.14050

Feel free to ask me if you have any specific questions while working on this

Regarding the step

Implement a structure to perform a relaxation and look for a good package/method to do the needed optimization step

in your TODO list above, I have an implementation of the relaxation step (there it is implemented as a callback) in https://github.com/JoshuaLampert/DispersiveShallowWater.jl/blob/a7eb1307169d1278f0c4e1afeae55b3e9a8cea64/src/callbacks_step/relaxation.jl, which might or might not help you.

Ok, thanks to both of you, I will take a look!

…g time

Theozeud · 2024-06-11T20:02:31Z

Update on my work :

The new structure with relaxation runs and works partially. I need to fix things related to tstops matching.
I am reading the article mentionned by Hendrik Ranocha https://arxiv.org/abs/2311.14050 about better step size control.

Theozeud · 2024-06-11T20:51:31Z

@ChrisRackauckas, @ranocha I have a small question if someone can help me. I compare my implementation Tsit5_for_relaxtion with the existing Tsit5 on a simple problem without relaxation such that results must be the same. I encounter a small problem of interpolation which could be due to my code, but I wonder if it could be due to the fact I do not define _ode_interpolant for Tsit5_for_relaxation.

oscardssmith · 2024-06-12T03:39:28Z

I wonder if it could be due to the fact I do not define _ode_interpolant for Tsit5_for_relaxation.

that is the reason.

exponential entropy doesn't work

Relaxation new version

all last modif I forgot

oscardssmith · 2024-06-25T17:50:25Z

src/OrdinaryDiffEq.jl

@@ -77,6 +77,8 @@ import DiffEqBase: resize!, deleteat!, addat!, full_cache, user_cache, u_cache,
                   add_saveat!, set_reltol!,
                   set_abstol!, postamble!, last_step_failed,
                   isautodifferentiable
+
+export change_u!, change_dt!, apriori_bounds_dt


do these need to be exported?

oscardssmith · 2024-06-25T17:50:59Z

src/algorithms/explicit_rk.jl

+thread::Thread = False()
+end
+
+# for backwards compatibility


backwards compatibility with what?

oscardssmith · 2024-06-25T17:51:33Z

src/caches/low_order_rk_caches.jl

@@ -448,6 +448,44 @@ if isdefined(Base, :Experimental) && isdefined(Base.Experimental, :silence!)
    Base.Experimental.silence!(Tsit5Cache)
 end

+@cache struct Tsit5Cache_for_relaxation{uType, rateType, uNoUnitsType, StageLimiter, StepLimiter,


could this just use the Tsit5Cache?

oscardssmith · 2024-06-25T17:52:05Z

src/integrators/integrator_interface.jl

+    integrator.dt_changed = dt
+end
+
+function change_u!(integrator::ODEIntegrator, u)


should this set integrator.umodified?

oscardssmith · 2024-06-25T17:53:28Z

src/perform_step/test_for_relaxation_perform_step.jl

+    end
+
+    # Calculate f(uₙ₊₁) if the aglortihm has the FSAL property
+    if isfsal(integrator.alg)


won't this always be true?

oscardssmith · 2024-06-25T17:55:09Z

test/relaxation/benchmark in time.ipynb

@@ -0,0 +1,255 @@
+{


shouldn't the benchmark be in SciMLBenchmarks rather than OrdinaryDiffEq?

oscardssmith · 2024-06-25T17:56:59Z

The fact that this requires ~1000 lines to add relaxation to a single algorithm seems like it should be improved. It seems like we should be able to duplicate less of the steps between the regular solvers and the relaxation ones. Currently, it seems like the chance of us making improvements to the regular and those changes not getting reflected in the relaxation solver is pretty high.

Theozeud · 2024-06-25T18:37:11Z

The fact that this requires ~1000 lines to add relaxation to a single algorithm seems like it should be improved. It seems like we should be able to duplicate less of the steps between the regular solvers and the relaxation ones. Currently, it seems like the chance of us making improvements to the regular and those changes not getting reflected in the relaxation solver is pretty high.

Thanks for your review. In fact, it does not require as many lines at all. I have replicated the Tsit5 solver as Tsit5_for_relaxation so as not to "break" the initial solver and make comparisons. But what I had in mind would be to modify the perform_step! function for all the methods for which we can do relaxations, into a general perform_step! function for all the methods. Instead, each method would have a computation! function and other functions such as the calculation of error estimates. I am aware that this is a big change to the general solver, but I can't see any other way of including relaxation before the loopfooter! function. (So we wouldn't be able to use all the variants of relaxation, including the R-FSAL method). The other solution would be to add a condtion like "if relaxation do relaxation end"
inside each method, with a lot of copy and paste, but this does not seem viable for me in the long term, for example if there are other different methods than relaxations. I therefore propose this structure, which of course may not be desirable for reasons I am unaware of due to lack of experience, so I wanted to discuss it with you before going any further.

In the 1000 lines, there are several test files that I have made so that you can test them and see that the code works with relaxation, as well as a jupiter notebook that I should remove that shows that the code is not slowed down by the new perform_step! structure. For the tests, I have taken the examples I found in the https://arxiv.org/abs/2311.14050 article.

oscardssmith · 2024-09-03T20:44:24Z

Closing in favor of #2283

Theozeud added 19 commits April 26, 2024 23:00

just ideas for relaxation

fbf962d

Update ideas_for_relaxation.jl

dc25f21

Update ideas_for_relaxation.jl

575826a

Update ideas_for_relaxation.jl

cdb53c9

Update ideas_for_relaxation.jl

7e2b73a

try_implementation_tsit5_for_relaxation

d0f8a22

progress relaxation

229438f

Merge branch 'SciML:master' into relaxation

9384053

all implement but failed compilation

a796157

Modif file : - integrators\type.jl + solve.jl: add u_propose, u_changed, dt_has_changed, dt_changed + initialisation - OrdinaryDiffEq.jl : add file and method - add cache - add a new file for performstep - add alg_order in alg.utils

Merge branch 'relaxation' of https://github.com/Theozeud/OrdinaryDiff…

eeda5a7

…Eq.jl into relaxation

compilation works but not tests

b3617ad

Merge branch 'SciML:master' into relaxation

945b9d3

code change structure works

ef5238b

Merge branch 'relaxation' of https://github.com/Theozeud/OrdinaryDiff…

cb94460

…Eq.jl into relaxation

fix mistakes

4323a68

improvement for regression

12e4a7a

clean up

e35d9fa

Merge branch 'master' into relaxation_method

7fada13

try test without relaxation

653436b

ChrisRackauckas requested a review from ranocha May 29, 2024 07:49

copy tist5 for non constant cache

730d4e1

fix tsit5_for_relaxation with non constant cache

db28be6

ranocha reviewed May 31, 2024

View reviewed changes

Theozeud added 3 commits June 3, 2024 22:20

first try of implementation of the relaxation step

2ffd004

add notes on some thoughts

a180073

add some tools

52f4461

Theozeud added 5 commits June 7, 2024 23:49

add some tools

b4d9c40

bound_dt

a01bc2a

trying to use Roots.jl instead of OptimizationOptimJL.jl

3fd63fd

test relaxation on oscillator

e926a61

work at the beginning for harmonic oscillator but there is bug in lon…

2789e9f

…g time

add some problem for later

89c10f3

Theozeud added 2 commits June 11, 2024 23:28

some change in the modif_step!

2c387d8

fix bug

83bd06c

Theozeud added 14 commits June 12, 2024 23:57

first test to compare relaxation with no relaxation

8ff5be9

Create benchmark in time.ipynb

5138202

Test FSAL-R

c884132

add more examples

fa0d5c3

exponential entropy doesn't work

Relaxation v2

3287f82

New way with PerformStepCallBack

1111c38

new structure runs

44ea89d

Merge pull request #4 from Theozeud/relaxation_new_version

9af57f8

Relaxation new version

all last modif

3d06894

Merge pull request #5 from Theozeud/relaxation_new_version

f6557b4

all last modif I forgot

almost all work well except R-FSAL which needs

ac1943d

find a way to make R-FSAL work

5c9b8e6

do each examples of the paper with different FSAL

2b0fb0a

small fix

f208a53

oscardssmith requested changes Jun 25, 2024

View reviewed changes

oscardssmith closed this Sep 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Relaxation Runge-Kutta #2227

[WIP] Relaxation Runge-Kutta #2227

Theozeud commented May 27, 2024 •

edited

Loading

ChrisRackauckas commented May 29, 2024

Theozeud commented May 30, 2024 •

edited

Loading

ranocha left a comment

JoshuaLampert commented May 31, 2024

Theozeud commented May 31, 2024

Theozeud commented Jun 11, 2024 •

edited

Loading

Theozeud commented Jun 11, 2024

oscardssmith commented Jun 12, 2024

oscardssmith Jun 25, 2024

oscardssmith Jun 25, 2024

oscardssmith Jun 25, 2024

oscardssmith Jun 25, 2024

oscardssmith Jun 25, 2024

oscardssmith Jun 25, 2024

oscardssmith commented Jun 25, 2024

Theozeud commented Jun 25, 2024

oscardssmith commented Sep 3, 2024

[WIP] Relaxation Runge-Kutta #2227

[WIP] Relaxation Runge-Kutta #2227

Conversation

Theozeud commented May 27, 2024 • edited Loading

Start of the implementation of relaxation Runge-Kutta methods (see #1029)

To do

From the relaxation method point of view :

From an interface point of view :

ChrisRackauckas commented May 29, 2024

Theozeud commented May 30, 2024 • edited Loading

ranocha left a comment

Choose a reason for hiding this comment

JoshuaLampert commented May 31, 2024

Theozeud commented May 31, 2024

Theozeud commented Jun 11, 2024 • edited Loading

Theozeud commented Jun 11, 2024

oscardssmith commented Jun 12, 2024

oscardssmith Jun 25, 2024

Choose a reason for hiding this comment

oscardssmith Jun 25, 2024

Choose a reason for hiding this comment

oscardssmith Jun 25, 2024

Choose a reason for hiding this comment

oscardssmith Jun 25, 2024

Choose a reason for hiding this comment

oscardssmith Jun 25, 2024

Choose a reason for hiding this comment

oscardssmith Jun 25, 2024

Choose a reason for hiding this comment

oscardssmith commented Jun 25, 2024

Theozeud commented Jun 25, 2024

oscardssmith commented Sep 3, 2024

Theozeud commented May 27, 2024 •

edited

Loading

Theozeud commented May 30, 2024 •

edited

Loading

Theozeud commented Jun 11, 2024 •

edited

Loading