Optimal control interface

[ErikQQY]

Close: #336

The current implementation concatenates the inequality/equality(the upper part) and the loss function([loss_bc; loss_collocation] as the lower part) as the constraints in OptimizationProblem. The Jacobian is handled by concatenating the Jacobian of inequality constraints(the upper square matrix) and the Jacobian of the loss function ([jac_loss_bc; jac_loss_collocation] as the lower rectangular matrix), resulting in a (N*M + length(bc_resid_prototype) + length(f_prototype)*(N-1)) x (N*M) Jacobian. Then the Jacobian is, upper part: banded matrix; lower part: almost banded matrix.

There are some weird issues with formulating the state variables and control variables, which may prevent the solver from finding an optimal solution. I’ll see what’s going wrong here.

Some TODOs:

• Better ucons and lcons handling. More friendly information on choosing ucons and lcons should be provided to users.
• Better cost function definition. The cost function definition should allow for using the interpolations, e.g. cost(sol, p) = sol(pi)[2]-1.0. Maybe we can also allow integral expressions.

[github-actions]

Benchmark Results

Click to check benchmark results

	master	300667d...	master / 300667d...
Simple Pendulum/IIP/BoundaryValueDiffEqMIRK.MIRK2()	0.314 ± 0.019 s	0.313 ± 0.019 s	1.01 ± 0.088
Simple Pendulum/IIP/BoundaryValueDiffEqMIRK.MIRK3()	9.86 ± 0.62 ms	9.83 ± 0.5 ms	1 ± 0.081
Simple Pendulum/IIP/BoundaryValueDiffEqMIRK.MIRK4()	2.51 ± 0.11 ms	2.48 ± 0.091 ms	1.01 ± 0.057
Simple Pendulum/IIP/BoundaryValueDiffEqMIRK.MIRK5()	7.33 ± 1.8 ms	7.29 ± 1.8 ms	1.01 ± 0.35
Simple Pendulum/IIP/BoundaryValueDiffEqMIRK.MIRK6()	1.33 ± 0.36 ms	1.32 ± 0.37 ms	1.01 ± 0.39
Simple Pendulum/IIP/MultipleShooting(10, Tsit5; grid_coarsening = false)	1.8 ± 0.95 ms	1.78 ± 1 ms	1.01 ± 0.79
Simple Pendulum/IIP/MultipleShooting(10, Tsit5; grid_coarsening = true)	2.99 ± 0.9 ms	2.98 ± 1 ms	1 ± 0.45
Simple Pendulum/IIP/MultipleShooting(100, Tsit5; grid_coarsening = false)	0.0762 ± 0.014 s	0.0776 ± 0.021 s	0.982 ± 0.32
Simple Pendulum/IIP/MultipleShooting(100, Tsit5; grid_coarsening = true)	0.0915 ± 0.026 s	0.0895 ± 0.026 s	1.02 ± 0.42
Simple Pendulum/IIP/Shooting(Tsit5())	0.239 ± 0.072 ms	0.241 ± 0.075 ms	0.994 ± 0.43
Simple Pendulum/OOP/BoundaryValueDiffEqMIRK.MIRK2()	0.495 ± 0.0065 s	0.493 ± 0.0031 s	1 ± 0.015
Simple Pendulum/OOP/BoundaryValueDiffEqMIRK.MIRK3()	12.7 ± 7.9 ms	12.7 ± 7.4 ms	1 ± 0.85
Simple Pendulum/OOP/BoundaryValueDiffEqMIRK.MIRK4()	2.9 ± 0.38 ms	2.89 ± 0.38 ms	1.01 ± 0.19
Simple Pendulum/OOP/BoundaryValueDiffEqMIRK.MIRK5()	8.53 ± 1.1 ms	8.47 ± 1.1 ms	1.01 ± 0.19
Simple Pendulum/OOP/BoundaryValueDiffEqMIRK.MIRK6()	1.53 ± 0.23 ms	1.51 ± 0.23 ms	1.01 ± 0.21
Simple Pendulum/OOP/MultipleShooting(10, Tsit5; grid_coarsening = false)	3.13 ± 3.3 ms	3.11 ± 3.7 ms	1.01 ± 1.6
Simple Pendulum/OOP/MultipleShooting(10, Tsit5; grid_coarsening = true)	5.32 ± 6 ms	5.38 ± 6.6 ms	0.989 ± 1.6
Simple Pendulum/OOP/MultipleShooting(100, Tsit5; grid_coarsening = false)	0.123 ± 0.02 s	0.124 ± 0.0082 s	0.998 ± 0.17
Simple Pendulum/OOP/MultipleShooting(100, Tsit5; grid_coarsening = true)	0.162 ± 0.087 s	0.17 ± 0.086 s	0.953 ± 0.71
Simple Pendulum/OOP/Shooting(Tsit5())	0.54 ± 0.48 ms	0.538 ± 0.51 ms	1 ± 1.3
time_to_load	4.3 ± 0.041 s	4.27 ± 0.013 s	1.01 ± 0.01

### Benchmark Plots A plot of the benchmark results has been uploaded as an artifact to the workflow run for this PR. Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).