All biomodels Copasi currently fails for

[luciansmith]

tellurium_failures.xlsx
copasi_failures.csv

I'm attaching two lists: one of all models where Tellurium currently fails, and one of all models where Copasi currently fails. In general, it should in theory be possible for Copasi to succeed at everything that Tellurium succeeds on, and it might succeed at a few more, but I wouldn't count on it.

The list has links to the logs of all 8/25/23 runs of all biomodels on Copasi, which should tell you what the problems were. Many of them are the same problem; i.e. for runs like

https://run.biosimulations.org/runs/64e932fc548ceb84f3effcf6#tab=log

where the problem is that the number of points, originally 1000, got read into a real value instead of an int, so couldn't be used later. Once that's fixed, many of the other runs will also be fixed. Similar things will be true of the other runs; I'm happy to explain how they work in Tellurium if you like.

[luciansmith]

Update! Here's the comparison for all models, run with version 4.36 (pre-basico) and 4.42 (the latest)
copasi_compare.csv

[pmendes]

Note that the models showing failed in version 4.36 are also not due to COPASI itself, but the adapter. For example BIOMD0000000451 : this is a model that was actually written and run in COPASI in the first place. Both the SED-ML and SBML work fine in COPASI 4.36. Obviously it is the adapter and/or the workflow that is causing the failure.

[jcschaff]

note: Lucians runs from 3 days ago (see previous post) compare containers built from biosimulators_copasi with tags 0.1.35 and 0.1.36

[luciansmith]

@pmendes I've assumed that all the failures are from the wrapper. It's possible there might be a corner case here or there due to COPASI itself, but even that is unlikely.

[pmendes]

@luciansmith yes, apart from the models with DDEs, I think we should be passing all of them (perhaps one or other SBML L3V2 that uses one of the new math functions may also fail but I doubt there are many yet)

[luciansmith]

From @CodeByDrescher :

Changes include:

• Fixed a bug where there would be duplicates in the DataFrame holding copasi's results. This in turn affected the data structure returned by the pandas methods processing the output (expected Series, got DataFrame).
• Identified bug with basico itself, that ignores output requests if the provided display name has nested parenthesis (not a problem in the original COPASI version, it used CNs). This will have to be fixed from the COPASI/basico side of things.
• Fixed a bug where float-division to calculate number of simulation steps would leave a small artifact decimal on an otherwise integer number, causing a sanity check to (properly) throw an exception. Now, any % error of less than 10^-8 is assumed to be a computer-division error and ignored.
• Identified bug with biosimulators-utils where it happily provided biosimulators-copasi (old and new versions) with an empty list of variables to solve for, and then proceeded to get angry when biosimulators-copasi correctly returned an empty set of variables. Currently working on fixing this bug.

Results attached!
copasi_compare.csv

[luciansmith]

Latest fixes mean we're up to 838! ('44b' in the new comparison).
copasi_compare.csv

[pmendes]

It's getting better, many of the failures are expected (DDEs), but there are still some that shouldn't be. For example BIOMD0000000676 and BIOMD0000000696 should work (biomodels db used COPASI in curation).

[luciansmith]

Yes, work is ongoing.

[luciansmith]

Another 50 successes! We're up to 888.

copasi_compare.csv

[luciansmith]

And with Frank's fixes to Copasi itself, we got another 100 models, up to 988!
copasi_compare.csv

[pmendes]

Great progress!

Just out of curiosity I looked up randomly one that still fails, and I find BIOMD0000000498 which is a model from my own lab (and obviously developed in COPASI itself). Curious to see why this one still fails...

[CodeByDrescher]

BIOMD0000000498 is failed by Biosimulators Utils because the SedML requests `steady state root-finding method' (KISAO_0000407), which none of COPASI's algorithms supposedly fall under. This warrants deeper investigation.

[fbergmann]

This term was introduced long ago, to denote just a steady state algorithm.

https://www.ebi.ac.uk/ols4/ontologies/kisao/classes/http%253A%252F%252Fwww.biomodels.net%252Fkisao%252FKISAO%2523KISAO_0000407

And we definitely have one of those in COPASI. When we import it we just take that to mean to run the steady state task.

[luciansmith]

I had to extensively update KiSAO a little while back to correctly organize the steady state and flux balance algorithms; there was zero support for steady state algorithmic substitution in the library like there was for simulations. So I believe the only thing that now needs to happen is for the copasi wrapper to make use of those new functions from the kisao library.

Also, OLS4 seems to have an out-of-date version of KiSAO, with the older scrambled steady state relationships. I'm not sure how to get those updated; I had thought at one point that pushing things to github also triggered pushing the ontology to various ontology front ends, but maybe OLS4 didn't make that list, or something else broke somehow.

[fbergmann]

@CodeByDrescher even if the definition changed, since COPASI does have a method to solve Nv = 0, which would fall under the definiton f(x) = 0 as stated in the new definition of 407, that qualifies to still be used.

If substitutions are allowed, we should also take all the other ones that tellurium accepts for steadystate, which are:

[99, 274, 282, 283, 355, 356, 407, 408, 409, 410, 411, 412, 413, 432, 437, 568, 569, ]

In my opinion, as soon as someone demonstrates: "This model cannot be solved in RoadRunner, because it does not implement LSODA" KISAO has failed. And for steady state it should also not matter if someone implements kinsol, nleq or else.

[luciansmith]

Yes, this was the functionality that I added. There were functions in libkisao that let you discover all the various ODE solvers, but there was nothing that let you discover the steady state functions. Now there are! They are now grouped together in the ontology, and there is also a new 'steady state root-finding problem' to parallel the ODE 'ordinary differential equation problem'.

But these didn't exist back when the copasi wrapper was first written, and can now be added (similarly for the other wrappers, too).