Transaction logic testing tool / load testing tool

[rdaum]

I would like a suite of tests to check for correctness of concurrent behaviour; linearizability, consistency, serialization conflicts, etc.

Still very vague in my mind, but probably a suite of Jepsen tests and/or Stateright (https://github.com/stateright/stateright).

Essentially, instantiating kernel + db with a predefined simple set of objects and then perform concurrent evaluations of mutations to provoke serialization conflicts, retries, and verify correct behaviour under concurrent load.

This will be essential in the short run run for a) finishing the transactional layer (retries on conflict) and b) making sure the Rocks etc layer works as hopes and in the medium/long run for swapping in alternative world state storage models.

[rdaum]

Stateright is more immediately easier to integrate into a Rust automated testing workflow, and their examples show how to do that. However its examples and focus is primarily on distributed systems testing, so teasing out how to do a good test of e.g. MVCC behaviour is something that would have to be learned.

Jepsen is the gold standard for database / distributed systems correctness testing, and it is likely that there's more concrete examplars to follow in this regard for testing on a single-node system for transaction correctness. Basically need to take a look at existing DB jepsen tests that test serialization conflict, and work from there to adapt for Moor. However it has its own external dependencies, runs in Java, etc. and is much more complicated to set up. Maybe running in a docker container? (see https://github.com/jepsen-io/jepsen/blob/main/docker/README.md)

[rdaum]

Added new test:

https://github.com/rdaum/moor/blob/main/crates/db/tests/jepsen.rs

[rdaum]

Ok so updating this, the above comments being very old and obsolete...

I now have a separate (model-checker) tool which puts the whole system under concurrent load doing property updates to append values to lists on a few properties.

It then outputs the read/append results to an EDN-format file that Jepsen's elle tool can read. Elle is then able to verify the transaction isolation model.

Using this I was able to prove that moor running with the fjall-db backend behaves properly with serializable transaction isolation and snapshot isolation levels. It does not however satisfy jepsen's strict-serializable model, though I'm unclear how that differs.