Make the Ampersand compiler read a dataset, to prepare for migration

[stefjoosten]

Problem

As a migration engineer, I want to use the population in an existing database as part of the initial population of a migrated database.

Solution

I want the compiler to read all triples from an external database, and write them to the current database.

Considerations

1. We may be able to reuse some code that is used to validate the Haskell semantics against the database semantics (--validate). That would be the code to attach to the database and to read stuff from it.
2. There are two tasks that can be done in parallel:
   • read the population from an existing (running) MariaDB server.
   • write the population to the current MariaDB server.
3. For future maintainability, we will want to reuse this for reading Excel sheets, and other structured sources.
4. To enable verifiable and maintainable code, we want to read data from the database into a structure that contains triples (i.e. a dataset).

[stefjoosten]

Design choices

1. This functionality shall be implemented as a command line command of the Ampersand compiler, so that it can be called from the migration orchestrator, a Github action, or any other external user.
2. There will be a Haskell data structure, that implements a set of triples together with a type function for atoms (see the theory)

   data Dataset