Merge pull request #33 from superfly/mo-docs

More docs

.github/workflows/book.yml

@@ -20,7 +20,7 @@ jobs:
 
       - uses: taiki-e/install-action@v2
         with:
-          tool: mdbook,mdbook-linkcheck
+          tool: mdbook,mdbook-linkcheck,mdbook-admonish
 
       - name: Build Book
         run: mdbook build

.github/workflows/ci.yml

@@ -59,7 +59,7 @@ jobs:
 
       - uses: taiki-e/install-action@v2
         with:
-          tool: mdbook,mdbook-linkcheck
+          tool: mdbook,mdbook-linkcheck,mdbook-admonish
 
       - name: Build Book
         run: mdbook build

book.toml

@@ -5,6 +5,11 @@ multilingual = false
 src = "doc"
 title = "Corrosion"
 
+[preprocessor.admonish]
+command = "mdbook-admonish"
+assets_version = "2.0.2" # do not edit: managed by `mdbook-admonish install`
+
 [output.html]
+additional-css = ["./doc/mdbook-admonish.css"]
 
-[output.linkcheck]
+[output.linkcheck]

doc/SUMMARY.md

@@ -3,9 +3,21 @@
 [Introduction](intro.md)
 
 # User guide
-- [Operations Guide]()
+- [Authorization]()
+- [Backup / Restore]()
+- [CRDTs](crdts.md)
+- [Encryption]()
+- [Gossip]()
+- [Schema](schema.md)
+- [Subscriptions]()
+- [Synchronization]()
+- [Templates]()
 
 # Reference
+- [API]()
+    - [POST /v1/exec]()
+    - [POST /v1/query]()
+    - [POST /v1/subscriptions]()
 - [Command-line Interface](cli/README.md)
     - [agent]()
     - [backup](cli/backup.md)
@@ -18,7 +30,7 @@
     - [template]()
     - [tls](cli/tls.md)
 - [Configuration](config/README.md)
-    - [db]()
+    - [db](config/db.md)
     - [gossip](config/gossip.md)
     - [api]()
     - [admin]()

doc/config/db.md

@@ -0,0 +1,27 @@
+# The `[db]` configuration
+
+The `[db]` block configures the Corrosion's SQLite database.
+
+### Required fields
+
+#### `db.path`
+
+Path of the sqlite3 database.
+
+```toml
+[db]
+path = "/var/lib/corrosion/db.sqlite"
+```
+
+### Optional fields
+
+#### `db.schema_paths`
+
+Array of paths where [schema]() .sql files are present.
+
+```toml
+[db]
+schema_paths = ["/etc/corrosion/schema", "/path/to/table_name.sql"]
+```
+
+If a directory is specified, all .sql files will be loaded.

doc/config/gossip.md

@@ -38,7 +38,9 @@ bootstrap = ["my-fly-app.internal:3333@[fdaa::3]:53"]
 
 Allows using QUIC without encryption. The only reason to set this to `true` is if you're running a toy cluster or if the underlying transport is already handling cryptography (such as WireGuard) AND authorization is bound by the network (such is the case for a [Fly.io](https://fly.io) app's private network).
 
+```admonish warning
 It's highly recommended to use the `gossip.tls` configuration block to setup encryption and `gossip.tls.client` to setup authorization.
+```
 
 #### `gossip.max_mtu`
 

doc/crdts.md

@@ -0,0 +1,25 @@
+# Conflict-free Replicated Data Types
+
+## What's a CRDT?
+
+[About CRDTs](https://crdt.tech/#:~:text=Conflict%2Dfree%20Replicated%20Data%20Types%20(CRDTs)%20are%20used%20in,merged%20into%20a%20consistent%20state.)
+
+> Conflict-free Replicated Data Types (CRDTs) are used in systems with optimistic replication, where they take care of conflict resolution. CRDTs ensure that, no matter what data modifications are made on different replicas, the data can always be merged into a consistent state. This merge is performed automatically by the CRDT, without requiring any special conflict resolution code or user intervention.
+
+## cr-sqlite
+
+Corrosion uses the [`cr-sqlite` SQLite extension](https://github.com/vlcn-io/cr-sqlite) to accomplish its multi-writer and eventual consistency promise. Here's a [short intro](https://vlcn.io/docs/cr-sqlite/intro) about how it generally works. The rest of this section assumes some knowledge of cr-sqlite.
+
+`cr-sqlite` provides functions to mark tables, in a SQLite database, as backed by CRDTs. These include Causal-Length ([pdf paper](https://dl.acm.org/doi/pdf/10.1145/3380787.3393678)) and Last-Write-Wins (LWW).
+
+### Basics
+
+With the extension loaded, writes to CRDT-backed tables will trigger insertions in internal tables for each column in a row.
+
+An aggregate view table is available as `crsql_changes`. By `SELECT`-ing from this table and `INSERT`-ing into it, it's possible to distribute changes to a cluster. Each set of changes (a transaction produces a single set of changes) gets a new `db_version`, unique to the database.
+
+## Corrosion and cr-sqlite
+
+Corrosion executes transactions by processing requests made to its client HTTP API. Each transaction triggers 1+ broadcast (big changesets are chunked). Each change is serialized in an efficient format and sent to ~random members of the cluster.
+
+The main caveat of this approach is: **writes to the database all have to go through Corrosion**. If a sqlite client were to issue writes w/ or w/o the proper extension loaded, then data would become inconsistent for CRDT-backed tables.