diff --git a/README.md b/README.md index e363b7f1..796c9913 100644 --- a/README.md +++ b/README.md @@ -1,55 +1,58 @@ # Corrosion +Corrosion: Gossip-based service discovery (and more) for large distributed systems, written in Rust. -## Launching on Fly +## Why we built Corrosion -``` -fly launch --no-deploy --org --region --copy-config --name -fly volumes create corro_data -y -s 10 -r -a -fly deploy --dockerfile examples/fly/Dockerfile -a -c examples/fly/fly.toml -fly machine clone -a --region --select -fly machine clone -a --region --select -``` - -## Reading data +We built Corrosion specifically for service discovery across a large global network, replacing Consul’s central state database with eventually consistent state distributed across our hosts. -The readable API surface for Corrosion-produced data is to use SQLite and read directly from the database. +Our new tool needed to deliver the following: -Adding indexes either through the API or schema files is the way to go to improve read performance on application-specific queries. +### Fast reads and writes -## Writing data +Getting state (data) from a central remote source can be incredibly expensive (at least 300ms for a round-trip to something on the other side of the world), but usually takes less than 1ms from a local source. -The only way to write data that propagates through the Corrosion cluster is to go through its HTTP API. +### Fast consistency -Corrosion's HTTP API is rqlite-compatible. This is both meant as a shortcut and to have ready-made clients for it. +Strong consistency by RAFT consensus can be too slow. Eventual consistency works, if it's fast. -### POST /v1/transactions +### Flexibility -See: https://rqlite.io/docs/api/api/#writing-data +Global state for a distributed system isn't one-size-fits-all. Flexible schemas and queries are essential. -Corrosion-specific limitations: -- All requests are treated as if the `transaction` query param had been passed (everything runs in a transaction) -- There's a limit to how many statements can be input at once to reduce changeset sizes (this shouldn't be the case for long) +## How Corrosion works -## Updating / migrating / changing the schema +In a nutshell, Corrosion: -Corrosion migrates schema changes automatically without the need for manual migration files. Each schema SQL file is expected to contain SQLite-compatible statements that create resources (e.g. `CREATE TABLE` and `CREATE INDEX`). `ALTER` or `DROP` statements are not supported. Destructive actions are ignored. +- Hosts a SQLite database on each node, for fast local reads and writes +- Uses [CR-SQLite](https://github.com/vlcn-io/cr-sqlite) for conflict resolution with CRDTs +- Uses [Foca](https://github.com/caio/foca) to manage cluster membership using a SWIM protocol +- Gossips changes from the local database throughout the cluster +- Periodically synchronizes with a subset of other cluster nodes, to ensure consistency -### File-based schemas +## Features -Configuring Corrosion with `schema_paths` lets you define filesystem directories where `.sql` files can be found. These files should, as a whole, represent the schema for your data. +- A flexible API to read from and write to Corrosion's store using SQL statements +- File-based schemas with on-the-fly updates +- HTTP streaming subscriptions based on SQL queries +- Live population of configuration files from Corrosion state with user-defined [Rhai](https://rhai.rs/) templates +- Storage and propagation of state from locally registered Consul services, replacing the central database with Corrosion's distributed state +- Vanilla SQLite storage for host-local data +- Secure peer-to-peer communication with the [QUIC](https://datatracker.ietf.org/doc/html/rfc9000) transport protocol (using [Quinn](https://github.com/quinn-rs/quinn)) -You can add or modify schema files in known paths or even modify the `schema_paths` and then send a SIGHUP signal to Corrosion to apply schema changes, without a restart. +## Usage overview -### POST /v1/migrations +Run the Corrosion agent on every node/host in the cluster. Other programs running on the node use [Corrosion's HTTP API](/doc/api/README.md) to query the local Corrosion SQLite database, add and update data, and subscribe to change notifications. -This endpoint accepts the same type of body as the `/v1/transactions` endpoint, except it will mutate Corrosion's schema. +The [Corrosion CLI](/doc/cli/README.md) provides commands for administration and access to database and features. -## Subscribing to changes +See the WIP [Corrosion documentation](/doc/SUMMARY.md) for more details. -One of Corrosion's most powerful features is the ability to subscribe to changes as they are applied to the local node. +## Building Corrosion -This works via websockets for maximum compatibility. +Clone [https://github.com/superfly/corrosion2.git](https://github.com/superfly/corrosion2.git). -## Migrating data +From within the repo directory: -(WIP) \ No newline at end of file +``` +cargo build --release && mv target/release/corrosion ./ +``` diff --git a/doc/SUMMARY.md b/doc/SUMMARY.md index e04b3c81..e7b801b8 100644 --- a/doc/SUMMARY.md +++ b/doc/SUMMARY.md @@ -3,18 +3,18 @@ [Introduction](intro.md) # User guide -- [Authorization]() -- [Backup / Restore]() -- [Consistency]() +- [Authorization]() (to come) +- [Backup / Restore]() (to come) +- [Consistency]() (to come) - [CRDTs](crdts.md) -- [Encryption]() -- [Gossip]() +- [Encryption]() (to come) +- [Gossip]() (to come) - [Schema](schema.md) -- [Subscriptions]() -- [Synchronization]() +- [Subscriptions]() (to come) +- [Synchronization]() (to come) - [Telemetry](telemetry/README.md) - [Prometheus](telemetry/prometheus.md) -- [Templates]() +- [Templates]() (to come) # Reference - [API](api/README.md) @@ -22,20 +22,20 @@ - [POST /v1/queries](api/queries.md) - [POST /v1/subscriptions](api/subscriptions.md) - [Command-line Interface](cli/README.md) - - [agent]() + - [agent](cli/agent.md) - [backup](cli/backup.md) - - [consul]() - - [exec]() - - [query]() - - [reload]() + - [consul]() (to come) + - [exec](cli/exec.md) + - [query](cli/query.md) + - [reload](cli/reload.md) - [restore](cli/restore.md) - - [sync]() - - [template]() + - [sync]() (to come) + - [template](cli/template.md) - [tls](cli/tls.md) - [Configuration](config/README.md) - [db](config/db.md) - [gossip](config/gossip.md) - - [api]() - - [admin]() - - [telemetry]() - - [consul]() \ No newline at end of file + - [api]() (to come) + - [admin]() (to come) + - [telemetry]() (to come) + - [consul]() (to come) \ No newline at end of file diff --git a/doc/api/README.md b/doc/api/README.md index 44cc9653..b0331dec 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -1,7 +1,11 @@ -## API +# API Each running Corrosion agent hosts a RESTful HTTP API for interacting with the cluster's synchronized database. Endpoints accept SQL statements in a JSON body, for versatility. +Each request is run in a transaction (as if the `transaction` query param had been passed). + +Endpoints: + - [POST /v1/transactions](transactions.md) for writes - [POST /v1/queries](queries.md) for reads - [POST /v1/subscriptions](subscriptions.md) to receive streaming updates for a desired query \ No newline at end of file diff --git a/doc/api/queries.md b/doc/api/queries.md index b92ce332..70be3e1c 100644 --- a/doc/api/queries.md +++ b/doc/api/queries.md @@ -1,15 +1,15 @@ -## POST /v1/queries +# POST /v1/queries Read from the Corrosion database. The `/v1/queries` endpoint accepts a single SQL statement in JSON format. -### Sample request +## Sample request ``` curl http://localhost:8080/v1/queries \ -H "content-type: application/json" \ -d "\"SELECT sandwich FROM sandwiches\"" ``` -### Sample response +## Sample response ```json {"columns":["sandwich"]} {"row":[1,["burger"]]} diff --git a/doc/api/subscriptions.md b/doc/api/subscriptions.md index 8c10ce53..2ca5bb73 100644 --- a/doc/api/subscriptions.md +++ b/doc/api/subscriptions.md @@ -3,14 +3,14 @@ Start receiving updates for a desired SQL query. The `/v1/subscriptions` endpoint accepts a single SQL statement in JSON format. The Corrosion agent responds with an HTML stream that notifies of any changes to the response to this query. -### Sample request +## Sample request ``` curl http://localhost:8080/v1/subscriptions \ -H "content-type: application/json" \ - -d "\"SELECT sandwich FROM sw\"" + -d "\"SELECT sandwich FROM sandwiches\"" ``` -### Sample response +## Sample response ```json {"columns":["sandwich"]} diff --git a/doc/api/transactions.md b/doc/api/transactions.md index f21058c4..e6d90547 100644 --- a/doc/api/transactions.md +++ b/doc/api/transactions.md @@ -1,15 +1,15 @@ -## POST /v1/transactions +# POST /v1/transactions Write changes to the Corrosion database for propagation through the cluster. The `/v1/transactions` endpoint accepts a JSON list of SQL statements. -### Sample request +## Sample request ``` curl http://localhost:8080/v1/transactions \ -H "content-type: application/json" \ -d "[\"INSERT OR IGNORE INTO sandwiches (pk, sandwich) VALUES (3, 'brie and cranberry')\"]" ``` -### Sample response +## Sample response ```json {"results":[{"rows_affected":1,"time":0.000027208}],"time":0.000300708}% ``` \ No newline at end of file diff --git a/doc/cli/README.md b/doc/cli/README.md index f288f9a4..0d023bb9 100644 --- a/doc/cli/README.md +++ b/doc/cli/README.md @@ -1,5 +1,14 @@ # Command-line Interface -See the chapters for each subcommand: +Corrosion has a CLI for managing the local Corrosion agent and database. It also provides commands to read from and write to the cluster's database. + +The base command is `corrosion`. Run `corrosion --help` for a list of subcommands. + +See the pages for each subcommand: +- [`corrosion agent`](agent.md) - [`corrosion backup`](backup.md) - [`corrosion restore`](restore.md) +- [`corrosion exec`](exec.md) +- [`corrosion query`](query.md) +- [`corrosion template`](template.md) +- [`corrosion reload`](reload.md) diff --git a/doc/cli/agent.md b/doc/cli/agent.md new file mode 100644 index 00000000..6d0c1e02 --- /dev/null +++ b/doc/cli/agent.md @@ -0,0 +1,18 @@ +# The `corrosion agent` command + +Starts Corrosion on the host. The `--api-addr`, `--db-path`, and `--admin-path` options override corresponding settings in the configuration file. + +```cmd +$ corrosion agent --help +Launches the agent + +Usage: corrosion agent [OPTIONS] + +Options: + -c, --config Set the config file path [default: /etc/corrosion/config.toml] + --api-addr + --db-path + --admin-path + -h, --help Print help +``` + diff --git a/doc/cli/exec.md b/doc/cli/exec.md new file mode 100644 index 00000000..93dddeab --- /dev/null +++ b/doc/cli/exec.md @@ -0,0 +1,25 @@ +# The `corrosion exec` command + +Writes to Corrosion's database, via the [`/v1/transactions/`](../api/transactions.md) endpoint hosted by the local Corrosion agent. + +Use `corrosion exec` to mutate data within existing CR-SQLite-enabled Corrosion database tables, for propagation throughout the cluster. + +Corrosion does not sync schema changes made using this command. Use Corrosion's [schema files](/schema.md) to create and update the cluster's database schema. + +``` +$ corrosion exec --help +Execute a SQL statement that mutates the state of Corrosion + +Usage: corrosion exec [OPTIONS] + +Arguments: + + +Options: + -c, --config Set the config file path [default: /etc/corrosion/config.toml] + --timer + --api-addr + --db-path + --admin-path + -h, --help Print help +``` \ No newline at end of file diff --git a/doc/cli/query.md b/doc/cli/query.md new file mode 100644 index 00000000..8dd0636c --- /dev/null +++ b/doc/cli/query.md @@ -0,0 +1,25 @@ +# The `corrosion query` command + +Reads from Corrosion's database, via the [`/v1/queries/`](../api/queries.md) endpoint hosted by the local Corrosion agent. + +Use the `--columns` option to see column headings in the output. + +``` +$ corrosion query --help +Query data from Corrosion w/ a SQL statement + +Usage: corrosion query [OPTIONS] + +Arguments: + + +Options: + -c, --config Set the config file path [default: /etc/corrosion/config.toml] + --columns + --api-addr + --timer + --db-path + --param + --admin-path + -h, --help Print help +``` diff --git a/doc/cli/reload.md b/doc/cli/reload.md new file mode 100644 index 00000000..16959c58 --- /dev/null +++ b/doc/cli/reload.md @@ -0,0 +1,18 @@ +# The `corrosion reload` command + +Reloads Corrosion configuration from a file. + +``` +$ corrosion reload --help +Reload the config + +Usage: corrosion reload [OPTIONS] + +Options: + -c, --config Set the config file path [default: /etc/corrosion/config.toml] + --api-addr + --db-path + --admin-path + -h, --help +``` + diff --git a/doc/cli/template.md b/doc/cli/template.md new file mode 100644 index 00000000..e28a6cae --- /dev/null +++ b/doc/cli/template.md @@ -0,0 +1,21 @@ +# The `corrosion template` command + +Uses Corrosion's template engine to generate and update a local file based on a [Rhai](https://rhai.rs/) script and the latest data from Corrosion. + +Specify the name of the `.rhai` file and the desired name for the output file. + +``` +$ corrosion template --help +Usage: corrosion template [OPTIONS] [TEMPLATE]... + +Arguments: + [TEMPLATE]... + +Options: + -c, --config Set the config file path [default: /etc/corrosion/config.toml] + -o, --once + --api-addr + --db-path + --admin-path + -h, --help Print help +``` \ No newline at end of file