Docs: update README #3507
Docs: update README #3507
Conversation
|
Perhaps we can move some of the details of the architecture and crate organization under docs/dev/? |
There was a problem hiding this comment.
Thanks a lot for updating our README and extending it with an architectural overview @haf. I think this can be helpful for community members to get started in the code base. As such, I agree with Ahmed, that it might better fit under docs/dev where we keep our developer documentation. We can link to there from the top-level README. Would you be up for moving it there?
I left a few quick comments. I assume that some parts of the overview where automatically generated, right? Which tool did you use? I am wondering how to maintain it as the architecture evolves. Maybe we can document the prompts that we used to generate the specific parts so that one can re-run it once it changed?
| graph TD | ||
| A["External Clients"] -->|HTTP / gRPC / Kafka| I(Ingress) | ||
| I -->|forward invocations| W(Workers) | ||
| W -->|append events| L[Distributed Log (Bifrost)] |
There was a problem hiding this comment.
Mermaid doesn't like L[Distributed Log (Bifrost)] (https://mermaid.live/edit). Are you able to render it?
| ```mermaid | ||
| flowchart LR | ||
| subgraph Binaries | ||
| CLI([cli/ – restate CLI]) | ||
| SRV([server/ – restate-server]) | ||
| end | ||
| subgraph CoreLibs | ||
| CORE([crates/core]) | ||
| TYPES([crates/types]) | ||
| ERR([crates/errors]) | ||
| end | ||
| subgraph ControlPlane | ||
| META([crates/metadata-server]) | ||
| ADMIN([crates/admin \n + admin-rest-model]) | ||
| end | ||
| subgraph LogLayer | ||
| BIF([crates/bifrost]) | ||
| LOGS([crates/log-server]) | ||
| WAL([crates/wal-protocol]) | ||
| end | ||
| subgraph Execution | ||
| WRK([crates/worker]) | ||
| INV([crates/invoker-impl]) | ||
| end | ||
| subgraph Ingress | ||
| HTTP([crates/ingress-http]) | ||
| KAF([crates/ingress-kafka]) | ||
| end | ||
| CLI-->CORE | ||
| SRV-->CORE | ||
| WRK-->CORE | ||
| INV-->WRK | ||
| WRK-->BIF | ||
| BIF-->LOGS | ||
| META-->LOGS | ||
| META-->WRK | ||
| HTTP-->WRK | ||
| KAF-->WRK | ||
| ``` |
There was a problem hiding this comment.
What's the idea of this flow chart? Is it to show which crate depends on which other?
| ```mermaid | ||
| flowchart LR | ||
| subgraph Binaries | ||
| CLI([cli/ – restate CLI]) | ||
| SRV([server/ – restate-server]) | ||
| end | ||
| subgraph CoreLibs | ||
| CORE([crates/core]) | ||
| TYPES([crates/types]) | ||
| ERR([crates/errors]) | ||
| end | ||
| subgraph ControlPlane | ||
| META([crates/metadata-server]) | ||
| ADMIN([crates/admin \n + admin-rest-model]) | ||
| end | ||
| subgraph LogLayer | ||
| BIF([crates/bifrost]) | ||
| LOGS([crates/log-server]) | ||
| WAL([crates/wal-protocol]) | ||
| end | ||
| subgraph Execution | ||
| WRK([crates/worker]) | ||
| INV([crates/invoker-impl]) | ||
| end | ||
| subgraph Ingress | ||
| HTTP([crates/ingress-http]) | ||
| KAF([crates/ingress-kafka]) | ||
| end | ||
| CLI-->CORE | ||
| SRV-->CORE | ||
| WRK-->CORE | ||
| INV-->WRK | ||
| WRK-->BIF | ||
| BIF-->LOGS | ||
| META-->LOGS | ||
| META-->WRK | ||
| HTTP-->WRK | ||
| KAF-->WRK | ||
| ``` |
There was a problem hiding this comment.
The wal-protocol is not part of the log layer but of the execution layer.
| CLI([cli/ – restate CLI]) | ||
| SRV([server/ – restate-server]) |
There was a problem hiding this comment.
We are missing restatectl
|
|
||
| * Format & lint: `cargo fmt && cargo clippy --all-targets`. | ||
| * Feature flags follow `crate-name/feature` convention – check each `Cargo.toml`. | ||
| * Docs live under [`docs/`](docs/) – update when changing public behaviour. |
There was a problem hiding this comment.
Only our development docs live under docs/dev. The actual documentation lives https://github.com/restatedev/documentation.
|
|
||
| ## Protobuf & API Surface | ||
|
|
||
| All cross-service contracts live under `protobuf/` directories next to their crates. To regenerate: `cargo xtask proto-gen`. |
There was a problem hiding this comment.
I xtask is not generating the protobuf files. This should be covered by our build.rs.
| cargo run -p server -- start --demo | ||
|
|
||
| # Spawn 3-node cluster using docker-compose | ||
| cd docker && docker compose up |
There was a problem hiding this comment.
There is no docker-compose file.
| restate sql "SELECT * FROM invocations LIMIT 10;" | ||
| ``` | ||
|
|
||
| *Configuration files* live in [`charts/restate-helm/templates`](charts/restate-helm/templates) and are reused by `docker/` images. |
There was a problem hiding this comment.
This is only relevant for the helm chart.
| 3. If it's a role service, expose a `Service` struct implementing `run(self) -> anyhow::Result<()>`. | ||
| 4. Wire it in `server/src/main.rs` behind a feature flag. |
There was a problem hiding this comment.
Shouldn't be needed for a generic crate.
Co-authored-by: Till Rohrmann <[email protected]>
Co-authored-by: Till Rohrmann <[email protected]>
Co-authored-by: Till Rohrmann <[email protected]>
Co-authored-by: Till Rohrmann <[email protected]>
Minor README improvements: refreshed quick links and clarified dev guide references.