Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions content/symbiotic/architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,32 @@ Symbiotic provides the shared security layer:
| Storage | `operator/src/storage/` | redb key-value store for messages, Merkle trees, and submissions |
| Crypto | `operator/src/crypto/` | Merkle tree construction, leaf hashing, signing message encoding |

## Production Topology

The starter kit runs everything on one machine; production needs a distributed layout. The parts that matter:

### Attestation API availability

For the Chainlink CCV path, executors fetch attestations from the URLs the verifier advertises on-chain (`getStorageLocations()`). **A single attestation node is a single point of failure that stalls the lane**: messages keep verifying, but executors cannot fetch attestations, so nothing executes until the node returns.

Every operator node serves the same attestation data (`GET /verifications`) — all of them hold the messages and the aggregated quorum signatures. Production options, in order of preference:

1. **Load balancer over multiple operator nodes** — advertise one stable URL, health-check `/healthz`, route to any live operator.
2. **Multiple URLs on-chain** — `getStorageLocations()` returns a list; advertise several operators directly and let executors fail over.
3. **One designated HA node** — acceptable only with real redundancy behind that name (failover, monitoring).

The advertised URL is on-chain state, not config: rotate it with the owner-gated `updateStorageLocations` and treat DNS you can repoint as more operable than raw hosts.

### Where aggregation happens

Individual operators do not serve partial signatures. Each operator signs the Merkle root through its Symbiotic relay sidecar; the relay network aggregates BLS signatures across operators, and every operator then stores the same aggregated result. That is why any single operator can answer attestation queries — availability of the *API* is your concern; signature aggregation already tolerates individual signer downtime as long as quorum voting power signs.

### Availability assumptions to engineer for

- The CCIP indexer polls your attestation API — sustained unavailability means unexecuted (not lost) messages, recoverable once service returns.
- Operators must reach source-chain RPC (event ingestion + finality gating) and the relay sidecars; run operators across failure domains.
- The OZ Monitor→operator webhook path is at-least-once, not exactly-once; operators de-duplicate. Monitor outages are recovered by block backfill on restart — bound your `max_past_blocks` accordingly.

## Adding a New Provider

1. Implement the `Provider` trait in `operator/src/provider/<provider>.rs`.
Expand Down
76 changes: 68 additions & 8 deletions content/symbiotic/chainlink-ccv.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,24 @@ This template supports the Symbiotic CCV path only. The Chainlink auxiliary deve

</Callout>

## On-Chain Architecture: Resolver + Verifier

The CCV identity that CCIP references is split across two contracts:

- **Resolver** — Chainlink's stock `VersionedVerifierResolver`, deployed via Chainlink's `CREATE2Factory` so it has the **same address on every supported chain**. This is the permanent address: it is what OnRamp/OffRamp configuration and the CCIP indexer register, and it never changes. It holds two registries: version prefix → inbound verifier implementation, and destination chain selector → outbound implementation.
- **Verifier implementation** — `SymbioticVerifier`, which inherits Chainlink's `BaseVerifier` and adds Symbiotic BLS quorum verification against `Settlement`. Implementations are versioned and replaceable.

Every verifier result starts with a 4-byte version tag (`0x1a75bd93` for v1, derived from `keccak256("SymbioticCCV 1.0.0")`). Signers commit to `keccak256(versionTag ‖ messageId)`, so the tag both routes the message to the right implementation and prevents a signature produced for one implementation from being replayed against another.

### Upgrading the verifier

Deploying verification logic v2 never changes the address integrators use:

1. Deploy the new `SymbioticVerifier` with a new version tag and configure its lanes.
2. Register it on the resolver: `applyInboundImplementationUpdates([{tag_v2, v2}])` — v1 and v2 now coexist, and in-flight messages signed under v1 still resolve to v1.
3. Switch outbound per lane: `applyOutboundImplementationUpdates([{selector, v2}])` — new messages carry v2's tag.
4. After v1 traffic drains, optionally retire v1's tag by registering it to `address(0)`.

## Message Flow

```mermaid
Expand All @@ -25,7 +43,8 @@ sequenceDiagram
participant Relay as Symbiotic Relay (BLS)
participant Relayer as OZ Relayer
participant OffRamp as OffRamp (Dest)
participant CCV as SymbioticCCV (Dest)
participant Resolver as Resolver (Dest)
participant Verifier as SymbioticVerifier (Dest)

App->>OnRamp: sendMessage()
OnRamp-->>Monitor: CCIPMessageSent event
Expand All @@ -35,18 +54,58 @@ sequenceDiagram
Relay-->>Operators: aggregated signature
Operators->>Relayer: OffRamp.execute calldata
Relayer->>OffRamp: execute(message, ccvs, verifierResults)
OffRamp->>CCV: verifyMessage(message, messageId, verifierResults)
CCV-->>OffRamp: verified
OffRamp->>Resolver: getInboundImplementation(verifierResults[:4])
Resolver-->>OffRamp: SymbioticVerifier (by version tag)
OffRamp->>Verifier: verifyMessage(message, messageId, verifierResults)
Verifier-->>OffRamp: verified (RMN curse check, OffRamp auth via Router, BLS quorum)
OffRamp-->>OffRamp: emit MessageExecuted(messageId)
```

## Verification Gas Budget

Each lane's `gasForVerification` (set via `applyRemoteChainConfigUpdates`, returned by `getFee`) must cover the full `verifyMessage` call. Its cost scales with the validator set in three ways:

- **proof handling in the verifier** — the quorum proof embeds the entire validator set (64 bytes per validator), and the current `SymbioticVerifier` implementation copies the proof byte-by-byte before handing it to `Settlement`. At large sets this copy dominates everything else;
- **signature verification** (`SigVerifierBlsBn254Simple`) — hashes the full validator-set data against the committed set, then pays a key decompression plus curve addition for each validator that did *not* sign, plus one BLS pairing (~113k, fixed);
- **fixed base** — `BaseVerifier` ramp/RMN checks and `Settlement` epoch bookkeeping.

Measured end to end with the repo's gas harness (real `SymbioticVerifier` → `Settlement` hop → real `SigVerifierBlsBn254Simple`, synthetic equal-power validator sets):

```bash
cd contracts && forge test --match-contract VerificationGas -vv
```

| Validators | All sign | 33% non-signers (by count) |
|-----------:|---------:|---------------------------:|
| 3 | 292k | 296k |
| 10 | 407k | 421k |
| 25 | 657k | 693k |
| 50 | 1.09M | 1.16M |
| 100 | 2.00M | 2.17M |

The 3-validator row is within ~5% of what Chainlink measured on the staging deployment (299k–312k across live messages), so treat the table as representative. Interpolate for your set size and add a **safety margin (recommend 20%)**. The staging config value of `400000` sits *below* the measured cost of a 10-validator set (407k before margin) — it is sized for the current small staging set, not a constant. (Do not benchmark against the unit tests: `SymbioticVerifier.t.sol` stubs `Settlement`, so its gas numbers exclude all of the above.)

Two caveats when sizing the non-signer allowance:

- Quorum is enforced on **voting power, not head count**. With unequal stake weights, far more than a third of validators (by count) can abstain while quorum still passes — and each absent validator costs decompression + addition. Derive your worst case from the actual power distribution; the table's second column is the equal-power case.
- **Re-evaluate whenever the validator set grows** — operator registration changes the set automatically at epoch boundaries, so an under-provisioned value does not fail immediately; it fails when the set outgrows it, and messages then stall at verification. Alert on validator-set size, not on failures.

Because per-message cost grows roughly linearly with the set (~18k gas per validator end to end in the current implementation), Symbiotic's ZK verification mode — near-constant gas regardless of set size — becomes the cheaper option well before very large sets. If your set is growing past a few dozen validators, evaluate the switch; it deploys as a verifier-implementation upgrade (see above — the resolver address does not change).

## Recommended Verifier Composition (Token Issuers)

**Run the Chainlink committee verifier alongside the Symbiotic verifier as your default.** Two independent CCVs give defense in depth out of the box: a message executes only when both verifiers attest, so compromising either the committee or the Symbiotic validator set alone is not sufficient to forge a message.

Configure both CCVs (the Chainlink committee verifier plus this resolver's address) in your token pool / app CCV list. Symbiotic-only operation is supported, but treat it as a **conscious, documented opt-out** — you are removing an independent verification layer, and you should record that decision and its rationale in your own security documentation.

Token issuers can also **supply their own token as the collateral securing the verifier network** (self-securing collateral). This needs nothing CCV-specific: create a Symbiotic vault, deposit the asset, and have operators opt in — standard [Symbiotic vault creation](https://docs.symbiotic.fi/), with your token as the staked asset backing the validator set's economic security.

## Contracts and Code

### Contracts

- `contracts/src/ccv/SymbioticCCV.sol`
- `contracts/src/ccv/interfaces/`
- `contracts/src/ccv/libraries/`
- `contracts/src/chainlink/SymbioticVerifier.sol` — verifier implementation (inherits Chainlink `BaseVerifier`)
- Resolver + factory: stock contracts from `@chainlink/contracts-ccip` (`VersionedVerifierResolver`, `CREATE2Factory`) — deployed from the package, no local source
- `contracts/src/symbiotic/Settlement.sol`
- `contracts/src/symbiotic/KeyRegistry.sol`
- `contracts/src/symbiotic/Driver.sol`
Expand Down Expand Up @@ -77,16 +136,17 @@ Address resolution order:
1. `CCV_*` environment variables
2. `deployments/<env>.json`

The resolver address is the exception: it has no environment override and is always read from `deployments/<env>.json` (`chainlinkCcv.resolver`), matching the CREATE2 determinism guarantee.

Available overrides:

| Variable | Description |
|----------|-------------|
| `CCV_SOURCE_ADDRESS` | SymbioticCCV on source chain |
| `CCV_DEST_ADDRESS` | SymbioticCCV on destination chain |
| `CCV_SOURCE_ONRAMP_ADDRESS` | Source OnRamp-compatible contract |
| `CCV_DEST_OFFRAMP_ADDRESS` | Destination OffRamp submit target |
| `CCV_SOURCE_CHAIN_SELECTOR` | Override the source CCIP chain selector |
| `CCV_DEST_CHAIN_SELECTOR` | Override the destination CCIP chain selector |
| `CCV_STORAGE_LOCATION_URIS` | Comma-separated attestation API base URLs advertised on-chain (required for non-local deploys) |

Local and testnet are supported (testnet uses the `testnet-ccv` environment: `config/environments/testnet-ccv.json` and `deployments/testnet-ccv.json`, run via `ENV=testnet-ccv`); mainnet is not yet supported for CCV. `make watch` only succeeds once destination `MessageExecuted(messageId)` is observed.

Expand Down
22 changes: 21 additions & 1 deletion content/symbiotic/cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,15 @@ Send one test message through the active provider.
make send
make send MSG="test message"
make send ENV=testnet MSG="hello"
make send ENV=local-ccv EXECUTOR=0x... FINALITY=safe
```

| Variable | Description |
|----------|-------------|
| `MSG` | Message payload (default: "hello") |
| `EXECUTOR` | Executor address encoded into the message (CCV local mock send only). Operators self-execute only when this matches their configured executor address. |
| `FINALITY` | Source-finality requirement encoded into the message (CCV local mock send only): `finalized` (default), `safe`, or a block count `N`. |

### `make watch`

Watch a previously sent message until the destination path succeeds.
Expand All @@ -44,12 +51,15 @@ Send a message, then watch it to completion.
make e2e
make e2e MSG="custom message"
make e2e ENV=testnet MSG="hello" TIMEOUT=180
make e2e ENV=local-ccv EXECUTOR=0x... TIMEOUT=180
```

`EXECUTOR` and `FINALITY` pass through exactly as in `make send`.

LayerZero starter OApp addresses are published under:

```text
deployments/<env>.json -> layerzero.oapp.{source,destination}
deployments/<env>.json -> {source,destination}.layerzero.exampleApp
```

## Direct xtask Usage
Expand Down Expand Up @@ -134,6 +144,16 @@ LayerZero operators also expose:
| `POST /api/v1/layerzero/proof` | Return Merkle proofs for processed messages |
| `POST /api/v1/layerzero/verify` | Verify a Merkle proof in test workflows |

### Chainlink CCV Verification Endpoint

CCV operators expose the attestation API advertised on-chain via `getStorageLocations()` (host ports 3001-3003):

```bash
curl "http://localhost:3001/verifications?messageID=0x..."
```

`messageID` is repeatable (up to 20 per request). Responses carry Chainlink-CCV-shaped verifier results whose `ccv_data` is prefixed with the verifier version tag. Requests with no `messageID`, more than 20, or a malformed id return HTTP 400.

## Webhook Template Requirements

OZ Monitor trigger templates live under `config/templates/oz-monitor/triggers/` and are copied into `generated/<env>/oz-monitor/triggers/`.
Expand Down
Loading
Loading