-
Notifications
You must be signed in to change notification settings - Fork 5
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: egress client - ucanto integration (#123)
## Egress Client - UCanto Integration ### Context This PR introduces `EgressClient` to the application context, enabling capability invocations such as `Space.egressRecord` for logging egress directly to the UCanto Server (Upload API). With this integration, the previous Accounting Service is now replaced by a method in `EgressClient` for egress logging. ### Key Changes - **Middleware Integration**: Added `withEgressClient` middleware to simplify capability invocations across the application. - **EgressClient Creation**: Introduced a `create` function to instantiate `UCantoClient`, establishing a secure connection to the UCanto Server based on environment configurations. - **Egress Recording**: Refactored `EgressClient.record` function to use the `Space.egressRecord` capability, allowing for efficient egress byte tracking in our infrastructure. - **Connection Management**: Added a `connect` function to handle connection setup with the UCanto Server. - **Environment Variables**: Updated the environment variables, including service Web DIDs and service URLs. - **Enhanced Context**: Added the `delegationProofs` to the application context, so we can use that information to invoke the `egressRecord` capabilities. Also added the `GatewayIdentity` to the application context. - **Accounting Service**: There is no Accounting Service anymore. Instead, we use the EgressClient to record the egress event. - **Wrangler**: Updated the configs for all environments and the wrangler lib to the latest version. - **Telemetry**: Added a feature flag for Open Telemetry - if enabled, the Egress Record call will fail to execute the `this.fetch` function call. See storacha/project-tracking#176 for more details. --------- Signed-off-by: Felipe Forbeck <[email protected]>
- Loading branch information
Showing
27 changed files
with
2,715 additions
and
3,205 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,5 +2,5 @@ node_modules | |
| dist | ||
| .mf | ||
| .env | ||
| .dev.vars | ||
| .dev.vars* | ||
| .wrangler | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -101,7 +101,7 @@ npm run test:unit | |
| **Integration Tests** | ||
| ```sh | ||
| TBD | ||
| npm run test:integration | ||
| ``` | ||
| ## Deployment | ||
|
|
||
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,38 +1,73 @@ | ||
| import sade from 'sade' | ||
| import { getClient } from '@web3-storage/w3cli/lib.js' | ||
| import * as ed25519 from '@ucanto/principal/ed25519' | ||
| import * as serve from '../src/capabilities/serve.js' | ||
| import { Space } from '@web3-storage/capabilities' | ||
|
|
||
| const cli = sade('delegate-serve.js <space> [token]') | ||
| const cli = sade('delegate-serve.js [space] [token] [accountDID] [gatewayDID]') | ||
|
|
||
| cli | ||
| .option('--space', 'The space DID to delegate. If not provided, a new space will be created.') | ||
| .option('--token', 'The auth token to use. If not provided, the delegation will not be authenticated.') | ||
| .option('--accountDID', 'The account DID to use when creating a new space.') | ||
| .option('--gatewayDID', 'The gateway DID to use when delegating the space/content/serve capability. Defaults to did:web:staging.w3s.link.') | ||
| .describe( | ||
| `Delegates ${serve.star.can} to the Gateway for <space>, with an optional token. Outputs a base64url string suitable for the stub_delegation query parameter. Pipe the output to pbcopy or similar for the quickest workflow.` | ||
| `Delegates ${Space.contentServe.can} to the Gateway for a test space generated by the script, with an optional auth token. Outputs a base64url string suitable for the stub_delegation query parameter. Pipe the output to pbcopy or similar for the quickest workflow.` | ||
| ) | ||
| .action(async (space, token) => { | ||
| .action(async (space, token, accountDID, gatewayDID, options) => { | ||
| const { space: spaceOption, token: tokenOption, accountDID: accountDIDOption, gatewayDID: gatewayDIDOption } = options | ||
| space = spaceOption || undefined | ||
| token = tokenOption || undefined | ||
| accountDID = accountDIDOption || undefined | ||
| gatewayDID = gatewayDIDOption || 'did:web:staging.w3s.link' | ||
| const client = await getClient() | ||
|
|
||
| const gatewayIdentity = (await ed25519.Signer.generate()).withDID( | ||
| 'did:web:w3s.link' | ||
| ) | ||
|
|
||
| const delegation = await serve.star.delegate({ | ||
| issuer: client.agent.issuer, | ||
| audience: gatewayIdentity, | ||
| with: space, | ||
| nb: { token: token ?? null }, | ||
| expiration: Infinity, | ||
| proofs: client.proofs([ | ||
| let spaceDID | ||
| let proofs = [] | ||
| if (!space) { | ||
| const provider = /** @type {`did:web:${string}`} */ (client.defaultProvider()) | ||
| const account = client.accounts()[accountDID] | ||
| const newSpace = await client.agent.createSpace('test') | ||
| const provision = await account.provision(newSpace.did(), { provider }) | ||
| if (provision.error) throw provision.error | ||
| await newSpace.save() | ||
| const authProof = await newSpace.createAuthorization(client.agent) | ||
| proofs = [authProof] | ||
| spaceDID = newSpace.did() | ||
| } else { | ||
| client.addSpace(space) | ||
| spaceDID = space | ||
| proofs = client.proofs([ | ||
| { | ||
| can: serve.star.can, | ||
| with: space | ||
| can: Space.contentServe.can, | ||
| with: spaceDID | ||
| } | ||
| ]) | ||
| } | ||
|
|
||
| /** @type {import('@ucanto/client').Principal<`did:${string}:${string}`>} */ | ||
| const gatewayIdentity = { | ||
| did: () => gatewayDID | ||
| } | ||
|
|
||
| // @ts-expect-error - The client still needs to be updated to support the capability type | ||
| const delegation = await client.createDelegation(gatewayIdentity, [Space.contentServe.can], { | ||
| expiration: Infinity, | ||
| proofs | ||
| }) | ||
|
|
||
| await client.capability.access.delegate({ | ||
| delegations: [delegation] | ||
| }) | ||
|
|
||
| const carResult = await delegation.archive() | ||
| if (carResult.error) throw carResult.error | ||
| process.stdout.write(Buffer.from(carResult.ok).toString('base64url')) | ||
| const base64Url = Buffer.from(carResult.ok).toString('base64url') | ||
| process.stdout.write(`Agent Proofs: ${proofs.flatMap(p => p.capabilities).map(c => `${c.can} with ${c.with}`).join('\n')}\n`) | ||
| process.stdout.write(`Issuer: ${client.agent.issuer.did()}\n`) | ||
| process.stdout.write(`Audience: ${gatewayIdentity.did()}\n`) | ||
| process.stdout.write(`Space: ${spaceDID}\n`) | ||
| process.stdout.write(`Token: ${token ?? 'none'}\n`) | ||
| process.stdout.write(`Delegation: ${delegation.capabilities.map(c => `${c.can} with ${c.with}`).join('\n')}\n`) | ||
| process.stdout.write(`Stubs: stub_space=${spaceDID}&stub_delegation=${base64Url}&authToken=${token ?? ''}\n`) | ||
| }) | ||
|
|
||
| cli.parse(process.argv) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.