Adding GenericBundle

[brnstz]

Issue #, if available:

Description of changes:

Replace CodeRepoBundle with GenericBundle.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[shinstev333]

should we include a metadata field? We can use that to include codeRepo or other fields into it.

[brnstz]

We could.... but why not model these fields directly when we need them?

[shinstev333]

I prefer not to model these fields directly since we're unsure if they'll be required for all generic bundles. A metadata map gives us more flexibility while we're still discovering what fields are truly core to all bundles. Once our bundle model becomes more defined and we better understand which fields are consistently needed, we can then move them into direct fields in the model

[brnstz]

Hey Steven, this is the second time you mentioned this metadata field, so I don't want to discount it entirely since you've done most of the API work.

Still it looks like Richard and I both don't see the utility of having unmodeled metadata.

My main concern is that we're going to end up storing important data in metadata, people will start depending on it, then will will be stuck with it because we don't even have a way to deprecate a particular field.

As far as codeRepoUrl, I don't think we have a reason to store this currently.

Is there something I'm missing here?

[rhernandez35]

We can't put the blob inside the bundle. It's going to be inflated by up to 33% during b64 encoding, and the resulting artifacts are going to be gigantic. I'd like to make this field a union to capture the two cases we have in mind right now:

union GenericArtifact {
  /// We need to call the registry again to download this artifact.
  binary: ArtifactId
  /// There's nothing to download. The bundle already contains everything we need
  /// to install and run the binary.
  empty: Unit
}

/// Describes an artifact stored in the registry
structure ArtifactId {
  version: String
  name: String
}

operation GetArtifact {
   input := {
     @required
     artifactId: ArtifactId
  }
  output := {
    @required
    payload: StreamingBlob
  }
}

We can add the download stuff later.

[adwsingh]

+1. You can have something like the Location union we use in the MCP CLI. It isn't even necessary we might download the artifact from the registry service, it could also be something like a S3 Presigned URL or just null.

[brnstz]

Agreed we shouldn't expose the blob here.

Can we get away with.... ?

structure GenericBundle with [CommonBundleConfig] {
    artifactLocation: Location
    ....
}

union Location {
    /// A presigned S3 URL to download the artifact
    url: String
    /// If no download is necessary
    empty: Unit
}

And not need the GetArtifact operation?

[rhernandez35]

I have two problems with the presigned url:

1. It directly exposes S3 as our storage medium when it should be an implementation detail. That would prevent us from doing something like adding caching for hot bundles in the future.
2. Presigned URLs expire but we may need to re-download the artifact at any time (for example, if the user manually deletes the downloaded binary)

[brnstz]

OK. I pushed another commit with this in mind. Main difference from what's posted here is that ArtifactId is simply an opaque string rather than a name/version. I am thinking we can generate unique strings for artifact download rather than requiring the user pass in the name/version structure

[brnstz]

In interest of time, support only the "empty" format.