-
Notifications
You must be signed in to change notification settings - Fork 16
Feature spec: compute extensibility user experience #96
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Feature spec: compute extensibility user experience #96
Conversation
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
… 2.5 Pro assisted in VSCode) Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
|
||
### Non-goals (out of scope) | ||
<!-- What are we explicitly not trying to accomplish? --> | ||
- Running the Radius control plane on a non-Kubernetes platform. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is out of scope for now or forever?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For now, it's do-able.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
out of scope for now, added a link to the roadmap item here.
} | ||
} | ||
// This container requests confidential compute | ||
extensions: [ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we support extensions on UDTs? Wouldn't these be additional optional properties consumed as recipe inputs instead?
1. **Configure Environment-Specific Recipe Parameters (Optional)**: | ||
* If recipes have parameters that need to be set globally for an environment, configure them using `rad recipe update` or during registration. | ||
|
||
#### Packaging and Registering a "Recipe Pack" for an Environment: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a fantastic implementation, I like that version is already included.
Signed-off-by: Will Tsai <[email protected]>
|
||
### Non-goals (out of scope) | ||
<!-- What are we explicitly not trying to accomplish? --> | ||
- Running the Radius control plane on a non-Kubernetes platform. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean that with extensibility and core types as UDTs we shouldn't expect to be able to host UCP in a different platform?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct
|
||
### Positive user outcome | ||
<!-- What is the positive outcome for the user if we deliver this, i.e. what is the value proposition for the user? Remember, this is user-centric. --> | ||
As a platform engineer, I can confidently adopt Radius across my organization, knowing I can extend its capabilities to support any compute platform, secret store, or gateway my teams require, without waiting for built-in support or modifying Radius core. I can register, customize, and share recipes that define how Radius provisions these resources, ensuring consistency with our application definitions while maintaining flexibility in our infrastructure choices and adhering to organizational standards. This empowers my development teams to leverage Radius benefits regardless of the underlying infrastructure. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are talking about containers, secret stores, and gateways a lot. Are these the main resource types that we are focusing on? Why are we not talking about other core resource types?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, we should include volumes, Will T said extenders are out of scope which I agree as I believe they'd just be replaced by new types/recipes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yup, added the inclustion of volume
resources too now.
} | ||
} | ||
- providers: { | ||
- // provider configurations no longer hard-coded here |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we saying we will add info such as subscription and respurce grpup to the bicep recipe itself?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We already have a requirement from some users that we will need to move those to the environment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On second thought, this shouldn't be encoded in the recipe. We can keep providers
as is for now.
Signed-off-by: Will Tsai <[email protected]>
* Once the Recipe Pack is registered, the environment is configured with all the specified recipes for the core UDTs. | ||
* When applications are deployed to this environment, Radius automatically uses the corresponding recipes from the pack to provision `Applications.Core/containers@2025-05-01-preview`, `Applications.Core/gateways@2025-05-01-preview`, and `Applications.Core/secretStores@2025-05-01-preview` resources. | ||
1. **Manage and Update Recipe Packs**: | ||
* Platform engineers can update the Recipe Pack manifest (e.g., point to new recipe versions, change default parameters) and re-register it. The CLI could offer options to overwrite existing registrations or manage versions of the pack within the environment. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to design/review/implement recipe versioning. Will that be a prerequisite before we work on recipe packs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think Recipes versioning will be a separate feature that we don't need to implement as a part of Recipe Packs. However, we should include versioning in scope for the implementation of Recipe Packs from the get go, even if the underlying Recipes packaged within the Pack don't support versioning yet.
Clarified this in the goals/non-goals
- } | ||
] | ||
// This container requests confidential compute | ||
+ runtimes: { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the confidential compute property be part of the ACI-specific recipe? Or is this an interim case while we look at Radius supporting conf containers?
Signed-off-by: Will Tsai <[email protected]>
2. **Discover/Create Recipes**: | ||
* Find community-provided recipes for desired platforms (e.g., ACI, AWS Fargate) from an OCI registry or Radius documentation. | ||
* Or, create custom Bicep/Terraform recipes for `Applications.Core/containers@2025-05-01-preview`, `Applications.Core/gateways@2025-05-01-preview`, and `Applications.Core/secretStores@2025-05-01-preview` to target a specific platform or customize existing behavior. | ||
3. **Register Recipes for Core Types**: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a way to constraint certain app files to be used against only certain recipe? Example: App with confidential container property set to true, should be allowed only on confidential recipe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not currently, we discussed this as well as the ability to only allow certain apps to use certain options/properties in a recipe.
|
||
## Topic Summary | ||
<!-- A paragraph or two to summarize the topic area. Just define it in summary form so we all know what it is. --> | ||
Radius will be enhanced to support multiple compute platforms, secret stores, and gateway resources through a recipe-based extensibility model. This approach decouples Radius's core logic from platform-specific provisioning code. Core resource types (`containers`, `gateways`, and `secretStores`) will be implemented as User-Defined Types (UDTs) and will allow platform engineers to register Bicep or Terraform recipes for them. Radius will provide default recipes for Kubernetes and Azure Container Instances (ACI), but platform engineers can use, modify, or replace these to customize how Radius provisions resources to different environments, or to add support for entirely new platforms without requiring changes to Radius core. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't core resource types include volumes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, it should. i'll add it.
} | ||
} | ||
} | ||
- providers: { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The provider stores the AWS and Azure details that get past to the recipe context right? https://docs.radapp.io/reference/context-schema/#environment So we'll need the AWS account and region, and Azure subscription and resource group in the environment somewhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, good point, this shouldn't be encoded in the recipe. We can keep providers
in the environment as is for now.
* The platform engineer configures `std-env` with recipes for `Applications.Core/containers@2025-05-01-preview` that deploy to standard compute (e.g., regular ACI or Kubernetes pods). These recipes might ignore or log a warning for confidential container requests if they don't support them. | ||
* Example recipe registration (conceptual): | ||
```bash | ||
rad recipe register std-container-recipe --environment std-env \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I strongly believe we should not have any recipe names. This leaks the implementation abstraction into the developers' world. If the developer needs a different recipe, that should be a different resource type.
// This container requests confidential compute | ||
extensions: [ | ||
{ | ||
kind: 'confidentialCompute' // Example extension kind |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there are additional properties needed to be set by the developer, that should be a different resource type. I do not think extension is the right thing here, nor do I understand how one would create an extension.
1. **Configure Environment-Specific Recipe Parameters (Optional)**: | ||
* If recipes have parameters that need to be set globally for an environment, configure them using `rad recipe update` or during registration. | ||
|
||
#### Packaging and Registering a "Recipe Pack" for an Environment: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A "resource pack" would be more useful. It would be a manifest of resource types and associated recipes. I would specify the resource pack during rad init
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we define a set of resources and a set of recipes within a single environment resource, and deploy that? That could replace the need for recipe or resource packs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@brooke-hamilton see answer in a previous comment: https://github.com/radius-project/design-notes/pull/96/files#r2141274374
@zachcasper - I'm thinking Recipes and Resources would be bundled into the same pack - it seems pointless to have them separately as one wouldn't be used without the other. Thoughts?
``` | ||
1. **Package the Recipe Pack (Optional but Recommended)**: | ||
* The manifest file and any local recipe files (if not using OCI URIs exclusively) could be bundled into an OCI artifact or a simple archive (e.g., .zip, .tar.gz) for easier distribution and versioning. | ||
1. **Register the Recipe Pack to an Environment**: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Register isn't the right verb here. We're configuring the environment. rad environment create --recipe-pack <location>
seems more accurate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, I've changed it to leverage the rad environment update my-env --recipe-pack ...
command instead.
This option is where the advanced capabilities are not exposed in the standard container properties provided by Radius out of the box, but rather require platform engineers to modify the container resource definitions to enable these capabilities. This could be done by creating a custom Radius Resource Type (RRT) that extends the standard container RRT to include platform-specific properties or configurations. | ||
|
||
**Pros:** | ||
- Keeps the standard container resource type clean and focused on core properties, avoiding clutter from platform-specific configurations or misuse of advanced container configurations by developers. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How is this cleaner than a runtimes property not used by developers in 99% of cases?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the common container properties that make up 90% of the developer use cases need to be elevated into the top-level container properties in the core container resource type. Properties such as restartPolicy
that aren't part of the top-level container properties today should added into the core container RRT as this is a configuration common across platforms. This would be the mitigation against the strong need for punch-throughs today.
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
- // These properties are removed from the volume resource | ||
- // and are instead defined in the recipe | ||
- kind: 'azure.com.keyvault' | ||
- resource: keyvault.id | ||
- secrets: { | ||
- mysecret: { | ||
- name: 'mysecret' | ||
- } | ||
- } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that this would replace the Azure KeyVault specific implementation of volume that we have today: https://docs.radapp.io/reference/resource-schema/core-schema/volumes/azure-keyvault/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is there an implied restriction for an environment to use one type of volume to be attached to containers since a default recipe will be associated with Applications.Core/volumes@2025-05-01-preview
. (the earlier implementation had a kind
field ).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, the implementation of volumes
will be done on the Recipe going forward
Signed-off-by: Will Tsai <[email protected]>
Signed-off-by: Will Tsai <[email protected]>
Co-authored-by: Zach Casper <[email protected]> Signed-off-by: Will <[email protected]>
* Familiarize yourself with the Radius project's general contribution guidelines (often found in `CONTRIBUTING.md`). | ||
* Check for any specific guidelines related to recipes or extensibility. | ||
2. **Develop Your Recipe(s)**: | ||
* Create high-quality, well-tested Bicep or Terraform recipes for one or more core types (`Applications.Core/containers@2025-05-01-preview`, `Applications.Core/gateways@2025-05-01-preview`, `Applications.Core/secretStores@2025-05-01-preview`). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the new community contributed types, what namespace should we have them follow? Our tutorials and samples reference Radius.Resources
as the namespace but the core types have Applications.Core
. I think it should be consistent across the core types and the others.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good question. I'll bring this up for discussion on the naming convention for RRTs going forward.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
High level thoughts:
- What are the proposed schema changes for the core types? I would expect the new schema to be defined in this doc.
- I'm very confused and concerned about
allowAdvancedContainerCapabilities
but there are not a lot of details here. I'm not supportive of an unstructured map. We should find the platform-specific properties that we want to expose for ACI and Kubernetes and put them in the extensions block. - What does a vanilla environment look like? Does it have all of the recipes enumerated?
- Where are the recipes stored? How does that work in an air-gapped environment.
- We need a user story for modifying the schema for a core type. I can use rad resource-type create but I will need the original resource definition. Today there is no way to get the current definition.
<!-- Step 1 | ||
Step 2 | ||
… --> | ||
#### Setting up and deploying an application to a Radius environment using existing Recipes for core types: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do not understand what this user story is really saying. There are two personas here in one sentence, the platform engineer creating the environment and the developer deploying an application. There should only be one persona to keep the doc clear.
I think you are saying As a platform engineer, I need to create a new Radius environment that when applications are deployed to it, the containers are deployed to Azure Container Instances. But I'm not sure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, that is right, i'll edit to make this more clear
#### Setting up and deploying an application to a Radius environment using existing Recipes for core types: | ||
|
||
1. **Discover existing Recipes**: | ||
* Find community-provided recipes for desired platforms (e.g., ACI, AWS Fargate) from an OCI registry or Terraform module repo, or something like [Artifact Hub](https://artifacthub.io/). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why would the platform engineer need to find community-provided recipes? Radius should ship maintained and tested recipes for various container platforms.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there's two paths:
- Platform engineer uses the "stock" maintained and tested recipe packs that come with Radius for various container platforms.
- Platform engineer wants to mix-n-match the "stock" recipes with some others, perhaps community-contributed ones they find or (more likely) bring their own custom recipes. They can either build their own recipe packs using this mixed bag of recipes, or they can register each individual recipe into their environment.
I'll make this distinction more clear with illustrated examples of both.
resource env 'Applications.Core/environments@2025-05-01-preview' = { | ||
name: 'my-aci-env' | ||
properties: { | ||
- compute: { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At a minimum, you are removing the Kubernetes namespace. You're also removing the Identity property, but I do not understand what the property does.
Where are you re-defining the Kubernetes namespace for this environment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good catch, the k8s namespace will move to the providers
property in the Environment resource
'Applications.Core/containers': { | ||
default: { | ||
templateKind: 'bicep' | ||
plainHttp: true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does plainHttp
mean? If this is an existing property, it is not documented and is confusing in this context.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is a typo, i'll remove it
- } | ||
] | ||
// This container requests confidential compute | ||
+ runtimes: { // Consider renaming this to `options` or `platform`, etc. for clarity |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like extensions
. It's a platform extension. Runtime doesn't make sense to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't like extensions
because there is already an extensions
property that will hold the Dapr configs. I propose we go with platformOptions
* `rad deploy ./app.bicep --environment std-env` | ||
* Radius uses the `std-container-recipe` registered in `std-env`. | ||
* The `frontend` container is deployed as a standard container. | ||
* The `backend` container, despite its platform-specific confidential container property, is deployed as a standard container because the `std-container-recipe` in `std-env` does not support confidential compute (or is configured to treat it as standard). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see any platform-specific properties on the backend container.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will add concrete examples
name: 'my-aci-env' | ||
properties: { | ||
recipes: { | ||
+ 'Applications.Core/volumes': { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm confused as to what a default, vanilla environment definition looks like. Does it have recipes for every core resource type? If it doesn't, what recipes does an environment point to by default? If it's jutst GitHub, how would this work in an air-gapped environment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the default environments that ship with Radius will have platform specific recipe packs per environment, I will add an example to show that.
application: app.id | ||
- // These properties are removed from the volume resource | ||
- // and are instead defined in the recipe | ||
- kind: 'azure.com.keyvault' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we only support Azure Key Vault volume types? If that's the case, shouldn't this just be a secret? I don't think you can have a generic volume resource type and push the kind into the recipe if it's going to be a Key Vault key.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is supported today, where you can mount a secretstore (Azure Key Vault) to a container as a volume: https://docs.radapp.io/reference/resource-schema/core-schema/volumes/azure-keyvault/
#### Initialize Radius for specific platforms with default recipes: | ||
|
||
1. **Kubernetes**: | ||
* `rad init kubernetes` initializes a Radius environment for Kubernetes with default recipes for core types. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We retain the interactive questions for rad init
rather than introduce sub-commands. Keep the sub-commands on rad install
.
Co-authored-by: Zach Casper <[email protected]> Signed-off-by: Will <[email protected]>
Add feature spec for compute extensibility to define requirements and user experience