From a6876d145efc79fa3cefb2f804d3f9af2f024576 Mon Sep 17 00:00:00 2001 From: MichaelTansiniSeqera Date: Fri, 12 Jun 2026 16:27:18 +0100 Subject: [PATCH 01/11] docs: add Intelligent Compute page for AWS Cloud (preview) Adds a new standalone page covering the Seqera Intelligent Compute preview feature for AWS Cloud compute environments, including IAM permissions, setup steps, and configuration options. Updates the Cloud sidebar to include the new page after aws-cloud. Co-Authored-By: Claude Sonnet 4.6 --- platform-cloud/cloud-sidebar.json | 1 + .../docs/compute-envs/intelligent-compute.mdx | 106 ++++++++++++++++++ 2 files changed, 107 insertions(+) create mode 100644 platform-cloud/docs/compute-envs/intelligent-compute.mdx diff --git a/platform-cloud/cloud-sidebar.json b/platform-cloud/cloud-sidebar.json index 8eef893f4..b2238ce14 100644 --- a/platform-cloud/cloud-sidebar.json +++ b/platform-cloud/cloud-sidebar.json @@ -69,6 +69,7 @@ "compute-envs/seqera-compute", "compute-envs/aws-batch", "compute-envs/aws-cloud", + "compute-envs/intelligent-compute", "compute-envs/azure-batch", "compute-envs/azure-cloud", "compute-envs/google-cloud-batch", diff --git a/platform-cloud/docs/compute-envs/intelligent-compute.mdx b/platform-cloud/docs/compute-envs/intelligent-compute.mdx new file mode 100644 index 000000000..cf3737cac --- /dev/null +++ b/platform-cloud/docs/compute-envs/intelligent-compute.mdx @@ -0,0 +1,106 @@ +--- +title: "Intelligent Compute" +description: "Set up Seqera Intelligent Compute on an AWS Cloud compute environment" +date created: "2026-06-12" +last updated: "2026-06-12" +tags: [intelligent compute, aws, ecs, compute environments] +toc_min_heading_level: 2 +toc_max_heading_level: 4 +--- + +import CodeBlock from '@theme/CodeBlock'; +import AwsCloudIntelligentComputePolicy from './_policies/aws-cloud-intelligent-compute-policy.json?raw'; + +:::info[Preview] +Seqera Intelligent Compute is in preview and must be enabled for your organization by Seqera before you can use it. Contact your account manager to request access. +::: + +:::caution +Intelligent Compute may assign different CPU and memory values to tasks than those specified in your pipeline's `process` directives. The scheduler selects the most cost-effective instance that meets each task's resource request rather than provisioning exactly what the directive specifies. +::: + +Intelligent Compute is supported on **AWS Cloud compute environments only**. + +## What is Intelligent Compute + +Intelligent Compute is a scheduling service that runs Nextflow pipelines on a Seqera-managed Amazon ECS cluster. It allocates compute resources based on what each task actually needs rather than what the pipeline requests, reducing cost and improving utilization across a run. + +Unlike the standard AWS Cloud compute environment, which runs a pipeline on a single EC2 instance with a local executor, Intelligent Compute provisions and manages a multi-node ECS cluster. This allows pipelines to scale beyond a single instance while preserving fast startup times. + +When Intelligent Compute is enabled on an AWS Cloud compute environment, Seqera provisions and manages the following resources in your AWS account on first use: + +- An Amazon ECS cluster per compute environment configuration +- ECS capacity providers (Managed Instances or Auto Scaling Groups) +- ECS task definitions per container image and resource shape +- IAM roles for ECS task execution, EC2 instance profiles, and infrastructure management +- CloudWatch log groups under `/seqera/sched` + +All managed resources use the `seqera-sched-` prefix. Seqera creates them on first use and removes them automatically when no longer needed. + +## IAM permissions + +In addition to the [standard AWS Cloud IAM permissions](./aws-cloud#required-platform-iam-permissions), Intelligent Compute requires an additional policy attached to the same IAM user or role that Seqera uses to access your AWS account. + +
+Intelligent Compute IAM policy + +{AwsCloudIntelligentComputePolicy} + +
+ +[Download aws-cloud-intelligent-compute-policy.json](./_policies/aws-cloud-intelligent-compute-policy.json) + +### What each permission group does + +| Group | Purpose | +|-------|---------| +| `ECSScopedOperations` | Create, delete, describe, and tag ECS clusters, capacity providers, and tasks. Scoped to `seqera-sched-*` resources. | +| `ECSUnscopedOperations` | Register, deregister, list, and describe ECS task definitions. ECS task definition APIs do not support resource-level permissions. | +| `IAMRoleManagement` | Create, update, and delete IAM roles and instance profiles scoped to `seqera-sched-*`. Seqera creates four role types on first use: execution role, infrastructure role, per-cluster instance role, and per-cluster task role. | +| `PassRoleToECS` | Pass `seqera-sched-*` and `TowerForge-*` roles to ECS, ECS tasks, and EC2. Required to attach roles to ECS infrastructure and task definitions. | +| `ServiceLinkedRoles` | Create service-linked roles for ECS, autoscaling, and Spot. Required only if these roles do not already exist in your account. | +| `CloudWatchLogs` | Create and manage log groups under `/seqera/sched`, and read log events. Task stdout and stderr are written to CloudWatch. | +| `EC2NetworkDiscovery` | Describe VPCs, subnets, security groups, and route tables. Create security groups and VPC endpoints. Used for VPC auto-discovery and network setup. | +| `ECRAccess` | Authorize ECR and pull container images. ECS tasks pull images from ECR. | +| `S3Access` | Read objects and list buckets. Used to read Fusion trace files and pipeline work directory content. | +| `ASGEC2Operations` | Describe instance types and create or delete EC2 launch templates. Required only for Auto Scaling Group-backed clusters. | +| `ASGManagement` | Create, update, and delete Auto Scaling Groups scoped to `seqera-sched-*`. Required only for Auto Scaling Group-backed clusters. | +| `ASGDescribe` | Describe Auto Scaling Groups. Required only for Auto Scaling Group-backed clusters. | +| `SSMECSOptimizedAmi` | Read the ECS-optimized AMI ID from SSM Parameter Store. Used to look up the latest Amazon Linux 2023 ECS-optimized AMI. | +| `CostExplorer` | Query `ce:GetCostAndUsage`. Used to display cost predictions at pipeline launch. If this permission is absent, cost predictions do not appear. No error is surfaced to users. | + +**Conditional statements:** +- `ASGEC2Operations`, `ASGManagement`, and `ASGDescribe` are required only if Auto Scaling Group-backed clusters are enabled. You can omit them for Managed Instances deployments. +- `ServiceLinkedRoles` is required only if the listed service-linked roles do not already exist in your AWS account. +- `CostExplorer` is required only if you want cost predictions shown at pipeline launch. + +### Create the additional IAM policy + +1. Open the [AWS IAM console](https://console.aws.amazon.com/iam). +1. Select **Policies** under **Access management**, then select **Create policy**. +1. Select the **JSON** tab and paste the Intelligent Compute policy. +1. Select **Next**, enter a name (for example, `SeqeraIntelligentComputePolicy`), then select **Create policy**. +1. Attach the policy to the same IAM user or role that Seqera uses for your AWS Cloud compute environment. + +## Set up an AWS Cloud compute environment with Intelligent Compute + +Confirm with your account manager that Intelligent Compute is enabled for your organization before proceeding. + +1. In your Seqera workspace, select **Compute Environments**, then **Add compute environment**. +1. Enter a name and select **AWS Cloud** as the platform. +1. Select your AWS credentials. The credential must have both the standard AWS Cloud permissions and the Intelligent Compute permissions attached. +1. Select the **Region** where the ECS cluster will be provisioned. +1. Enter a **Work directory** (S3 URI, for example `s3://my-bucket/work`). +1. Under **Intelligent Compute**, enable the **Seqera Intelligent Compute** toggle. +1. Configure the [Intelligent Compute options](#configuration-options) below as needed. +1. Select **Add**. + +Seqera validates credentials and configuration on save. On first use, it provisions the required IAM roles and ECS cluster in your account. Provisioning is automatic and does not require additional steps. + +## Configuration options + +| Option | Values | Default | Description | +|--------|--------|---------|-------------| +| **Seqera Intelligent Compute** | Enabled / Disabled | Disabled | Enables the Intelligent Compute scheduler for this compute environment. This option only appears if Intelligent Compute is enabled for your organization. | +| **Provisioning model** | `spotFirst`, `spot`, `ondemand` | `spotFirst` | Instance procurement strategy. `spotFirst` uses Spot instances and falls back to On-Demand if Spot capacity is unavailable. `spot` uses Spot instances only. `ondemand` uses On-Demand instances only. | +| **Instance types** | Comma-separated EC2 instance type identifiers (for example, `m5.xlarge, c5.2xlarge`) | Empty | Restricts which instance types the scheduler can select. When empty, the scheduler selects the most cost-effective type for each task automatically. Specifying types here overrides automatic selection. | From 26cd6d29aaa77ca1e152204c0c0d05e50c653a4b Mon Sep 17 00:00:00 2001 From: MichaelTansiniSeqera Date: Fri, 12 Jun 2026 16:32:03 +0100 Subject: [PATCH 02/11] Update intelligent-compute.mdx Signed-off-by: MichaelTansiniSeqera --- platform-cloud/docs/compute-envs/intelligent-compute.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform-cloud/docs/compute-envs/intelligent-compute.mdx b/platform-cloud/docs/compute-envs/intelligent-compute.mdx index cf3737cac..16308d35d 100644 --- a/platform-cloud/docs/compute-envs/intelligent-compute.mdx +++ b/platform-cloud/docs/compute-envs/intelligent-compute.mdx @@ -25,7 +25,7 @@ Intelligent Compute is supported on **AWS Cloud compute environments only**. Intelligent Compute is a scheduling service that runs Nextflow pipelines on a Seqera-managed Amazon ECS cluster. It allocates compute resources based on what each task actually needs rather than what the pipeline requests, reducing cost and improving utilization across a run. -Unlike the standard AWS Cloud compute environment, which runs a pipeline on a single EC2 instance with a local executor, Intelligent Compute provisions and manages a multi-node ECS cluster. This allows pipelines to scale beyond a single instance while preserving fast startup times. +Unlike the standard AWS Cloud compute environment, which runs a pipeline on a single EC2 instance with a local executor, Intelligent Compute provisions and manages multi-node clusters. This allows pipelines to scale beyond a single instance while preserving fast startup times. When Intelligent Compute is enabled on an AWS Cloud compute environment, Seqera provisions and manages the following resources in your AWS account on first use: From d9cdab0f2ba9d88e42b1d04e6542a912edfbe591 Mon Sep 17 00:00:00 2001 From: MichaelTansiniSeqera Date: Fri, 12 Jun 2026 16:39:55 +0100 Subject: [PATCH 03/11] docs: add IAM policy inline, resource metrics, and task status reference MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Remove
collapse from IAM policy — policy is now always visible and copy-pasteable - Add Resource metrics section explaining Requested/Allocated/Used and how to interpret the gap between them - Add Task and run statuses reference table for troubleshooting Co-Authored-By: Claude Sonnet 4.6 --- .../docs/compute-envs/intelligent-compute.mdx | 46 +++++++++++++++++-- 1 file changed, 41 insertions(+), 5 deletions(-) diff --git a/platform-cloud/docs/compute-envs/intelligent-compute.mdx b/platform-cloud/docs/compute-envs/intelligent-compute.mdx index 16308d35d..b6ec943ff 100644 --- a/platform-cloud/docs/compute-envs/intelligent-compute.mdx +++ b/platform-cloud/docs/compute-envs/intelligent-compute.mdx @@ -41,13 +41,8 @@ All managed resources use the `seqera-sched-` prefix. Seqera creates them on fir In addition to the [standard AWS Cloud IAM permissions](./aws-cloud#required-platform-iam-permissions), Intelligent Compute requires an additional policy attached to the same IAM user or role that Seqera uses to access your AWS account. -
-Intelligent Compute IAM policy - {AwsCloudIntelligentComputePolicy} -
- [Download aws-cloud-intelligent-compute-policy.json](./_policies/aws-cloud-intelligent-compute-policy.json) ### What each permission group does @@ -97,6 +92,22 @@ Confirm with your account manager that Intelligent Compute is enabled for your o Seqera validates credentials and configuration on save. On first use, it provisions the required IAM roles and ECS cluster in your account. Provisioning is automatic and does not require additional steps. +## Resource metrics + +The **Metrics** tab for a workflow run on Intelligent Compute shows three resource values for CPU and memory: **Requested**, **Allocated**, and **Used**. + +| Metric | Source | What it represents | +|--------|--------|-------------------| +| **Requested** | Pipeline `process` directives | The CPU and memory your pipeline asked for, as written in your `process` directives (for example, `cpus = 4`, `memory = 8 GB`). | +| **Allocated** | Scheduler decision | The CPU and memory the scheduler actually assigned to the task container. Intelligent Compute may assign values different from what was requested — it selects the most cost-effective instance shape that satisfies the task's requirements. | +| **Used** | Nextflow trace data | The CPU and memory the task actually consumed, measured from Nextflow's trace metrics (`pcpu` × `realtime` for CPU, `peakRss` for memory). Requires Fusion to be enabled. Absent for tasks that did not produce trace data. | + +**How to read the numbers:** + +- If **Requested** is much higher than **Allocated**, the scheduler found a more efficient instance shape than your directives implied. +- If **Allocated** is much higher than **Used**, the task ran with significant idle headroom. You may be able to reduce your process resource directives on future runs to lower cost. +- If **Used** is close to **Allocated**, resource utilization is near-optimal for that task. + ## Configuration options | Option | Values | Default | Description | @@ -104,3 +115,28 @@ Seqera validates credentials and configuration on save. On first use, it provisi | **Seqera Intelligent Compute** | Enabled / Disabled | Disabled | Enables the Intelligent Compute scheduler for this compute environment. This option only appears if Intelligent Compute is enabled for your organization. | | **Provisioning model** | `spotFirst`, `spot`, `ondemand` | `spotFirst` | Instance procurement strategy. `spotFirst` uses Spot instances and falls back to On-Demand if Spot capacity is unavailable. `spot` uses Spot instances only. `ondemand` uses On-Demand instances only. | | **Instance types** | Comma-separated EC2 instance type identifiers (for example, `m5.xlarge, c5.2xlarge`) | Empty | Restricts which instance types the scheduler can select. When empty, the scheduler selects the most cost-effective type for each task automatically. Specifying types here overrides automatic selection. | + +## Task and run statuses + +### Task statuses + +| Status | Description | +|--------|-------------| +| `SUBMITTED` | Task is queued or submitted to the compute backend. | +| `RUNNING` | Task is actively executing on a compute instance. | +| `SUCCEEDED` | Task completed with exit code 0. | +| `FAILED` | Task failed. This covers non-retriable execution failures (non-zero exit code, container startup errors) and spot quota exhaustion after retries are exhausted. | +| `CANCELLED` | Task was cancelled by the user. | +| `PREEMPTED` | The Spot instance running this task was reclaimed by AWS. The scheduler retries the task automatically. If the retry limit is reached, the task transitions to `FAILED`. | +| `UNSCHEDULABLE` | No instance type could satisfy the task's placement constraints. This occurs when the requested resources exceed what any available instance type can provide, or when specified instance types are unavailable in the region. Check your **Instance types** configuration and the `cpus` and `memory` directives in the failing process. | +| `UNKNOWN` | Task status could not be determined, typically due to a transient backend failure. | + +### Run statuses + +| Status | Description | +|--------|-------------| +| `ACTIVE` | The run is in progress. | +| `TERMINATING` | The run is shutting down — final tasks are completing or being cancelled. | +| `TERMINATED` | The run ended normally. | +| `FAILED` | The run failed. | +| `DANGLING` | The Nextflow process stopped sending heartbeats. This typically means the launcher process crashed or lost connectivity. Tasks already dispatched to ECS may still be running. Check CloudWatch logs under `/seqera/sched` for details. | From 9c443b47b5b36f5ff9fe73a45b920a2373e03182 Mon Sep 17 00:00:00 2001 From: MichaelTansiniSeqera Date: Fri, 12 Jun 2026 17:02:34 +0100 Subject: [PATCH 04/11] docs: add serviceAccountTokenCreator requirement for WIF + Data Explorer on GCP When WIF credentials are used for Data Explorer, Platform has no embedded private key and must call the GCP IAM signBlob API to generate presigned URLs. roles/iam.serviceAccountTokenCreator on the SA itself is required for this to succeed. Without it, file viewing and download fail silently with a signing error. Running pipelines is unaffected. Updates both Cloud and Enterprise Google Cloud Batch docs. Co-Authored-By: Claude Sonnet 4.6 --- platform-cloud/docs/compute-envs/google-cloud-batch.md | 9 +++++++++ .../compute-envs/google-cloud-batch.md | 9 +++++++++ 2 files changed, 18 insertions(+) diff --git a/platform-cloud/docs/compute-envs/google-cloud-batch.md b/platform-cloud/docs/compute-envs/google-cloud-batch.md index 13a2aa078..5a008f10c 100644 --- a/platform-cloud/docs/compute-envs/google-cloud-batch.md +++ b/platform-cloud/docs/compute-envs/google-cloud-batch.md @@ -112,6 +112,15 @@ Setting up WIF requires the following steps in the GCP Console: tityPools/{POOL}/providers/{PROVIDER}`. If you specify a custom value, it must match exactly what you enter in the Token audience field when creating the Google WIF credential in Seqera. 4. Define an attribute mapping and condition. At a minimum set `google.subject=assertion.sub`. This maps the subject claim from Seqera's JWT to GCP's identity space. For more information see [here](https://docs.cloud.google.com/iam/docs/workload-identity-federation-with-other-providers#mappings-and-conditions). You may see a pop-up asking to configure your application and provide an OIDC ID token path. This pop-up can be dismissed. 5. Grant `roles/iam.workloadIdentityUser` on the service account that WIF will impersonate to the Workload Identity Pool principal. This can be set for all pool identities or for a specific workspace. If you have not yet created a service account do so following the guidelines above. +6. If you use the same WIF credential for Data Explorer, grant `roles/iam.serviceAccountTokenCreator` on the service account to itself: + + ```bash + gcloud iam service-accounts add-iam-policy-binding SA_EMAIL \ + --member="serviceAccount:SA_EMAIL" \ + --role="roles/iam.serviceAccountTokenCreator" + ``` + + Replace `SA_EMAIL` with the service account email. Without this role, viewing or downloading file contents in Data Explorer fails with a signing error. Running pipelines is not affected. After setting up WIF in the GCP Console, you need the following information to create a credential in Seqera Platform: diff --git a/platform-enterprise_docs/compute-envs/google-cloud-batch.md b/platform-enterprise_docs/compute-envs/google-cloud-batch.md index 9f89ce681..f58311f98 100644 --- a/platform-enterprise_docs/compute-envs/google-cloud-batch.md +++ b/platform-enterprise_docs/compute-envs/google-cloud-batch.md @@ -107,6 +107,15 @@ Workload Identity Federation (WIF) is the recommended authentication method for 3. Set the Allowed audiences. If left empty, GCP derives a default audience from the provider resource path in the format `//iam.googleapis.com/projects/{PROJECT}/locations/global/workloadIdentityPools/{POOL}/providers/{PROVIDER}`. If you specify a custom value, it must match exactly what you enter in the Token audience field when creating the Google WIF credential in Seqera. 4. Define an attribute mapping and condition. At a minimum set `google.subject=assertion.sub`. This maps the subject claim from Seqera's JWT to GCP's identity space. For more information see [here](https://docs.cloud.google.com/iam/docs/workload-identity-federation-with-other-providers#mappings-and-conditions) 5. Grant `roles/iam.workloadIdentityUser` on the service account created above to the Workload Identity Pool principal. This can be set for all pool identities or for a specific workspace. +6. If you use the same WIF credential for Data Explorer, grant `roles/iam.serviceAccountTokenCreator` on the service account to itself: + + ```bash + gcloud iam service-accounts add-iam-policy-binding SA_EMAIL \ + --member="serviceAccount:SA_EMAIL" \ + --role="roles/iam.serviceAccountTokenCreator" + ``` + + Replace `SA_EMAIL` with the service account email. Without this role, viewing or downloading file contents in Data Explorer fails with a signing error. Running pipelines is not affected. WIF requires an OIDC signing key and for Seqera Platform's OIDC provider to be configured. See [Cryptographic options](https://docs.seqera.io/platform-enterprise/enterprise/configuration/overview#cryptographic-options). From 99e38703f9075c176b16cd8d3069fba51140a2b1 Mon Sep 17 00:00:00 2001 From: Chris Hakkaart Date: Tue, 16 Jun 2026 16:05:39 +1200 Subject: [PATCH 05/11] docs: Docs/intelligent compute AWS preview (#1567) * Deslop and tidy tables * Remove bold * Update intelligent-compute.mdx Signed-off-by: MichaelTansiniSeqera * [automated] Fix code formatting * chore: add 'intelligent compute' to doc-tags-allowed.txt --------- Signed-off-by: MichaelTansiniSeqera Co-authored-by: MichaelTansiniSeqera Co-authored-by: Seqera Docs Bot Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> --- .github/doc-tags-allowed.txt | 1 + .../docs/compute-envs/intelligent-compute.mdx | 70 +++++++++++-------- 2 files changed, 40 insertions(+), 31 deletions(-) diff --git a/.github/doc-tags-allowed.txt b/.github/doc-tags-allowed.txt index b3f773621..a1d040716 100644 --- a/.github/doc-tags-allowed.txt +++ b/.github/doc-tags-allowed.txt @@ -141,6 +141,7 @@ image input installation integration +intelligent compute interactive jupyter k8s diff --git a/platform-cloud/docs/compute-envs/intelligent-compute.mdx b/platform-cloud/docs/compute-envs/intelligent-compute.mdx index b6ec943ff..bef759639 100644 --- a/platform-cloud/docs/compute-envs/intelligent-compute.mdx +++ b/platform-cloud/docs/compute-envs/intelligent-compute.mdx @@ -12,20 +12,16 @@ import CodeBlock from '@theme/CodeBlock'; import AwsCloudIntelligentComputePolicy from './_policies/aws-cloud-intelligent-compute-policy.json?raw'; :::info[Preview] -Seqera Intelligent Compute is in preview and must be enabled for your organization by Seqera before you can use it. Contact your account manager to request access. +Seqera Intelligent Compute is in preview. Seqera must enable it for your organization before you can use it. Contact your account manager to request access. ::: :::caution Intelligent Compute may assign different CPU and memory values to tasks than those specified in your pipeline's `process` directives. The scheduler selects the most cost-effective instance that meets each task's resource request rather than provisioning exactly what the directive specifies. ::: -Intelligent Compute is supported on **AWS Cloud compute environments only**. +Intelligent Compute is a scheduling service that runs Nextflow pipelines on a Seqera-managed Amazon ECS cluster. It allocates compute resources based on what each task needs rather than what the pipeline requests, which reduces cost and improves utilization across a run. Intelligent Compute is supported on AWS Cloud compute environments only. -## What is Intelligent Compute - -Intelligent Compute is a scheduling service that runs Nextflow pipelines on a Seqera-managed Amazon ECS cluster. It allocates compute resources based on what each task actually needs rather than what the pipeline requests, reducing cost and improving utilization across a run. - -Unlike the standard AWS Cloud compute environment, which runs a pipeline on a single EC2 instance with a local executor, Intelligent Compute provisions and manages multi-node clusters. This allows pipelines to scale beyond a single instance while preserving fast startup times. +Unlike the standard AWS Cloud compute environment, which runs a pipeline on a single EC2 instance with a local executor, Intelligent Compute provisions and manages multi-node clusters. Pipelines can scale beyond a single instance while keeping startup times short. When Intelligent Compute is enabled on an AWS Cloud compute environment, Seqera provisions and manages the following resources in your AWS account on first use: @@ -41,11 +37,14 @@ All managed resources use the `seqera-sched-` prefix. Seqera creates them on fir In addition to the [standard AWS Cloud IAM permissions](./aws-cloud#required-platform-iam-permissions), Intelligent Compute requires an additional policy attached to the same IAM user or role that Seqera uses to access your AWS account. -{AwsCloudIntelligentComputePolicy} +
+ IAM policy JSON + {AwsCloudIntelligentComputePolicy} +
[Download aws-cloud-intelligent-compute-policy.json](./_policies/aws-cloud-intelligent-compute-policy.json) -### What each permission group does +### Permission groups | Group | Purpose | |-------|---------| @@ -62,7 +61,7 @@ In addition to the [standard AWS Cloud IAM permissions](./aws-cloud#required-pla | `ASGManagement` | Create, update, and delete Auto Scaling Groups scoped to `seqera-sched-*`. Required only for Auto Scaling Group-backed clusters. | | `ASGDescribe` | Describe Auto Scaling Groups. Required only for Auto Scaling Group-backed clusters. | | `SSMECSOptimizedAmi` | Read the ECS-optimized AMI ID from SSM Parameter Store. Used to look up the latest Amazon Linux 2023 ECS-optimized AMI. | -| `CostExplorer` | Query `ce:GetCostAndUsage`. Used to display cost predictions at pipeline launch. If this permission is absent, cost predictions do not appear. No error is surfaced to users. | +| `CostExplorer` | Query `ce:GetCostAndUsage`. Used to display cost predictions at pipeline launch. If this permission is absent, cost predictions do not appear and Seqera shows no error. | **Conditional statements:** - `ASGEC2Operations`, `ASGManagement`, and `ASGDescribe` are required only if Auto Scaling Group-backed clusters are enabled. You can omit them for Managed Instances deployments. @@ -79,33 +78,40 @@ In addition to the [standard AWS Cloud IAM permissions](./aws-cloud#required-pla ## Set up an AWS Cloud compute environment with Intelligent Compute -Confirm with your account manager that Intelligent Compute is enabled for your organization before proceeding. +:::info[**Prerequisites**] + +You need the following: + +- Intelligent Compute enabled for your organization by Seqera. Contact your account manager to request access. +- AWS credentials with both the standard AWS Cloud permissions and the Intelligent Compute permissions attached. + +::: 1. In your Seqera workspace, select **Compute Environments**, then **Add compute environment**. 1. Enter a name and select **AWS Cloud** as the platform. -1. Select your AWS credentials. The credential must have both the standard AWS Cloud permissions and the Intelligent Compute permissions attached. -1. Select the **Region** where the ECS cluster will be provisioned. +1. Select your AWS credentials. +1. Select the **Region** where Seqera provisions the ECS cluster. 1. Enter a **Work directory** (S3 URI, for example `s3://my-bucket/work`). 1. Under **Intelligent Compute**, enable the **Seqera Intelligent Compute** toggle. -1. Configure the [Intelligent Compute options](#configuration-options) below as needed. +1. Configure the [Intelligent Compute options](#configuration-options) as needed. 1. Select **Add**. -Seqera validates credentials and configuration on save. On first use, it provisions the required IAM roles and ECS cluster in your account. Provisioning is automatic and does not require additional steps. +Seqera validates credentials and configuration on save. On first use, it provisions the required IAM roles and ECS cluster in your account. This provisioning is automatic and requires no further steps. ## Resource metrics -The **Metrics** tab for a workflow run on Intelligent Compute shows three resource values for CPU and memory: **Requested**, **Allocated**, and **Used**. +The **Metrics** tab for a run on Intelligent Compute shows three resource values for CPU and memory: **Requested**, **Allocated**, and **Used**. | Metric | Source | What it represents | |--------|--------|-------------------| | **Requested** | Pipeline `process` directives | The CPU and memory your pipeline asked for, as written in your `process` directives (for example, `cpus = 4`, `memory = 8 GB`). | -| **Allocated** | Scheduler decision | The CPU and memory the scheduler actually assigned to the task container. Intelligent Compute may assign values different from what was requested — it selects the most cost-effective instance shape that satisfies the task's requirements. | +| **Allocated** | Scheduler decision | The CPU and memory the scheduler assigned to the task container. Intelligent Compute may assign values different from what was requested. It selects the most cost-effective instance shape that satisfies the task's requirements. | | **Used** | Nextflow trace data | The CPU and memory the task actually consumed, measured from Nextflow's trace metrics (`pcpu` × `realtime` for CPU, `peakRss` for memory). Requires Fusion to be enabled. Absent for tasks that did not produce trace data. | **How to read the numbers:** - If **Requested** is much higher than **Allocated**, the scheduler found a more efficient instance shape than your directives implied. -- If **Allocated** is much higher than **Used**, the task ran with significant idle headroom. You may be able to reduce your process resource directives on future runs to lower cost. +- If **Allocated** is much higher than **Used**, the task ran with idle headroom. - If **Used** is close to **Allocated**, resource utilization is near-optimal for that task. ## Configuration options @@ -118,25 +124,27 @@ The **Metrics** tab for a workflow run on Intelligent Compute shows three resour ## Task and run statuses +Intelligent Compute reports a status for each task and for the run as a whole. Use these statuses to track progress and diagnose failures. + ### Task statuses | Status | Description | |--------|-------------| -| `SUBMITTED` | Task is queued or submitted to the compute backend. | -| `RUNNING` | Task is actively executing on a compute instance. | -| `SUCCEEDED` | Task completed with exit code 0. | -| `FAILED` | Task failed. This covers non-retriable execution failures (non-zero exit code, container startup errors) and spot quota exhaustion after retries are exhausted. | -| `CANCELLED` | Task was cancelled by the user. | -| `PREEMPTED` | The Spot instance running this task was reclaimed by AWS. The scheduler retries the task automatically. If the retry limit is reached, the task transitions to `FAILED`. | -| `UNSCHEDULABLE` | No instance type could satisfy the task's placement constraints. This occurs when the requested resources exceed what any available instance type can provide, or when specified instance types are unavailable in the region. Check your **Instance types** configuration and the `cpus` and `memory` directives in the failing process. | -| `UNKNOWN` | Task status could not be determined, typically due to a transient backend failure. | +| SUBMITTED | Task is queued or submitted to the compute backend. | +| RUNNING | Task is actively executing on a compute instance. | +| SUCCEEDED | Task completed with exit code 0. | +| FAILED | Task failed. This covers non-retriable execution failures (non-zero exit code, container startup errors) and spot quota exhaustion after retries are exhausted. | +| CANCELLED | Task was cancelled by the user. | +| PREEMPTED | The Spot instance running this task was reclaimed by AWS. The scheduler retries the task automatically. If the retry limit is reached, the task transitions to `FAILED`. | +| UNSCHEDULABLE | No instance type could satisfy the task's placement constraints. This occurs when the requested resources exceed what any available instance type can provide, or when specified instance types are unavailable in the region. Check your **Instance types** configuration and the `cpus` and `memory` directives in the failing process. | +| UNKNOWN | Task status could not be determined, typically due to a transient backend failure. | ### Run statuses | Status | Description | |--------|-------------| -| `ACTIVE` | The run is in progress. | -| `TERMINATING` | The run is shutting down — final tasks are completing or being cancelled. | -| `TERMINATED` | The run ended normally. | -| `FAILED` | The run failed. | -| `DANGLING` | The Nextflow process stopped sending heartbeats. This typically means the launcher process crashed or lost connectivity. Tasks already dispatched to ECS may still be running. Check CloudWatch logs under `/seqera/sched` for details. | +| ACTIVE | The run is in progress. | +| TERMINATING | The run is shutting down. Final tasks are completing or being cancelled. | +| TERMINATED | The run ended normally. | +| FAILED | The run failed. | +| DANGLING | The Nextflow process stopped sending heartbeats. This typically means the launcher process crashed or lost connectivity. Tasks already dispatched to ECS may still be running. Check CloudWatch logs under `/seqera/sched` for details. | From 4343d513ecb4784b6cea3c036f9a8e1097584701 Mon Sep 17 00:00:00 2001 From: MichaelTansiniSeqera Date: Wed, 24 Jun 2026 14:13:53 +0100 Subject: [PATCH 06/11] =?UTF-8?q?docs:=20fix=20PREEMPTED=20status=20descri?= =?UTF-8?q?ption=20=E2=80=94=20terminal=20state,=20not=20transition=20to?= =?UTF-8?q?=20FAILED?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PREEMPTED is itself terminal. The scheduler retries internally (Spot then On-Demand fallback for spotFirst); when all attempts are exhausted the task stays PREEMPTED, it does not transition to FAILED. Addresses review comment from jonmarti on PR #1565. Co-Authored-By: Claude Sonnet 4.6 --- platform-cloud/docs/compute-envs/intelligent-compute.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform-cloud/docs/compute-envs/intelligent-compute.mdx b/platform-cloud/docs/compute-envs/intelligent-compute.mdx index bef759639..f09317477 100644 --- a/platform-cloud/docs/compute-envs/intelligent-compute.mdx +++ b/platform-cloud/docs/compute-envs/intelligent-compute.mdx @@ -135,7 +135,7 @@ Intelligent Compute reports a status for each task and for the run as a whole. U | SUCCEEDED | Task completed with exit code 0. | | FAILED | Task failed. This covers non-retriable execution failures (non-zero exit code, container startup errors) and spot quota exhaustion after retries are exhausted. | | CANCELLED | Task was cancelled by the user. | -| PREEMPTED | The Spot instance running this task was reclaimed by AWS. The scheduler retries the task automatically. If the retry limit is reached, the task transitions to `FAILED`. | +| PREEMPTED | The Spot instance running this task was reclaimed by AWS. The scheduler retries the task automatically on Spot, then falls back to On-Demand for `spotFirst`. If all retries are exhausted, the task ends as `PREEMPTED`. | | UNSCHEDULABLE | No instance type could satisfy the task's placement constraints. This occurs when the requested resources exceed what any available instance type can provide, or when specified instance types are unavailable in the region. Check your **Instance types** configuration and the `cpus` and `memory` directives in the failing process. | | UNKNOWN | Task status could not be determined, typically due to a transient backend failure. | From 83abf350a610c5f099d2e90be39943fe3ee2637f Mon Sep 17 00:00:00 2001 From: Justine Geffen Date: Wed, 24 Jun 2026 22:41:51 +0200 Subject: [PATCH 07/11] docs: deslop intelligent-compute and google-cloud-batch pages Tightens prose on the AWS Intelligent Compute (preview) page and the GCP Workload Identity Federation step 6 in both Cloud and Enterprise copies of the Google Cloud Batch page. Cuts duplicated points and filler adverbs, splits loose connectors, swaps passive constructions for active, and normalizes a few status-table descriptions. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../docs/compute-envs/google-cloud-batch.md | 2 +- .../docs/compute-envs/intelligent-compute.mdx | 24 +++++++++---------- .../compute-envs/google-cloud-batch.md | 2 +- 3 files changed, 14 insertions(+), 14 deletions(-) diff --git a/platform-cloud/docs/compute-envs/google-cloud-batch.md b/platform-cloud/docs/compute-envs/google-cloud-batch.md index faaa30bc5..331fb13f5 100644 --- a/platform-cloud/docs/compute-envs/google-cloud-batch.md +++ b/platform-cloud/docs/compute-envs/google-cloud-batch.md @@ -120,7 +120,7 @@ Setting up WIF requires the following steps in the GCP Console: --role="roles/iam.serviceAccountTokenCreator" ``` - Replace `SA_EMAIL` with the service account email. Without this role, viewing or downloading file contents in Data Explorer fails with a signing error. Running pipelines is not affected. + Replace `SA_EMAIL` with the service account email. Without this role, viewing or downloading file contents in Data Explorer fails with a signing error. Pipeline runs are not affected. After setting up WIF in the GCP Console, you need the following information to create a credential in Seqera Platform: diff --git a/platform-cloud/docs/compute-envs/intelligent-compute.mdx b/platform-cloud/docs/compute-envs/intelligent-compute.mdx index f09317477..4661041db 100644 --- a/platform-cloud/docs/compute-envs/intelligent-compute.mdx +++ b/platform-cloud/docs/compute-envs/intelligent-compute.mdx @@ -16,14 +16,14 @@ Seqera Intelligent Compute is in preview. Seqera must enable it for your organiz ::: :::caution -Intelligent Compute may assign different CPU and memory values to tasks than those specified in your pipeline's `process` directives. The scheduler selects the most cost-effective instance that meets each task's resource request rather than provisioning exactly what the directive specifies. +Intelligent Compute may assign different CPU and memory values to tasks than your pipeline's `process` directives specify. The scheduler picks the most cost-effective instance shape that meets each task's resource request. ::: -Intelligent Compute is a scheduling service that runs Nextflow pipelines on a Seqera-managed Amazon ECS cluster. It allocates compute resources based on what each task needs rather than what the pipeline requests, which reduces cost and improves utilization across a run. Intelligent Compute is supported on AWS Cloud compute environments only. +Intelligent Compute is a scheduling service that runs Nextflow pipelines on a Seqera-managed Amazon ECS cluster. It allocates compute resources based on what each task needs rather than what the pipeline requests. This reduces cost and improves utilization across a run. Intelligent Compute is available only on AWS Cloud compute environments. Unlike the standard AWS Cloud compute environment, which runs a pipeline on a single EC2 instance with a local executor, Intelligent Compute provisions and manages multi-node clusters. Pipelines can scale beyond a single instance while keeping startup times short. -When Intelligent Compute is enabled on an AWS Cloud compute environment, Seqera provisions and manages the following resources in your AWS account on first use: +When you enable Intelligent Compute on an AWS Cloud compute environment, Seqera provisions and manages the following resources in your AWS account on first use: - An Amazon ECS cluster per compute environment configuration - ECS capacity providers (Managed Instances or Auto Scaling Groups) @@ -35,7 +35,7 @@ All managed resources use the `seqera-sched-` prefix. Seqera creates them on fir ## IAM permissions -In addition to the [standard AWS Cloud IAM permissions](./aws-cloud#required-platform-iam-permissions), Intelligent Compute requires an additional policy attached to the same IAM user or role that Seqera uses to access your AWS account. +Intelligent Compute requires an additional policy beyond the [standard AWS Cloud IAM permissions](./aws-cloud#required-platform-iam-permissions). Attach this policy to the same IAM user or role that Seqera uses to access your AWS account.
IAM policy JSON @@ -96,7 +96,7 @@ You need the following: 1. Configure the [Intelligent Compute options](#configuration-options) as needed. 1. Select **Add**. -Seqera validates credentials and configuration on save. On first use, it provisions the required IAM roles and ECS cluster in your account. This provisioning is automatic and requires no further steps. +Seqera validates credentials and configuration on save. On first use, it provisions the required IAM roles and ECS cluster in your account. ## Resource metrics @@ -106,7 +106,7 @@ The **Metrics** tab for a run on Intelligent Compute shows three resource values |--------|--------|-------------------| | **Requested** | Pipeline `process` directives | The CPU and memory your pipeline asked for, as written in your `process` directives (for example, `cpus = 4`, `memory = 8 GB`). | | **Allocated** | Scheduler decision | The CPU and memory the scheduler assigned to the task container. Intelligent Compute may assign values different from what was requested. It selects the most cost-effective instance shape that satisfies the task's requirements. | -| **Used** | Nextflow trace data | The CPU and memory the task actually consumed, measured from Nextflow's trace metrics (`pcpu` × `realtime` for CPU, `peakRss` for memory). Requires Fusion to be enabled. Absent for tasks that did not produce trace data. | +| **Used** | Nextflow trace data | The CPU and memory the task consumed, measured from the Nextflow trace metrics (`pcpu` × `realtime` for CPU, `peakRss` for memory). Requires Fusion to be enabled. Absent for tasks that did not produce trace data. | **How to read the numbers:** @@ -124,20 +124,20 @@ The **Metrics** tab for a run on Intelligent Compute shows three resource values ## Task and run statuses -Intelligent Compute reports a status for each task and for the run as a whole. Use these statuses to track progress and diagnose failures. +Intelligent Compute reports a status for each task and for the run as a whole. ### Task statuses | Status | Description | |--------|-------------| | SUBMITTED | Task is queued or submitted to the compute backend. | -| RUNNING | Task is actively executing on a compute instance. | +| RUNNING | Task is running on a compute instance. | | SUCCEEDED | Task completed with exit code 0. | -| FAILED | Task failed. This covers non-retriable execution failures (non-zero exit code, container startup errors) and spot quota exhaustion after retries are exhausted. | +| FAILED | Task failed. Covers non-retriable execution failures (non-zero exit code, container startup errors) and Spot capacity exhaustion after retries. | | CANCELLED | Task was cancelled by the user. | -| PREEMPTED | The Spot instance running this task was reclaimed by AWS. The scheduler retries the task automatically on Spot, then falls back to On-Demand for `spotFirst`. If all retries are exhausted, the task ends as `PREEMPTED`. | -| UNSCHEDULABLE | No instance type could satisfy the task's placement constraints. This occurs when the requested resources exceed what any available instance type can provide, or when specified instance types are unavailable in the region. Check your **Instance types** configuration and the `cpus` and `memory` directives in the failing process. | -| UNKNOWN | Task status could not be determined, typically due to a transient backend failure. | +| PREEMPTED | The Spot instance running this task was reclaimed by AWS. The scheduler retries the task on Spot, then falls back to On-Demand for `spotFirst`. If all retries are exhausted, the task ends as `PREEMPTED`. | +| UNSCHEDULABLE | No instance type satisfies the task's placement constraints. This occurs when the requested resources exceed what any available instance type can provide, or when specified instance types are unavailable in the region. Check your **Instance types** configuration and the `cpus` and `memory` directives in the failing process. | +| UNKNOWN | Task status cannot be determined, typically because of a transient backend failure. | ### Run statuses diff --git a/platform-enterprise_docs/compute-envs/google-cloud-batch.md b/platform-enterprise_docs/compute-envs/google-cloud-batch.md index f58311f98..3afbd76a6 100644 --- a/platform-enterprise_docs/compute-envs/google-cloud-batch.md +++ b/platform-enterprise_docs/compute-envs/google-cloud-batch.md @@ -115,7 +115,7 @@ Workload Identity Federation (WIF) is the recommended authentication method for --role="roles/iam.serviceAccountTokenCreator" ``` - Replace `SA_EMAIL` with the service account email. Without this role, viewing or downloading file contents in Data Explorer fails with a signing error. Running pipelines is not affected. + Replace `SA_EMAIL` with the service account email. Without this role, viewing or downloading file contents in Data Explorer fails with a signing error. Pipeline runs are not affected. WIF requires an OIDC signing key and for Seqera Platform's OIDC provider to be configured. See [Cryptographic options](https://docs.seqera.io/platform-enterprise/enterprise/configuration/overview#cryptographic-options). From 7a0552cc27fc10faa3c09acac9cb9d1295dd7470 Mon Sep 17 00:00:00 2001 From: Justine Geffen Date: Thu, 25 Jun 2026 14:51:01 +0200 Subject: [PATCH 08/11] Update intelligent-compute.mdx Signed-off-by: Justine Geffen --- .../docs/compute-envs/intelligent-compute.mdx | 29 +------------------ 1 file changed, 1 insertion(+), 28 deletions(-) diff --git a/platform-cloud/docs/compute-envs/intelligent-compute.mdx b/platform-cloud/docs/compute-envs/intelligent-compute.mdx index 4661041db..d4e3e7afc 100644 --- a/platform-cloud/docs/compute-envs/intelligent-compute.mdx +++ b/platform-cloud/docs/compute-envs/intelligent-compute.mdx @@ -106,7 +106,7 @@ The **Metrics** tab for a run on Intelligent Compute shows three resource values |--------|--------|-------------------| | **Requested** | Pipeline `process` directives | The CPU and memory your pipeline asked for, as written in your `process` directives (for example, `cpus = 4`, `memory = 8 GB`). | | **Allocated** | Scheduler decision | The CPU and memory the scheduler assigned to the task container. Intelligent Compute may assign values different from what was requested. It selects the most cost-effective instance shape that satisfies the task's requirements. | -| **Used** | Nextflow trace data | The CPU and memory the task consumed, measured from the Nextflow trace metrics (`pcpu` × `realtime` for CPU, `peakRss` for memory). Requires Fusion to be enabled. Absent for tasks that did not produce trace data. | +| **Used** | Nextflow trace data | The CPU and memory the task consumed, measured from the Nextflow trace metrics (`pcpu` × `realtime` for CPU, `peakRss` for memory). Absent for tasks that did not produce trace data. | **How to read the numbers:** @@ -121,30 +121,3 @@ The **Metrics** tab for a run on Intelligent Compute shows three resource values | **Seqera Intelligent Compute** | Enabled / Disabled | Disabled | Enables the Intelligent Compute scheduler for this compute environment. This option only appears if Intelligent Compute is enabled for your organization. | | **Provisioning model** | `spotFirst`, `spot`, `ondemand` | `spotFirst` | Instance procurement strategy. `spotFirst` uses Spot instances and falls back to On-Demand if Spot capacity is unavailable. `spot` uses Spot instances only. `ondemand` uses On-Demand instances only. | | **Instance types** | Comma-separated EC2 instance type identifiers (for example, `m5.xlarge, c5.2xlarge`) | Empty | Restricts which instance types the scheduler can select. When empty, the scheduler selects the most cost-effective type for each task automatically. Specifying types here overrides automatic selection. | - -## Task and run statuses - -Intelligent Compute reports a status for each task and for the run as a whole. - -### Task statuses - -| Status | Description | -|--------|-------------| -| SUBMITTED | Task is queued or submitted to the compute backend. | -| RUNNING | Task is running on a compute instance. | -| SUCCEEDED | Task completed with exit code 0. | -| FAILED | Task failed. Covers non-retriable execution failures (non-zero exit code, container startup errors) and Spot capacity exhaustion after retries. | -| CANCELLED | Task was cancelled by the user. | -| PREEMPTED | The Spot instance running this task was reclaimed by AWS. The scheduler retries the task on Spot, then falls back to On-Demand for `spotFirst`. If all retries are exhausted, the task ends as `PREEMPTED`. | -| UNSCHEDULABLE | No instance type satisfies the task's placement constraints. This occurs when the requested resources exceed what any available instance type can provide, or when specified instance types are unavailable in the region. Check your **Instance types** configuration and the `cpus` and `memory` directives in the failing process. | -| UNKNOWN | Task status cannot be determined, typically because of a transient backend failure. | - -### Run statuses - -| Status | Description | -|--------|-------------| -| ACTIVE | The run is in progress. | -| TERMINATING | The run is shutting down. Final tasks are completing or being cancelled. | -| TERMINATED | The run ended normally. | -| FAILED | The run failed. | -| DANGLING | The Nextflow process stopped sending heartbeats. This typically means the launcher process crashed or lost connectivity. Tasks already dispatched to ECS may still be running. Check CloudWatch logs under `/seqera/sched` for details. | From c5eb40fbe5f9dede2693008270e98012abb0f7e8 Mon Sep 17 00:00:00 2001 From: MichaelTansiniSeqera Date: Thu, 25 Jun 2026 14:26:28 +0100 Subject: [PATCH 09/11] docs(intelligent-compute): address Stefano's review comments and add config options MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Fix 'organization' → 'workspace' (3 occurrences) - Note ECS cluster teardown after setup step - Add 'Allocated matches Requested' bullet to resource metrics guidance - Add prediction model, backend strategy, Fusion snapshots, NVMe, and warm pool options to the configuration table Co-Authored-By: Claude Sonnet 4.6 --- .../docs/compute-envs/intelligent-compute.mdx | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/platform-cloud/docs/compute-envs/intelligent-compute.mdx b/platform-cloud/docs/compute-envs/intelligent-compute.mdx index d4e3e7afc..428140d14 100644 --- a/platform-cloud/docs/compute-envs/intelligent-compute.mdx +++ b/platform-cloud/docs/compute-envs/intelligent-compute.mdx @@ -12,7 +12,7 @@ import CodeBlock from '@theme/CodeBlock'; import AwsCloudIntelligentComputePolicy from './_policies/aws-cloud-intelligent-compute-policy.json?raw'; :::info[Preview] -Seqera Intelligent Compute is in preview. Seqera must enable it for your organization before you can use it. Contact your account manager to request access. +Seqera Intelligent Compute is in preview. Seqera must enable it for your workspace before you can use it. Contact your account manager to request access. ::: :::caution @@ -82,7 +82,7 @@ Intelligent Compute requires an additional policy beyond the [standard AWS Cloud You need the following: -- Intelligent Compute enabled for your organization by Seqera. Contact your account manager to request access. +- Intelligent Compute enabled for your workspace by Seqera. Contact your account manager to request access. - AWS credentials with both the standard AWS Cloud permissions and the Intelligent Compute permissions attached. ::: @@ -96,7 +96,7 @@ You need the following: 1. Configure the [Intelligent Compute options](#configuration-options) as needed. 1. Select **Add**. -Seqera validates credentials and configuration on save. On first use, it provisions the required IAM roles and ECS cluster in your account. +Seqera validates credentials and configuration on save. On first use, it provisions the required IAM roles and ECS cluster in your account. Clusters and associated resources are removed automatically when no longer needed. ## Resource metrics @@ -113,11 +113,19 @@ The **Metrics** tab for a run on Intelligent Compute shows three resource values - If **Requested** is much higher than **Allocated**, the scheduler found a more efficient instance shape than your directives implied. - If **Allocated** is much higher than **Used**, the task ran with idle headroom. - If **Used** is close to **Allocated**, resource utilization is near-optimal for that task. +- If **Allocated** matches **Requested**, confirm whether Seqera Intelligent Compute has been set up with an optimization model. ## Configuration options | Option | Values | Default | Description | |--------|--------|---------|-------------| -| **Seqera Intelligent Compute** | Enabled / Disabled | Disabled | Enables the Intelligent Compute scheduler for this compute environment. This option only appears if Intelligent Compute is enabled for your organization. | +| **Seqera Intelligent Compute** | Enabled / Disabled | Disabled | Enables the Intelligent Compute scheduler for this compute environment. This option only appears if Intelligent Compute is enabled for your workspace. | | **Provisioning model** | `spotFirst`, `spot`, `ondemand` | `spotFirst` | Instance procurement strategy. `spotFirst` uses Spot instances and falls back to On-Demand if Spot capacity is unavailable. `spot` uses Spot instances only. `ondemand` uses On-Demand instances only. | | **Instance types** | Comma-separated EC2 instance type identifiers (for example, `m5.xlarge, c5.2xlarge`) | Empty | Restricts which instance types the scheduler can select. When empty, the scheduler selects the most cost-effective type for each task automatically. Specifying types here overrides automatic selection. | +| **Prediction model** | `none`, `qr/v1`, `qr/v2` | `none` | Resource-prediction model used by Intelligent Compute to size tasks. When set to `none`, the scheduler does not apply resource optimization and allocated resources match what the pipeline requests. Contact your account manager to enable a prediction model. | +| **Backend strategy** | `ECS`, `EC2` | `ECS` | Task execution backend. `ECS` delegates execution to AWS ECS (default). `EC2` runs tasks directly on EC2 instances and enables warm pool support. | +| **Fusion snapshots** | Enabled / Disabled | Disabled | Allow interrupted tasks (for example, after a Spot reclaim) to resume from a Fusion snapshot instead of restarting from scratch. | +| **Use NVMe instance storage** | Enabled / Disabled | Disabled | Restrict Intelligent Compute to instance types that provide local SSD (NVMe) storage for faster I/O. | +| **Enable warm pool** | Enabled / Disabled | Disabled | Keep a pool of idle VMs ready to absorb incoming tasks with minimal start latency. Only available when **Backend strategy** is set to `EC2`. | +| **Desired warm VMs** | Integer | — | Target number of idle VMs to keep warm. Must be greater than zero. Only visible when **Enable warm pool** is enabled. | +| **Scale-to-zero timeout** | Seconds | — | Seconds of inactivity after which the warm pool scales to zero. Set to `0` to never scale to zero. Only visible when **Enable warm pool** is enabled. | From ff7fb4ecb7203ebae8bd46892de9caa3c7a58ca8 Mon Sep 17 00:00:00 2001 From: MichaelTansiniSeqera Date: Thu, 25 Jun 2026 14:27:51 +0100 Subject: [PATCH 10/11] docs(intelligent-compute): clarify per-workspace enablement and ECS-only backend Co-Authored-By: Claude Sonnet 4.6 --- platform-cloud/docs/compute-envs/intelligent-compute.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/platform-cloud/docs/compute-envs/intelligent-compute.mdx b/platform-cloud/docs/compute-envs/intelligent-compute.mdx index 428140d14..b65099576 100644 --- a/platform-cloud/docs/compute-envs/intelligent-compute.mdx +++ b/platform-cloud/docs/compute-envs/intelligent-compute.mdx @@ -12,7 +12,7 @@ import CodeBlock from '@theme/CodeBlock'; import AwsCloudIntelligentComputePolicy from './_policies/aws-cloud-intelligent-compute-policy.json?raw'; :::info[Preview] -Seqera Intelligent Compute is in preview. Seqera must enable it for your workspace before you can use it. Contact your account manager to request access. +Seqera Intelligent Compute is in preview. Seqera must enable it on a per-workspace basis before you can use it. Contact your account manager to request access for one or more workspaces. ::: :::caution @@ -123,7 +123,7 @@ The **Metrics** tab for a run on Intelligent Compute shows three resource values | **Provisioning model** | `spotFirst`, `spot`, `ondemand` | `spotFirst` | Instance procurement strategy. `spotFirst` uses Spot instances and falls back to On-Demand if Spot capacity is unavailable. `spot` uses Spot instances only. `ondemand` uses On-Demand instances only. | | **Instance types** | Comma-separated EC2 instance type identifiers (for example, `m5.xlarge, c5.2xlarge`) | Empty | Restricts which instance types the scheduler can select. When empty, the scheduler selects the most cost-effective type for each task automatically. Specifying types here overrides automatic selection. | | **Prediction model** | `none`, `qr/v1`, `qr/v2` | `none` | Resource-prediction model used by Intelligent Compute to size tasks. When set to `none`, the scheduler does not apply resource optimization and allocated resources match what the pipeline requests. Contact your account manager to enable a prediction model. | -| **Backend strategy** | `ECS`, `EC2` | `ECS` | Task execution backend. `ECS` delegates execution to AWS ECS (default). `EC2` runs tasks directly on EC2 instances and enables warm pool support. | +| **Backend strategy** | `ECS`, `EC2` | `ECS` | Task execution backend. Use `ECS` (default). `EC2` is reserved for future use. | | **Fusion snapshots** | Enabled / Disabled | Disabled | Allow interrupted tasks (for example, after a Spot reclaim) to resume from a Fusion snapshot instead of restarting from scratch. | | **Use NVMe instance storage** | Enabled / Disabled | Disabled | Restrict Intelligent Compute to instance types that provide local SSD (NVMe) storage for faster I/O. | | **Enable warm pool** | Enabled / Disabled | Disabled | Keep a pool of idle VMs ready to absorb incoming tasks with minimal start latency. Only available when **Backend strategy** is set to `EC2`. | From 4b9becced97eca0b08d5435ef8eb8ad0939dff50 Mon Sep 17 00:00:00 2001 From: MichaelTansiniSeqera Date: Thu, 25 Jun 2026 14:46:46 +0100 Subject: [PATCH 11/11] Update intelligent-compute.mdx Signed-off-by: MichaelTansiniSeqera --- .../docs/compute-envs/intelligent-compute.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/platform-cloud/docs/compute-envs/intelligent-compute.mdx b/platform-cloud/docs/compute-envs/intelligent-compute.mdx index b65099576..f9eb0f221 100644 --- a/platform-cloud/docs/compute-envs/intelligent-compute.mdx +++ b/platform-cloud/docs/compute-envs/intelligent-compute.mdx @@ -12,16 +12,16 @@ import CodeBlock from '@theme/CodeBlock'; import AwsCloudIntelligentComputePolicy from './_policies/aws-cloud-intelligent-compute-policy.json?raw'; :::info[Preview] -Seqera Intelligent Compute is in preview. Seqera must enable it on a per-workspace basis before you can use it. Contact your account manager to request access for one or more workspaces. +Seqera Intelligent Compute is in preview. Seqera must enable it on a per-workspace basis before you can use it. Contact your account manager to request access. ::: -:::caution +:::info Intelligent Compute may assign different CPU and memory values to tasks than your pipeline's `process` directives specify. The scheduler picks the most cost-effective instance shape that meets each task's resource request. ::: -Intelligent Compute is a scheduling service that runs Nextflow pipelines on a Seqera-managed Amazon ECS cluster. It allocates compute resources based on what each task needs rather than what the pipeline requests. This reduces cost and improves utilization across a run. Intelligent Compute is available only on AWS Cloud compute environments. +Intelligent Compute is a scheduling service that runs Nextflow pipelines on a Seqera-managed clusters. It allocates compute resources based on what each task needs rather than what the pipeline requests. This reduces cost and improves utilization across a run. Intelligent Compute is initially available only on AWS Cloud compute environments. -Unlike the standard AWS Cloud compute environment, which runs a pipeline on a single EC2 instance with a local executor, Intelligent Compute provisions and manages multi-node clusters. Pipelines can scale beyond a single instance while keeping startup times short. +Unlike the standard AWS Cloud compute environment, which runs a pipeline on a single EC2 instance, Intelligent Compute provisions and manages multi-node clusters. Pipelines can scale beyond a single instance while keeping startup times short. When you enable Intelligent Compute on an AWS Cloud compute environment, Seqera provisions and manages the following resources in your AWS account on first use: @@ -122,7 +122,7 @@ The **Metrics** tab for a run on Intelligent Compute shows three resource values | **Seqera Intelligent Compute** | Enabled / Disabled | Disabled | Enables the Intelligent Compute scheduler for this compute environment. This option only appears if Intelligent Compute is enabled for your workspace. | | **Provisioning model** | `spotFirst`, `spot`, `ondemand` | `spotFirst` | Instance procurement strategy. `spotFirst` uses Spot instances and falls back to On-Demand if Spot capacity is unavailable. `spot` uses Spot instances only. `ondemand` uses On-Demand instances only. | | **Instance types** | Comma-separated EC2 instance type identifiers (for example, `m5.xlarge, c5.2xlarge`) | Empty | Restricts which instance types the scheduler can select. When empty, the scheduler selects the most cost-effective type for each task automatically. Specifying types here overrides automatic selection. | -| **Prediction model** | `none`, `qr/v1`, `qr/v2` | `none` | Resource-prediction model used by Intelligent Compute to size tasks. When set to `none`, the scheduler does not apply resource optimization and allocated resources match what the pipeline requests. Contact your account manager to enable a prediction model. | +| **Prediction model** | `none`, `qr/v1`, `qr/v2` | `none` | For private preview use `qr/v2` | | **Backend strategy** | `ECS`, `EC2` | `ECS` | Task execution backend. Use `ECS` (default). `EC2` is reserved for future use. | | **Fusion snapshots** | Enabled / Disabled | Disabled | Allow interrupted tasks (for example, after a Spot reclaim) to resume from a Fusion snapshot instead of restarting from scratch. | | **Use NVMe instance storage** | Enabled / Disabled | Disabled | Restrict Intelligent Compute to instance types that provide local SSD (NVMe) storage for faster I/O. |