docs: Intelligent Compute page for AWS Cloud (preview)#1565
docs: Intelligent Compute page for AWS Cloud (preview)#1565MichaelTansiniSeqera wants to merge 14 commits into
Conversation
Adds a new standalone page covering the Seqera Intelligent Compute preview feature for AWS Cloud compute environments, including IAM permissions, setup steps, and configuration options. Updates the Cloud sidebar to include the new page after aws-cloud. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
✅ Deploy Preview for seqera-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Signed-off-by: MichaelTansiniSeqera <michael.tansini@seqera.io>
- Remove <details> collapse from IAM policy — policy is now always visible and copy-pasteable - Add Resource metrics section explaining Requested/Allocated/Used and how to interpret the gap between them - Add Task and run statuses reference table for troubleshooting Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rer on GCP When WIF credentials are used for Data Explorer, Platform has no embedded private key and must call the GCP IAM signBlob API to generate presigned URLs. roles/iam.serviceAccountTokenCreator on the SA itself is required for this to succeed. Without it, file viewing and download fail silently with a signing error. Running pipelines is unaffected. Updates both Cloud and Enterprise Google Cloud Batch docs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Deslop and tidy tables * Remove bold * Update intelligent-compute.mdx Signed-off-by: MichaelTansiniSeqera <michael.tansini@seqera.io> * [automated] Fix code formatting * chore: add 'intelligent compute' to doc-tags-allowed.txt --------- Signed-off-by: MichaelTansiniSeqera <michael.tansini@seqera.io> Co-authored-by: MichaelTansiniSeqera <michael.tansini@seqera.io> Co-authored-by: Seqera Docs Bot <info@seqera.io> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com>
|
run vale |
|
🔍 Running Vale on the changed documentation in this PR… |
|
run vale |
|
🔍 Running Vale on the changed documentation in this PR… |
|
run vale |
|
🔍 Running Vale on the changed documentation in this PR… |
|
run vale |
|
🔍 Running Vale on the changed documentation in this PR… |
|
@jonmarti can you re-review the updated PR please |
…on to FAILED PREEMPTED is itself terminal. The scheduler retries internally (Spot then On-Demand fallback for spotFirst); when all attempts are exhausted the task stays PREEMPTED, it does not transition to FAILED. Addresses review comment from jonmarti on PR #1565. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tightens prose on the AWS Intelligent Compute (preview) page and the GCP Workload Identity Federation step 6 in both Cloud and Enterprise copies of the Google Cloud Batch page. Cuts duplicated points and filler adverbs, splits loose connectors, swaps passive constructions for active, and normalizes a few status-table descriptions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
||
| You need the following: | ||
|
|
||
| - Intelligent Compute enabled for your organization by Seqera. Contact your account manager to request access. |
There was a problem hiding this comment.
It's enabled per workspace, not organization
| ## Task and run statuses | ||
|
|
||
| Intelligent Compute reports a status for each task and for the run as a whole. | ||
|
|
||
| ### Task statuses | ||
|
|
||
| | Status | Description | | ||
| |--------|-------------| | ||
| | <code style={{whiteSpace: 'nowrap'}}>SUBMITTED</code> | Task is queued or submitted to the compute backend. | | ||
| | <code style={{whiteSpace: 'nowrap'}}>RUNNING</code> | Task is running on a compute instance. | | ||
| | <code style={{whiteSpace: 'nowrap'}}>SUCCEEDED</code> | Task completed with exit code 0. | | ||
| | <code style={{whiteSpace: 'nowrap'}}>FAILED</code> | Task failed. Covers non-retriable execution failures (non-zero exit code, container startup errors) and Spot capacity exhaustion after retries. | | ||
| | <code style={{whiteSpace: 'nowrap'}}>CANCELLED</code> | Task was cancelled by the user. | | ||
| | <code style={{whiteSpace: 'nowrap'}}>PREEMPTED</code> | The Spot instance running this task was reclaimed by AWS. The scheduler retries the task on Spot, then falls back to On-Demand for `spotFirst`. If all retries are exhausted, the task ends as `PREEMPTED`. | | ||
| | <code style={{whiteSpace: 'nowrap'}}>UNSCHEDULABLE</code> | No instance type satisfies the task's placement constraints. This occurs when the requested resources exceed what any available instance type can provide, or when specified instance types are unavailable in the region. Check your **Instance types** configuration and the `cpus` and `memory` directives in the failing process. | | ||
| | <code style={{whiteSpace: 'nowrap'}}>UNKNOWN</code> | Task status cannot be determined, typically because of a transient backend failure. | | ||
|
|
||
| ### Run statuses | ||
|
|
||
| | Status | Description | | ||
| |--------|-------------| | ||
| | <code style={{whiteSpace: 'nowrap'}}>ACTIVE</code> | The run is in progress. | | ||
| | <code style={{whiteSpace: 'nowrap'}}>TERMINATING</code> | The run is shutting down. Final tasks are completing or being cancelled. | | ||
| | <code style={{whiteSpace: 'nowrap'}}>TERMINATED</code> | The run ended normally. | | ||
| | <code style={{whiteSpace: 'nowrap'}}>FAILED</code> | The run failed. | | ||
| | <code style={{whiteSpace: 'nowrap'}}>DANGLING</code> | The Nextflow process stopped sending heartbeats. This typically means the launcher process crashed or lost connectivity. Tasks already dispatched to ECS may still be running. Check CloudWatch logs under `/seqera/sched` for details. | |
There was a problem hiding this comment.
Aren't these IntelligentCompute internal statused that are not exposed to Cloud customers directly? I believe Cloud end user will only ever see the mapped statuses to the usual Platform statuses
There was a problem hiding this comment.
Yes, let's remove these as these are only retrievable via the Sched API
| **How to read the numbers:** | ||
|
|
||
| - If **Requested** is much higher than **Allocated**, the scheduler found a more efficient instance shape than your directives implied. | ||
| - If **Allocated** is much higher than **Used**, the task ran with idle headroom. |
There was a problem hiding this comment.
Should we document that if Allocated == Requested, is probably because no prediction model was configured?
There was a problem hiding this comment.
We can add a check
| |--------|--------|-------------------| | ||
| | **Requested** | Pipeline `process` directives | The CPU and memory your pipeline asked for, as written in your `process` directives (for example, `cpus = 4`, `memory = 8 GB`). | | ||
| | **Allocated** | Scheduler decision | The CPU and memory the scheduler assigned to the task container. Intelligent Compute may assign values different from what was requested. It selects the most cost-effective instance shape that satisfies the task's requirements. | | ||
| | **Used** | Nextflow trace data | The CPU and memory the task consumed, measured from the Nextflow trace metrics (`pcpu` × `realtime` for CPU, `peakRss` for memory). Requires Fusion to be enabled. Absent for tasks that did not produce trace data. | |
There was a problem hiding this comment.
Requires Fusion to be enabled.
Fusion is enabled by default, maybe this is about fusion-tracer that requires the NXF_TRACE_FUSION env var to be set to true? I'd have to check if it's actually needed or the executor sets it for you
There was a problem hiding this comment.
I think it's superfluous and we can just delete the sentence
| 1. Configure the [Intelligent Compute options](#configuration-options) as needed. | ||
| 1. Select **Add**. | ||
|
|
||
| Seqera validates credentials and configuration on save. On first use, it provisions the required IAM roles and ECS cluster in your account. |
There was a problem hiding this comment.
Should we mention that ECS cluster are also taken down after they are no longer needed?
There was a problem hiding this comment.
yes, will do
Summary
intelligent-compute.mdxpage underplatform-cloud/docs/compute-envs/provisioningModel,machineTypes)cloud-sidebar.jsonto insert the page afteraws-cloudStatus
WIP — pending editorial review and internal sign-off before publish.
Checklist
🤖 Generated with Claude Code