Skip to content

docs: Intelligent Compute page for AWS Cloud (preview)#1565

Open
MichaelTansiniSeqera wants to merge 14 commits into
masterfrom
docs/intelligent-compute-aws-preview
Open

docs: Intelligent Compute page for AWS Cloud (preview)#1565
MichaelTansiniSeqera wants to merge 14 commits into
masterfrom
docs/intelligent-compute-aws-preview

Conversation

@MichaelTansiniSeqera

Copy link
Copy Markdown
Contributor

Summary

  • Adds a new standalone intelligent-compute.mdx page under platform-cloud/docs/compute-envs/
  • Covers: preview admonition, what IC is, IAM permissions (with policy JSON), AWS Cloud CE setup steps, and configuration options (provisioningModel, machineTypes)
  • Updates cloud-sidebar.json to insert the page after aws-cloud

Status

WIP — pending editorial review and internal sign-off before publish.

Checklist

  • Editorial review pass
  • Confirm IAM policy JSON is up to date with latest sched release
  • Confirm sidebar placement is correct
  • Add screenshot(s) once UI is final

🤖 Generated with Claude Code

Adds a new standalone page covering the Seqera Intelligent Compute
preview feature for AWS Cloud compute environments, including IAM
permissions, setup steps, and configuration options. Updates the
Cloud sidebar to include the new page after aws-cloud.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@netlify

netlify Bot commented Jun 12, 2026

Copy link
Copy Markdown

Deploy Preview for seqera-docs ready!

Name Link
🔨 Latest commit 9e84c4d
🔍 Latest deploy log https://app.netlify.com/projects/seqera-docs/deploys/6a3c41245216750009fd4796
😎 Deploy Preview https://deploy-preview-1565--seqera-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

MichaelTansiniSeqera and others added 3 commits June 12, 2026 16:32
Signed-off-by: MichaelTansiniSeqera <michael.tansini@seqera.io>
- Remove <details> collapse from IAM policy — policy is now always visible and copy-pasteable
- Add Resource metrics section explaining Requested/Allocated/Used and how to interpret the gap between them
- Add Task and run statuses reference table for troubleshooting

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rer on GCP

When WIF credentials are used for Data Explorer, Platform has no embedded
private key and must call the GCP IAM signBlob API to generate presigned
URLs. roles/iam.serviceAccountTokenCreator on the SA itself is required
for this to succeed. Without it, file viewing and download fail silently
with a signing error. Running pipelines is unaffected.

Updates both Cloud and Enterprise Google Cloud Batch docs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread platform-cloud/docs/compute-envs/intelligent-compute.mdx Outdated
Comment thread platform-cloud/docs/compute-envs/intelligent-compute.mdx Outdated
Comment thread platform-cloud/docs/compute-envs/intelligent-compute.mdx Outdated
christopher-hakkaart and others added 2 commits June 16, 2026 16:05
* Deslop and tidy tables

* Remove bold

* Update intelligent-compute.mdx

Signed-off-by: MichaelTansiniSeqera <michael.tansini@seqera.io>

* [automated] Fix code formatting

* chore: add 'intelligent compute' to doc-tags-allowed.txt

---------

Signed-off-by: MichaelTansiniSeqera <michael.tansini@seqera.io>
Co-authored-by: MichaelTansiniSeqera <michael.tansini@seqera.io>
Co-authored-by: Seqera Docs Bot <info@seqera.io>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com>
@justinegeffen

Copy link
Copy Markdown
Contributor

run vale

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Running Vale on the changed documentation in this PR…

@justinegeffen

Copy link
Copy Markdown
Contributor

run vale

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Running Vale on the changed documentation in this PR…

@christopher-hakkaart

Copy link
Copy Markdown
Member

run vale

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Running Vale on the changed documentation in this PR…

@justinegeffen

Copy link
Copy Markdown
Contributor

run vale

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Running Vale on the changed documentation in this PR…

@MichaelTansiniSeqera

Copy link
Copy Markdown
Contributor Author

@jonmarti can you re-review the updated PR please

@MichaelTansiniSeqera MichaelTansiniSeqera marked this pull request as ready for review June 22, 2026 10:23
justinegeffen and others added 2 commits June 22, 2026 16:31
…on to FAILED

PREEMPTED is itself terminal. The scheduler retries internally (Spot
then On-Demand fallback for spotFirst); when all attempts are exhausted
the task stays PREEMPTED, it does not transition to FAILED.

Addresses review comment from jonmarti on PR #1565.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@justinegeffen justinegeffen requested a review from jonmarti June 24, 2026 20:41
Tightens prose on the AWS Intelligent Compute (preview) page and the
GCP Workload Identity Federation step 6 in both Cloud and Enterprise
copies of the Google Cloud Batch page. Cuts duplicated points and
filler adverbs, splits loose connectors, swaps passive constructions
for active, and normalizes a few status-table descriptions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@justinegeffen justinegeffen added 1. Editor review Needs a language review 1. Dev/PM/SME Needs a review by a Dev/PM/SME labels Jun 24, 2026

You need the following:

- Intelligent Compute enabled for your organization by Seqera. Contact your account manager to request access.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's enabled per workspace, not organization

Comment on lines +125 to +150
## Task and run statuses

Intelligent Compute reports a status for each task and for the run as a whole.

### Task statuses

| Status | Description |
|--------|-------------|
| <code style={{whiteSpace: 'nowrap'}}>SUBMITTED</code> | Task is queued or submitted to the compute backend. |
| <code style={{whiteSpace: 'nowrap'}}>RUNNING</code> | Task is running on a compute instance. |
| <code style={{whiteSpace: 'nowrap'}}>SUCCEEDED</code> | Task completed with exit code 0. |
| <code style={{whiteSpace: 'nowrap'}}>FAILED</code> | Task failed. Covers non-retriable execution failures (non-zero exit code, container startup errors) and Spot capacity exhaustion after retries. |
| <code style={{whiteSpace: 'nowrap'}}>CANCELLED</code> | Task was cancelled by the user. |
| <code style={{whiteSpace: 'nowrap'}}>PREEMPTED</code> | The Spot instance running this task was reclaimed by AWS. The scheduler retries the task on Spot, then falls back to On-Demand for `spotFirst`. If all retries are exhausted, the task ends as `PREEMPTED`. |
| <code style={{whiteSpace: 'nowrap'}}>UNSCHEDULABLE</code> | No instance type satisfies the task's placement constraints. This occurs when the requested resources exceed what any available instance type can provide, or when specified instance types are unavailable in the region. Check your **Instance types** configuration and the `cpus` and `memory` directives in the failing process. |
| <code style={{whiteSpace: 'nowrap'}}>UNKNOWN</code> | Task status cannot be determined, typically because of a transient backend failure. |

### Run statuses

| Status | Description |
|--------|-------------|
| <code style={{whiteSpace: 'nowrap'}}>ACTIVE</code> | The run is in progress. |
| <code style={{whiteSpace: 'nowrap'}}>TERMINATING</code> | The run is shutting down. Final tasks are completing or being cancelled. |
| <code style={{whiteSpace: 'nowrap'}}>TERMINATED</code> | The run ended normally. |
| <code style={{whiteSpace: 'nowrap'}}>FAILED</code> | The run failed. |
| <code style={{whiteSpace: 'nowrap'}}>DANGLING</code> | The Nextflow process stopped sending heartbeats. This typically means the launcher process crashed or lost connectivity. Tasks already dispatched to ECS may still be running. Check CloudWatch logs under `/seqera/sched` for details. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't these IntelligentCompute internal statused that are not exposed to Cloud customers directly? I believe Cloud end user will only ever see the mapped statuses to the usual Platform statuses

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, let's remove these as these are only retrievable via the Sched API

**How to read the numbers:**

- If **Requested** is much higher than **Allocated**, the scheduler found a more efficient instance shape than your directives implied.
- If **Allocated** is much higher than **Used**, the task ran with idle headroom.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we document that if Allocated == Requested, is probably because no prediction model was configured?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add a check

|--------|--------|-------------------|
| **Requested** | Pipeline `process` directives | The CPU and memory your pipeline asked for, as written in your `process` directives (for example, `cpus = 4`, `memory = 8 GB`). |
| **Allocated** | Scheduler decision | The CPU and memory the scheduler assigned to the task container. Intelligent Compute may assign values different from what was requested. It selects the most cost-effective instance shape that satisfies the task's requirements. |
| **Used** | Nextflow trace data | The CPU and memory the task consumed, measured from the Nextflow trace metrics (`pcpu` × `realtime` for CPU, `peakRss` for memory). Requires Fusion to be enabled. Absent for tasks that did not produce trace data. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requires Fusion to be enabled.

Fusion is enabled by default, maybe this is about fusion-tracer that requires the NXF_TRACE_FUSION env var to be set to true? I'd have to check if it's actually needed or the executor sets it for you

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's superfluous and we can just delete the sentence

1. Configure the [Intelligent Compute options](#configuration-options) as needed.
1. Select **Add**.

Seqera validates credentials and configuration on save. On first use, it provisions the required IAM roles and ECS cluster in your account.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mention that ECS cluster are also taken down after they are no longer needed?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, will do

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1. Dev/PM/SME Needs a review by a Dev/PM/SME 1. Editor review Needs a language review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants