feat: add composite upload option for large file writes by mishushakov · Pull Request #1254 · e2b-dev/E2B

mishushakov · 2026-04-03T10:22:59Z

Summary

Adds automatic composite upload for large files (>64MB) in both JS and Python SDKs — write() transparently splits data into 64MB chunks, uploads them in parallel, then composes them server-side using zero-copy concatenation via the new POST /files/compose endpoint
Adds /files/compose endpoint to the envd OpenAPI spec with ComposeRequest schema (source_paths, destination, username) and regenerates JS SDK types
Files at or below 64MB use the normal single upload path — no user-facing API changes needed
Supports gzip compression for chunk uploads
Supports both sync and async Python SDKs

Test plan

Upload a file >64MB and verify it is chunked and composed correctly
Upload a file <64MB and verify normal upload path is used
Test with gzip: true on large file uploads
Test with both JS and Python SDKs (sync and async)

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

changeset-bot · 2026-04-03T10:23:05Z

🦋 Changeset detected

Latest commit: 603f317

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@e2b/python-sdk	Minor
e2b	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

cursor · 2026-04-03T10:23:06Z

PR Summary

Medium Risk
Adds a new envd API endpoint and changes SDK upload behavior for large files (parallel chunk uploads + server-side compose), which could impact reliability, timeouts, and disk usage for write operations.

Overview
Adds a new envd API POST /files/compose (with ComposeRequest) to concatenate multiple uploaded parts into a single destination file, and regenerates the JS OpenAPI types accordingly.

Updates the JS SDK Filesystem.write() and the async Python SDK Filesystem.write() to transparently switch to a composite upload for files larger than 64MB: split into 64MB temp chunks under /tmp, upload chunks in parallel (optionally gzip-compressed), then call /files/compose to finalize the write. Includes a changeset bump for @e2b/python-sdk and e2b.

^{Written by Cursor Bugbot for commit 603f317. This will update automatically on new commits. Configure here.}

github-actions · 2026-04-03T10:23:57Z

Package Artifacts

Built from 8c9603e. Download artifacts from this workflow run.

JS SDK (e2b@2.19.1-mishushakov-composite-upload.0):

npm install ./e2b-2.19.1-mishushakov-composite-upload.0.tgz

CLI (@e2b/cli@2.9.1-mishushakov-composite-upload.0):

npm install ./e2b-cli-2.9.1-mishushakov-composite-upload.0.tgz

Python SDK (e2b==2.20.0+mishushakov-composite-upload):

pip install ./e2b-2.20.0+mishushakov.composite.upload-py3-none-any.whl

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Autofix Details

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

packages/python-sdk/e2b/sandbox_async/filesystem/filesystem.py

packages/js-sdk/src/sandbox/filesystem/index.ts

- Build chunk_paths deterministically before asyncio.gather in async _composite_write - Use Username type instead of bare string in JS compositeWrite Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

packages/python-sdk/e2b/sandbox_sync/filesystem/filesystem.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

packages/python-sdk/e2b/sandbox_async/filesystem/filesystem.py

packages/js-sdk/src/sandbox/filesystem/index.ts

Use already-materialized blob/content instead of re-reading original data. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

When data fits in a single chunk, fall through to the normal write path instead of duplicating the upload logic inside compositeWrite. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Remove the `composite` option from `write()`. Files over 64MB are now automatically chunked and uploaded via the composite path when the envd version supports it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

packages/python-sdk/e2b/sandbox_sync/filesystem/filesystem.py

When data is an IO object and ≤64MB, to_upload_body() consumes the stream. Pass the materialized bytes to write_files() instead of the exhausted IO object. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9de5bc1fe8

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

packages/js-sdk/src/sandbox/filesystem/index.ts

packages/python-sdk/e2b/sandbox_async/filesystem/filesystem.py

packages/js-sdk/src/sandbox/filesystem/index.ts

packages/python-sdk/e2b/sandbox_sync/filesystem/filesystem.py

Composite upload's primary benefit is parallel chunk uploading, which the sync SDK cannot leverage (sequential HTTP requests negate the performance advantage). Only the async Python SDK and JS SDK retain composite upload support via asyncio.gather() and Promise.all(). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

write_files() already calls to_upload_body internally, so the pre-materialization in write() was unnecessary after removing the composite upload size check. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace asyncio.gather with asyncio.TaskGroup for structured concurrency, and offload gzip compression to a thread to avoid blocking the event loop. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

packages/js-sdk/src/sandbox/filesystem/index.ts

asyncio.TaskGroup requires Python 3.11+, which the SDK's type checker does not support. Revert to asyncio.gather for broader compatibility. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat: add composite upload option for large file writes

71742bd

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

cursor bot reviewed Apr 3, 2026

View reviewed changes

packages/python-sdk/e2b/sandbox_async/filesystem/filesystem.py Outdated Show resolved Hide resolved

packages/js-sdk/src/sandbox/filesystem/index.ts Outdated Show resolved Hide resolved

fix: address PR review comments

c733a78

- Build chunk_paths deterministically before asyncio.gather in async _composite_write - Use Username type instead of bare string in JS compositeWrite Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

cursor bot reviewed Apr 3, 2026

View reviewed changes

packages/python-sdk/e2b/sandbox_sync/filesystem/filesystem.py Outdated Show resolved Hide resolved

feat: support gzip in composite upload, increase chunk size to 64MB

fc52b20

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

cursor bot reviewed Apr 3, 2026

View reviewed changes

packages/python-sdk/e2b/sandbox_async/filesystem/filesystem.py Outdated Show resolved Hide resolved

packages/js-sdk/src/sandbox/filesystem/index.ts Outdated Show resolved Hide resolved

mishushakov and others added 5 commits April 3, 2026 13:01

fix: avoid double-consuming streams/IO in composite single-chunk path

174b9b8

Use already-materialized blob/content instead of re-reading original data. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: simplify composite upload for single-chunk files

a144791

When data fits in a single chunk, fall through to the normal write path instead of duplicating the upload logic inside compositeWrite. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: auto-detect large files for composite upload

f59ec5d

Remove the `composite` option from `write()`. Files over 64MB are now automatically chunked and uploaded via the composite path when the envd version supports it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

chore: reset index.ts to main

afd10e6

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fix: restore index.ts exports to match main branch

12ebc88

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

cursor bot reviewed Apr 3, 2026

View reviewed changes

packages/python-sdk/e2b/sandbox_sync/filesystem/filesystem.py Outdated Show resolved Hide resolved

fix: avoid consuming IO streams twice when data fits in single chunk

9de5bc1

When data is an IO object and ≤64MB, to_upload_body() consumes the stream. Pass the materialized bytes to write_files() instead of the exhausted IO object. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mishushakov marked this pull request as ready for review April 3, 2026 13:36

mishushakov requested review from ValentaTomas and jakubno as code owners April 3, 2026 13:36

chore: add changeset for composite upload

197bc3e

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

chatgpt-codex-connector bot reviewed Apr 3, 2026

View reviewed changes

packages/js-sdk/src/sandbox/filesystem/index.ts Show resolved Hide resolved

packages/js-sdk/src/sandbox/filesystem/index.ts Show resolved Hide resolved

claude bot reviewed Apr 3, 2026

View reviewed changes

packages/python-sdk/e2b/sandbox_async/filesystem/filesystem.py Show resolved Hide resolved

packages/js-sdk/src/sandbox/filesystem/index.ts Show resolved Hide resolved

cursor bot reviewed Apr 3, 2026

View reviewed changes

packages/python-sdk/e2b/sandbox_sync/filesystem/filesystem.py Outdated Show resolved Hide resolved

mishushakov and others added 3 commits April 3, 2026 15:59

Remove redundant to_upload_body pre-materialization in sync write()

0512658

write_files() already calls to_upload_body internally, so the pre-materialization in write() was unnecessary after removing the composite upload size check. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Use TaskGroup and async gzip in async composite upload

111dd1f

Replace asyncio.gather with asyncio.TaskGroup for structured concurrency, and offload gzip compression to a thread to avoid blocking the event loop. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

cursor bot reviewed Apr 3, 2026

View reviewed changes

packages/js-sdk/src/sandbox/filesystem/index.ts Show resolved Hide resolved

Revert TaskGroup to asyncio.gather for Python <3.11 compat

603f317

asyncio.TaskGroup requires Python 3.11+, which the SDK's type checker does not support. Revert to asyncio.gather for broader compatibility. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mishushakov assigned matthewlouisbrockman Apr 3, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add composite upload option for large file writes#1254

feat: add composite upload option for large file writes#1254
mishushakov wants to merge 14 commits intomainfrom
mishushakov/composite-upload

mishushakov commented Apr 3, 2026 •

edited

Loading

Uh oh!

changeset-bot bot commented Apr 3, 2026 •

edited

Loading

Uh oh!

cursor bot commented Apr 3, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 3, 2026 •

edited

Loading

Uh oh!

cursor bot left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mishushakov commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

changeset-bot bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

cursor bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

github-actions bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Package Artifacts

Uh oh!

cursor bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mishushakov commented Apr 3, 2026 •

edited

Loading

changeset-bot bot commented Apr 3, 2026 •

edited

Loading

cursor bot commented Apr 3, 2026 •

edited

Loading

github-actions bot commented Apr 3, 2026 •

edited

Loading

cursor bot left a comment •

edited

Loading