Skip to content

Comments

Multiple new stream adapter updates#6081

Open
jasnell wants to merge 4 commits intomainfrom
jasnell/more-new-streams-adapters-tweaks
Open

Multiple new stream adapter updates#6081
jasnell wants to merge 4 commits intomainfrom
jasnell/more-new-streams-adapters-tweaks

Conversation

@jasnell
Copy link
Collaborator

@jasnell jasnell commented Feb 14, 2026

A number of cleanups and more improvements.

For the write adapter, when writing to an internal stream (think fetch, identity transform stream, etc)... we will directly support writing ArrayBuffer, ArrayBufferView, SharedArrayBuffer like we do currently, but we also support writing strings directly (as utf8 bytes) which will avoid hving to create a TextEncoder. And we'll also support writing sync iterables, e.g.

const { writable, readble } = new IdentityTransformStream();

const writer = writable.getWriter();

await writer.write('hello');  // will work

await writer.write(['hello', 'there']);  // will work

function* gen() {
  yield 'hello';
  yield 'world';
}
await writer.write(gen());  // will work

Why you ask? Pretty simple.. writing multiple chunks at once will help avoid the more expensive KJ/JS promise loop.

@jasnell jasnell requested review from a team as code owners February 14, 2026 07:20
@jasnell jasnell changed the title ReadableSourceKjAdapter doc comment updates Multiple new stream adapter updates Feb 14, 2026
Support vectored writes of sync iterables of bytes sources.
@jasnell jasnell force-pushed the jasnell/more-new-streams-adapters-tweaks branch from 7f53d0a to 87a1c98 Compare February 14, 2026 21:52
// is in progress.
//
// The returned promise will never resolve with more than maxBytes.
// The returned promise will never resolve with more than maxBytes
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// The returned promise will never resolve with more than maxBytes
// The returned promise will never resolve with more than maxBytes.

KJ_IF_SOME(fn, val.tryCast<JsFunction>()) {
return Signature([fn = JsRef(js, fn), obj = JsRef(js, object)](
jsg::Lock& js, Optional<JsValue> arg) -> GeneratorNext<JsValue> {
return js.tryCatch([&]() -> GeneratorNext<JsValue> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and elsewhere, you could use JSG_TRY / JSG_CATCH for new code now, if you want.

// or ArrayBufferView, we will attempt to detach the underlying buffer
// before writing it to the sink. Detaching is required by the
// streams spec but our original implementation does not detach
// and it turns out there are old workers depending on that behavior.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to worry about old workers still? My understanding it's a moot point whether we detach or not, because we have to use a non-V8 heap buffer anyway, but if detaching versus not detaching is a script-visible side effect, could removing this break anyone?

while (true) {
KJ_IF_SOME(next, gen.next(js)) {
JSG_REQUIRE(
pieces.size() <= 64, TypeError, "Too many pieces yielded from the iterable.");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: would be good to note the current limit in the error message

// this and make sure we are properly handling those cases.
// TODO(later): We can possibly optimize this by getting the memory protection key and
// avoiding the copy.
return js.tryCatch([&]() -> jsg::Promise<void> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: could be JSG_TRY

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants