Skip to content

feat: streaming responses #59

Description

@JNK234

Roadmap priority: B13

Summary

Stream partial responses as they generate, rather than waiting for the complete reply.

Why it matters

Long generations currently block with no feedback — the NetLogo UI freezes for the full duration of the call. Streaming gives progressive output, which matters for demos with visible agent "speech" and for any single call producing a long passage.

Currently all providers explicitly disable it (e.g. OllamaProvider.scala:50 sends "stream": false).

What needs to be done

  1. Decide whether this is even a good fit for NetLogo's execution model. A reporter must return a value; streaming implies incremental delivery, which has no natural NetLogo idiom.
  2. Plausible shapes: a callback command run per chunk, a mutable buffer the modeler polls, or streaming only into a display widget while the reporter still returns the final string.
  3. SSE parsing per provider — all four use different chunk formats.
  4. Interaction with historyLock — partial content must not be committed to history until complete.
  5. Interaction with thinking models, where reasoning and answer stream as distinct block types.

Open questions

  • Is there a real modeling use case, or is this mainly UI polish? Worth deciding before building, since the NetLogo-side design is the hard part rather than the HTTP side.
  • If the only benefit is a responsive UI during long calls, a progress indicator may deliver most of the value for far less work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions