You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Roadmap priority: B4 — described in the progress report as "the single biggest simulation-speed win for population-scale models."
Summary
Add a batched, tick-level inference primitive — an llm:chat-all-style call that fires every agent's request concurrently with one barrier per tick, instead of N sequential per-agent calls.
Why it matters
The dominant cost in a population-scale LLM model is wall-clock, not tokens. ask turtles [ let r llm:chat ... ] serializes: 100 turtles at 2s per call is over 3 minutes per tick. Firing them together and joining once per tick collapses that to roughly the slowest single call.
llm:chat-async already exists but retrieval is a blocking runresult, so the modeler has to hand-roll the two-pass pattern (fire all, then collect all) and gets no concurrency control.
What needs to be done
Decide the API shape. Options: a command that takes an agentset and a prompt reporter, a two-phase fire/collect pair, or a batched reporter returning a list aligned to the agentset.
Respect NetLogo's execution model — results must land back on the right agents, and the barrier must complete before the primitive returns.
Per-agent history must stay correct under concurrent commit (the historyLock / commitExchange path from 14f7c35 already handles this; verify it holds at batch scale).
Per-agent failure isolation — one failed call must not abort the batch.
Roadmap priority: B4 — described in the progress report as "the single biggest simulation-speed win for population-scale models."
Summary
Add a batched, tick-level inference primitive — an
llm:chat-all-style call that fires every agent's request concurrently with one barrier per tick, instead of N sequential per-agent calls.Why it matters
The dominant cost in a population-scale LLM model is wall-clock, not tokens.
ask turtles [ let r llm:chat ... ]serializes: 100 turtles at 2s per call is over 3 minutes per tick. Firing them together and joining once per tick collapses that to roughly the slowest single call.llm:chat-asyncalready exists but retrieval is a blockingrunresult, so the modeler has to hand-roll the two-pass pattern (fire all, then collect all) and gets no concurrency control.What needs to be done
historyLock/commitExchangepath from14f7c35already handles this; verify it holds at batch scale).Open questions
llm:chat-async— does this replace it, or sit alongside?Related