Problem
Provider and model are global to the extension. LLMExtension.scala:51 holds one configStore and :54 holds one currentProvider, both shared by every agent. There is no way for two agents in the same run to use different models.
This rules out a natural class of experiment: comparing model capability within a single simulation — mixed-capability populations, a strong model against a weak one under identical conditions, or a cheap model for routine decisions and an expensive one for hard ones.
mesa-llm supports this by construction: llm_model is a constructor argument on the agent, so each agent owns its own client.
Why the obvious workaround does not work
Swapping config files per agent looks plausible but is not viable:
ask turtles [
llm:load-config "gpt.txt" ;; replaces the GLOBAL config
let r llm:chat "..."
]
llm:load-config replaces global state rather than creating a second configuration. Consequences:
- Every agent mutates the same config, so ordering determines behaviour
ensureProvider (LLMExtension.scala:135-147) caches a single provider instance, so each swap forces a provider rebuild
- It breaks entirely with
llm:chat-async — an in-flight request would resolve against whichever config the next agent installed, silently attributing a response to the wrong model
Possible directions
Per-agent model override. A primitive such as llm:set-agent-model "gpt-4o-mini", with the override stored per agent in the same WeakHashMap[Agent, _] pattern already used for conversation history. Requires replacing the single currentProvider with a small cache keyed by (provider, model) so instances are reused rather than rebuilt per call. This is the smaller change and fits the existing architecture.
Named provider profiles. Load several configurations under names and select one per call — llm:use-profile "fast". More general, and would also cover lab setups where different groups have different keys or endpoints.
Open questions
- Should an agent-level override also cover provider, or only model within the current provider? Cross-provider is more useful and more work, since readiness checks and API keys differ.
- How does an override interact with
llm:load-config, which currently clears global config wholesale?
- Should
llm:active report the calling agent's effective model rather than the global one?
- Provider instances are currently created once and reused; a cache needs a bound so a model swept over many values does not accumulate instances.
Related
- Surfaced by the mesa-llm comparison (29 Jul 2026), where per-agent model selection is one of the few capability differences with no NetLogo equivalent.
- The provider-sensitivity demo currently approximates this by running each provider's chain as a separate configuration.
Problem
Provider and model are global to the extension.
LLMExtension.scala:51holds oneconfigStoreand:54holds onecurrentProvider, both shared by every agent. There is no way for two agents in the same run to use different models.This rules out a natural class of experiment: comparing model capability within a single simulation — mixed-capability populations, a strong model against a weak one under identical conditions, or a cheap model for routine decisions and an expensive one for hard ones.
mesa-llm supports this by construction:
llm_modelis a constructor argument on the agent, so each agent owns its own client.Why the obvious workaround does not work
Swapping config files per agent looks plausible but is not viable:
llm:load-configreplaces global state rather than creating a second configuration. Consequences:ensureProvider(LLMExtension.scala:135-147) caches a single provider instance, so each swap forces a provider rebuildllm:chat-async— an in-flight request would resolve against whichever config the next agent installed, silently attributing a response to the wrong modelPossible directions
Per-agent model override. A primitive such as
llm:set-agent-model "gpt-4o-mini", with the override stored per agent in the sameWeakHashMap[Agent, _]pattern already used for conversation history. Requires replacing the singlecurrentProviderwith a small cache keyed by (provider, model) so instances are reused rather than rebuilt per call. This is the smaller change and fits the existing architecture.Named provider profiles. Load several configurations under names and select one per call —
llm:use-profile "fast". More general, and would also cover lab setups where different groups have different keys or endpoints.Open questions
llm:load-config, which currently clears global config wholesale?llm:activereport the calling agent's effective model rather than the global one?Related