Skip to content

feat: per-agent model selection — allow different agents to use different models in one run #68

Description

@JNK234

Problem

Provider and model are global to the extension. LLMExtension.scala:51 holds one configStore and :54 holds one currentProvider, both shared by every agent. There is no way for two agents in the same run to use different models.

This rules out a natural class of experiment: comparing model capability within a single simulation — mixed-capability populations, a strong model against a weak one under identical conditions, or a cheap model for routine decisions and an expensive one for hard ones.

mesa-llm supports this by construction: llm_model is a constructor argument on the agent, so each agent owns its own client.

Why the obvious workaround does not work

Swapping config files per agent looks plausible but is not viable:

ask turtles [
  llm:load-config "gpt.txt"      ;; replaces the GLOBAL config
  let r llm:chat "..."
]

llm:load-config replaces global state rather than creating a second configuration. Consequences:

  • Every agent mutates the same config, so ordering determines behaviour
  • ensureProvider (LLMExtension.scala:135-147) caches a single provider instance, so each swap forces a provider rebuild
  • It breaks entirely with llm:chat-async — an in-flight request would resolve against whichever config the next agent installed, silently attributing a response to the wrong model

Possible directions

Per-agent model override. A primitive such as llm:set-agent-model "gpt-4o-mini", with the override stored per agent in the same WeakHashMap[Agent, _] pattern already used for conversation history. Requires replacing the single currentProvider with a small cache keyed by (provider, model) so instances are reused rather than rebuilt per call. This is the smaller change and fits the existing architecture.

Named provider profiles. Load several configurations under names and select one per call — llm:use-profile "fast". More general, and would also cover lab setups where different groups have different keys or endpoints.

Open questions

  • Should an agent-level override also cover provider, or only model within the current provider? Cross-provider is more useful and more work, since readiness checks and API keys differ.
  • How does an override interact with llm:load-config, which currently clears global config wholesale?
  • Should llm:active report the calling agent's effective model rather than the global one?
  • Provider instances are currently created once and reused; a cache needs a bound so a model swept over many values does not accumulate instances.

Related

  • Surfaced by the mesa-llm comparison (29 Jul 2026), where per-agent model selection is one of the few capability differences with no NetLogo equivalent.
  • The provider-sensitivity demo currently approximates this by running each provider's chain as a separate configuration.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions