Model-aware runtime profiles: per-model-family system prompt and edit-format tuning

[MervinPraison]

Summary

praisonaiagents builds a single, model-agnostic system prompt and advertises a fixed coding tool-set to every model. Different model families (GPT, Claude, Gemini, and reasoning variants) differ materially in how they parse tool schemas, follow output/formatting instructions, and succeed with different file-edit formats. A single prompt + single tool shape is a quality ceiling for a multi-provider SDK. This proposes an opt-in, protocol-driven "runtime profile" registry that maps a model family to a system-prompt profile and a preferred edit-tool format, with today's behaviour as the default profile.

Current behaviour

• src/praisonai-agents/praisonaiagents/agent/chat_mixin.py — _build_system_prompt(tools) composes one prompt (role/goal/backstory/rules/memory) used verbatim for every model. There is no per-family variation.
• src/praisonai-agents/praisonaiagents/toolsets.py (the coding toolset, ~line 270) always advertises the same edit_file + apply_patch + write_file set regardless of the active model.
• src/praisonai-agents/praisonaiagents/llm/llm.py — the only model-specific branching is narrow (prompt-cache control for Anthropic, a Gemini prompt tweak). There is no first-class notion of a per-family prompt/tool profile.
• No prompt_profile / model_profile registry exists anywhere in agent/, llm/, or config/ (grep is empty).

Net effect: prompt wording and edit-tool selection cannot be tuned per model without editing core code, and the SDK cannot ship best-practice defaults per family.

Desired behaviour

• A lightweight registry maps a model family (resolved from the model id/provider) to a RuntimeProfile: an optional system-prompt template/segment overrides and a preferred edit-tool format (e.g. patch-oriented vs string-replace vs whole-file), plus room for family-specific knobs.
• _build_system_prompt and tool materialisation consult the resolved profile; when no profile matches, behaviour is identical to today (fully backward compatible).
• Profiles are overridable by users at all layers (see 3-way surface) and extendable by third parties via an entry-point group, consistent with the existing registry pattern.

Layer placement

• Primary layer: core (praisonaiagents)
• Why not core: n/a — this is core: it changes how the agent runtime assembles the system prompt and selects built-in edit tools, the heart of the run loop.
• Why not wrapper: the wrapper cannot influence per-turn prompt assembly or tool materialisation; it only passes config down. It should merely expose the setting.
• Why not tools: individual tools do not (and should not) decide which prompt or which edit format the runtime uses; that is a runtime-level decision spanning multiple tools.
• Why not plugins: lifecycle hooks fire around a run but do not own baseline system-prompt construction or the built-in edit-tool set; a policy plugin could tweak text but cannot cleanly switch the advertised edit-tool format per model.
• Secondary touch (optional): wrapper — surface agent.profile/--profile-model and a config key; docs.
• 3-way surface (CLI + YAML + Python): yes — Agent(runtime_profile=...) in Python, agent.profile: in .praisonai/config.yaml, and a --profile-style flag for praisonai run.

Proposed approach

1. Add praisonaiagents/runtime/profiles.py with a RuntimeProfileProtocol and a thread-safe RuntimeProfileRegistry (mirroring runtime/registry.py and tools/registry.py), including entry-point discovery (praisonaiagents.runtime_profiles).
2. Resolve the family from the model id/provider (reuse existing model/provider resolution) and look up a profile; fall back to a default profile that reproduces current output exactly.
3. Have _build_system_prompt apply profile prompt overrides (segment-level, so caching and rules/memory injection are preserved) and have tool materialisation honour the profile's preferred edit-tool format.
4. Ship a small set of sensible built-in profiles; keep them data-driven so tuning does not require code changes.

Resolution sketch

• runtime/profiles.py: RuntimeProfileProtocol, RuntimeProfile dataclass (system_prompt_overrides, preferred_edit_format, extras), register_profile/resolve_profile/list_profiles.
• agent/chat_mixin.py::_build_system_prompt: after composing the base prompt, apply resolve_profile(model).apply(...); unchanged when profile is default.
• Tool materialisation / toolsets.py: choose edit tool by preferred_edit_format.
• Agent.__init__: accept runtime_profile (bool/str/dict/Profile), consistent with the existing config-object convention.

Severity

Medium — no correctness regression today, but it caps output quality/reliability across providers and forces core edits to tune per-model behaviour, which matters for a production, multi-provider SDK.

Validation

• Traced: single _build_system_prompt in agent/chat_mixin.py; fixed coding toolset in toolsets.py; only cache/Gemini branching in llm/llm.py; no profile registry present.
• Real agentic test: agent.start("<real coding task>") on two different model families should show the family-appropriate prompt profile and edit format being selected (and identical-to-today output when the default profile is active).
• Backward compatibility: with no profile configured, generated prompts and advertised tools are byte-for-byte unchanged.

[praisonai-triage-agent]

Claude finished @MervinPraison's task —— View job • claude/issue-2863-20260710-0854 • [Create PR ➔](https://github.com/MervinPraison/PraisonAI/compare/main...claude/issue-2863-20260710-0854?quick_pull=1&title=Issue%20%232863%3A%20Changes%20from%20Claude&body=This%20PR%20addresses%20issue%20%232863%0A%0AGenerated%20with%20%5BClaude%20Code%5D(https%3A%2F%2Fclaude.ai%2Fcode)

---

Done: Model-aware runtime profiles

PR: #2867

Architecture routing: CORE (praisonaiagents) — protocol + thread-safe registry + data-driven profiles only. No heavy implementation, only +4 lines to agent.py.

• Read AGENTS.md and analyze existing code (chat_mixin.py, toolsets.py, llm.py, runtime/registry.py)
• Create runtime/profiles.py — RuntimeProfileProtocol, RuntimeProfile dataclass, thread-safe RuntimeProfileRegistry, register_profile/resolve_profile/list_profiles/resolve_model_family, entry-point discovery (praisonaiagents.runtime_profiles)
• Export from runtime/__init__.py (lazy imports)
• Wire resolve_profile into _build_system_prompt (applied at the end; pure no-op unless a profile declares prompt overrides)
• Add runtime_profile param to Agent.__init__ (bool/str/dict/RuntimeProfile)
• Write tests — tests/unit/runtime/test_profiles.py (22 tests, all pass)
• Run test suite (new tests pass; pre-existing failures in test_resolve/test_resolver/test_turn_context verified as unrelated to this change)
• Create PR

What it does

A registry maps a resolved model family to a RuntimeProfile (system_prompt_prefix/suffix, preferred_edit_format, extras). Built-in profiles: default, anthropic (string-replace), openai (patch), gemini (whole-file).

Backward compatibility

• default profile → apply_system_prompt is a pure no-op.
• Built-in family profiles carry only preferred_edit_format and no prompt text, so generated prompts are byte-for-byte identical to today.
• Profile application is wrapped in try/except (debug-logged) so the run loop is never affected.

3-way surface: Agent(runtime_profile=...) is now supported in Python; wrapper CLI/YAML surfacing can be added later as a thin pass-through.
• Branch