Skip to content

feat: add opt-in heuristic prompt guard to reduce wasted token usage#1696

Open
ModelingSolver wants to merge 4 commits into
anthropics:mainfrom
ModelingSolver:feat/token-saving-heuristic
Open

feat: add opt-in heuristic prompt guard to reduce wasted token usage#1696
ModelingSolver wants to merge 4 commits into
anthropics:mainfrom
ModelingSolver:feat/token-saving-heuristic

Conversation

@ModelingSolver

Copy link
Copy Markdown

Summary

Adds an opt-in heuristic filter that detects malformed or incoherent prompts before they're sent to the API, avoiding wasted token usage on requests likely to fail or return low-value responses.

https://github.com/ModelingSolver/HEY-GAFAM-READ-THIS

How it works

  • Gated behind the ANTHROPIC_ENABLE_HEURISTIC_GUARD environment variable — disabled by default, zero impact on existing behavior.
  • When enabled, analyzes the last user message using lightweight heuristics (token count, presence of action-oriented keywords, lexical complexity ratio).
  • Raises a ValueError with the computed coherence score when a message falls below the admissibility threshold, preventing the request from reaching the API.

Scope / Known limitations

⚠️ This heuristic is currently French-language only. Action verbs and format keywords are French-specific (e.g. "donne", "génère", "analyse"); prompts in other languages will not be recognized by the keyword matching and may score lower than intended.

  • Current heuristics (action verbs, format keywords) are tuned for French-language prompts.
  • This is an initial, opt-in implementation; multi-language keyword sets and more robust malformation signals are natural follow-ups.

Testing

  • Verified: guard disabled → request passes through unaffected
  • Verified: guard enabled + well-formed prompt → request passes through
  • Verified: guard enabled + malformed/incoherent prompt → request blocked with ValueError

Checklist

  • Opt-in only, no change to default client behavior
  • Manually tested all three paths above
  • Documented current language scope limitation

@ModelingSolver ModelingSolver requested a review from a team as a code owner June 21, 2026 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant