Skip to content

Commit 006ccb5

Browse files
Tomkessclaude
andcommitted
test(gooddata-eval): assert exact MAQL string appears in simulated-user prompt
Addresses CodeRabbit review comment on #1718: the regression test only checked for generic instruction words ("verbatim", "every clause"), not that expected_output["maql"] itself made it into the prompt -- a regression that stripped the metric/label reference or filter value entirely could still pass. Assert the exact MAQL string is present. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent f782d24 commit 006ccb5

1 file changed

Lines changed: 1 addition & 0 deletions

File tree

packages/gooddata-eval/tests/test_agentic_metric_skill.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ def test_generate_simulated_response_prompt_preserves_maql_fidelity(monkeypatch)
4949
call_kwargs = mock_client.chat.completions.create.call_args.kwargs
5050
sent_prompt = call_kwargs["messages"][0]["content"]
5151

52+
assert expected_output["maql"] in sent_prompt
5253
assert "verbatim" in sent_prompt
5354
assert "every clause" in sent_prompt
5455
assert "WHERE" in sent_prompt or "filter" in sent_prompt.lower()

0 commit comments

Comments
 (0)