Commit 18d3def
committed
fix(gooddata-eval): fix create/execute pairing, score names, dataset default
- _extract_kda_calls picked the last create and last execute call
independently, not as a pair. create_1 -> execute_1(success) ->
create_2 (never executed) would wrongly report create_2's args as
executed/succeeded using execute_1's stale result. Fixed by clearing
execute_result whenever a new create call is seen.
- Score names executed/success/turn_completed now carry the kda_
prefix (kda_executed/kda_success/kda_turn_completed), matching
kda_triggered's existing convention -- gdc-nas's combo_report.py
verdict() must read the same names, updated in the same change.
- dataset_name default corrected from "kda_skill" to "agent_kda_skill",
matching the actual Langfuse dataset name used everywhere else.
JIRA: QA-28800
risk: nonprod1 parent 8557c38 commit 18d3def
2 files changed
Lines changed: 30 additions & 9 deletions
File tree
- packages/gooddata-eval
- src/gooddata_eval/core/agentic
- tests
Lines changed: 12 additions & 9 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
126 | 128 | | |
127 | 129 | | |
128 | 130 | | |
129 | 131 | | |
130 | 132 | | |
131 | 133 | | |
| 134 | + | |
132 | 135 | | |
133 | 136 | | |
134 | 137 | | |
| |||
430 | 433 | | |
431 | 434 | | |
432 | 435 | | |
433 | | - | |
| 436 | + | |
434 | 437 | | |
435 | 438 | | |
436 | 439 | | |
| |||
502 | 505 | | |
503 | 506 | | |
504 | 507 | | |
505 | | - | |
506 | | - | |
507 | | - | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
508 | 511 | | |
509 | 512 | | |
510 | 513 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
154 | 154 | | |
155 | 155 | | |
156 | 156 | | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
157 | 175 | | |
158 | 176 | | |
159 | 177 | | |
| |||
0 commit comments