Commit b08e48f
committed
feat(streaming): add ttat (time-to-first-answering-token)
ttft fires on the first content delta of any kind, which for reasoning
models means the first reasoning chunk — arrives quickly even when the
user-perceived latency is much longer. ttat fires only on the first
user-visible answer token (text delta or tool-call arguments delta),
excluding reasoning chunks. For non-reasoning models the two are equal;
for gpt-5-class / o-series models they differ by the reasoning duration.
This pairs with ttft for "did the model start thinking quickly?" vs
"how long did the user wait for an answer?" — both are valuable signals
that mean different things on reasoning workloads.
Implementation: a third bookmark variable (``first_answer_at``) set
inside the same up-front event-type check, restricted to
ResponseTextDeltaEvent / ResponseFunctionCallArgumentsDeltaEvent.
Adds one new histogram (``agentex.llm.ttat``) — same labels and units
as ttft.1 parent da85d7b commit b08e48f
2 files changed
Lines changed: 25 additions & 2 deletions
File tree
- src/agentex/lib/core
- observability
- temporal/plugins/openai_agents/models
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
50 | 59 | | |
51 | 60 | | |
52 | 61 | | |
| |||
Lines changed: 16 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
653 | 653 | | |
654 | 654 | | |
655 | 655 | | |
656 | | - | |
657 | | - | |
| 656 | + | |
| 657 | + | |
658 | 658 | | |
659 | 659 | | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
660 | 663 | | |
661 | 664 | | |
| 665 | + | |
662 | 666 | | |
663 | 667 | | |
664 | 668 | | |
| |||
686 | 690 | | |
687 | 691 | | |
688 | 692 | | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
689 | 701 | | |
690 | 702 | | |
691 | 703 | | |
| |||
1027 | 1039 | | |
1028 | 1040 | | |
1029 | 1041 | | |
| 1042 | + | |
| 1043 | + | |
1030 | 1044 | | |
1031 | 1045 | | |
1032 | 1046 | | |
| |||
0 commit comments