[MOD-6057] Improve and elaborate `FT.PROFILE` documentation #2532

meiravgri · 2025-12-11T15:06:14Z

MOD-12263 vector iterator details: Added detailed notes on vector iterator profiles, including vector search modes and batch execution statistics.
MOD-6056 rename “counter” and “size”

MOD-12816 update ResultProcessor GILTime

MOD-12414 Add internal cursor reads to profile documentation

MOD-12321 Add shard unique ID

Added detailed notes on vector iterator profiles, including vector search modes and batch execution statistics.

CLAassistant · 2025-12-11T15:06:23Z

All committers have signed the CLA.

github-actions · 2025-12-11T15:06:32Z

Staging links:
https://redis.io/docs/staging/meiravgri-ft-profile-docs/commands/ft.profile/

dwdougherty · 2025-12-11T15:25:20Z

Hi @meiravgri. Please sign the CLA. I'll review today.

should we remove vector filter?

content/commands/ft.profile.md

alonre24

Nice!

alonre24 · 2025-12-14T11:37:25Z

content/commands/ft.profile.md

 | `Total`&nbsp;`profile`&nbsp;`time`     | The total run time (ms) of the query. Normally just a few ms. |
 | `Parsing`&nbsp;`time`           | The time (ms) spent parsing the query and its parameters into a query plan. Normally just a few ms. |
 | `Pipeline`&nbsp;`creation`&nbsp;`time` | The creation time (ms) of the execution plan, including iterators, result processors, and reducers creation. Normally just a few ms for `FT.SEARCH` queries, but expect a larger number for `FT.AGGREGATE` queries. |
+| `Total`&nbsp;`GIL`&nbsp;`time`     | The total time (ms) the query held the Global Interpreter Lock (GIL) during execution. Relevant for multi-threaded deployments where queries run in background threads. |


@lerman25, Evntually, do we expect this field to exist in case of non multithreaded deployment? If so, what should be the value?

No, if we don't have workers - this field doesn't print (starting from RediSearch/RediSearch#7756)

alonre24 · 2025-12-14T11:40:23Z

content/commands/ft.profile.md

 | `Total`&nbsp;`profile`&nbsp;`time`     | The total run time (ms) of the query. Normally just a few ms. |
 | `Parsing`&nbsp;`time`           | The time (ms) spent parsing the query and its parameters into a query plan. Normally just a few ms. |
 | `Pipeline`&nbsp;`creation`&nbsp;`time` | The creation time (ms) of the execution plan, including iterators, result processors, and reducers creation. Normally just a few ms for `FT.SEARCH` queries, but expect a larger number for `FT.AGGREGATE` queries. |
+| `Total`&nbsp;`GIL`&nbsp;`time`     | The total time (ms) the query held the Global Interpreter Lock (GIL) during execution. Relevant for multi-threaded deployments where queries run in background threads. |


Let's refer it as the Global Redis Lock, and explain when we are expected to hold it (upon LOADing non sortable fields). Also, does it include the time we are waiting for the GIL, or only the time we are actually holding it?

@lerman25 ?

It include the time waiting for the GIL ( We start the timer, and then call RedisModule_ThreadSafeContextLock)

alonre24 · 2025-12-14T11:44:25Z

content/commands/ft.profile.md

 | `Pipeline`&nbsp;`creation`&nbsp;`time` | The creation time (ms) of the execution plan, including iterators, result processors, and reducers creation. Normally just a few ms for `FT.SEARCH` queries, but expect a larger number for `FT.AGGREGATE` queries. |
+| `Total`&nbsp;`GIL`&nbsp;`time`     | The total time (ms) the query held the Global Interpreter Lock (GIL) during execution. Relevant for multi-threaded deployments where queries run in background threads. |
 | `Warning`                | Errors that occurred during query execution. |
+| `Internal`&nbsp;`cursor`&nbsp;`reads` | The number of internal cursor read operations performed during a distributed `AGGREGATE` query in cluster mode. In cluster mode, the coordinator uses cursors to fetch results from shards - this counts the initial request plus any subsequent `FT.CURSOR READ`. |


Consider rephrasing instead of using ft.cursor read, so it will not mix up with user cursors. Maybe something like "... any subsequent batch of results featching from the shard"

alonre24 · 2025-12-14T11:45:33Z

content/commands/ft.profile.md

 * `EMPTY`
 * `WILDCARD`
 * `OPTIONAL`
+* `OPTIMIZER` with `Optimizer mode` - Enabled by default in dialect 4+, or explicitly with `WITHOUTCOUNT`.


Is this true for both ft.search and ft.aggregate? cause ft.aggregate is without count by default AFAIK

lerman25 · 2025-12-15T16:52:06Z

content/commands/ft.profile.md

 | `Total`&nbsp;`profile`&nbsp;`time`     | The total run time (ms) of the query. Normally just a few ms. |
 | `Parsing`&nbsp;`time`           | The time (ms) spent parsing the query and its parameters into a query plan. Normally just a few ms. |
 | `Pipeline`&nbsp;`creation`&nbsp;`time` | The creation time (ms) of the execution plan, including iterators, result processors, and reducers creation. Normally just a few ms for `FT.SEARCH` queries, but expect a larger number for `FT.AGGREGATE` queries. |
+| `Total`&nbsp;`GIL`&nbsp;`time`     | The total time (ms) the query held the Global Interpreter Lock (GIL) during execution. Relevant for multi-threaded deployments where queries run in background threads. |


Suggested change

| `Total` `GIL` `time` | The total time (ms) the query held the Global Interpreter Lock (GIL) during execution. Relevant for multi-threaded deployments where queries run in background threads. |

| `Total` `GIL` `time` | The total time (ms) the query held the Global Interpreter Lock (GIL) during execution. This field is only applicable for shards in clustered environments and may present a value on the coordinator that does not reflect actual GIL usage.|

| `Total` `GIL` `time` | In multi-threaded deployments, the total time (ms) the query spent holding and waiting for the Redis Global Lock (referred to as GIL) during execution. Note: this value is only valid in the shards profile section. |

?

WDYT about emphasizing that's invalid any other place?

| Total GIL time | In multi-threaded deployments, the total time (ms) the query spent holding and waiting for the Redis Global Lock (referred to as GIL) during execution. Note: this value is only valid in the shards profile section; invalid elsewhere.. |

dont think necessary

lerman25 · 2025-12-15T16:53:38Z

content/commands/ft.profile.md

 | `Scorer`          | The `Scorer` processor assigns a relevance score to each document based on the query’s specified scoring function. This function could involve factors like term frequency, inverse document frequency, or other weighted metrics. |
 | `Sorter`          | The `Sorter` processor arranges the query results based on a specified sorting criterion. This could be a field value (e.g., sorting by price, date, or another attribute) or by the score assigned during the scoring phase. It operates after documents are fetched and scored, ensuring the results are ordered as required by the query (e.g., ascending or descending order). `Scorer` results will always be present in `FT.SEARCH` profiles. |
 | `Loader`          | The `Loader` processor retrieves the document contents after the results have been sorted and filtered. It ensures that only the fields specified by the query are loaded, which improves efficiency, especially when dealing with large documents where only a few fields are needed. |
+| `Threadsafe-Loader` | The `Threadsafe-Loader` processor safely loads document contents when the query is running in a background thread. It acquires the GIL to access document data. Reports an additional `GIL-Time` field showing how long (ms) it held the GIL. |


Suggested change

| `Threadsafe-Loader` | The `Threadsafe-Loader` processor safely loads document contents when the query is running in a background thread. It acquires the GIL to access document data. Reports an additional `GIL-Time` field showing how long (ms) it held the GIL. |

| `Threadsafe-Loader` | The `Threadsafe-Loader` processor safely loads document contents when the query is running in a background thread. It acquires the GIL to access document data. Reports an additional `GIL-Time` field representing both GIL hold time and time spent waiting to obtain the GIL.|

| `Threadsafe-Loader` | The `Threadsafe-Loader` processor safely loads document contents when the query runs in a background thread (relevant for multi-threaded deployments). It acquires the Redis Global Lock (referred to as GIL) to access document data. Reports an additional `GIL-Time` field representing the time (ms) spent holding the GIL and waiting to acquire it. |

?

dwdougherty

I made some changes as part of my copy-edit pass. Approving.

MOD-12263 vector iterator details

c0895d4

Added detailed notes on vector iterator profiles, including vector search modes and batch execution statistics.

MOD-6056 rename “counter” and “size”

a9056bb

dwdougherty self-assigned this Dec 11, 2025

dwdougherty added dev cmds labels Dec 11, 2025

MOD-12816 update ResultProcessor GILTime

8c47b85

meiravgri changed the title ~~MOD-12263 vector iterator details~~ [MOD-6057] Improve and elaborate FT.PROFILE documentaion Dec 11, 2025

MOD-12414 Add internal cursor reads to profile documentation

b8bc7a1

dwdougherty changed the title ~~[MOD-6057] Improve and elaborate FT.PROFILE documentaion~~ [MOD-6057] Improve and elaborate FT.PROFILE documentation Dec 12, 2025

meiravgri added 4 commits December 14, 2025 07:01

add missing iterators

917ab31

update RP section

16001a3

should we remove vector filter?

fix example with new names

6b461bd

remove vector filter

7c3947b

meiravgri commented Dec 14, 2025

View reviewed changes

content/commands/ft.profile.md Show resolved Hide resolved

meiravgri requested review from alonre24 and lerman25 December 14, 2025 08:32

alonre24 reviewed Dec 14, 2025

View reviewed changes

refine internal cursor description

ea606d5

lerman25 reviewed Dec 15, 2025

View reviewed changes

fix GIL entreies & optimizer

2ecda01

meiravgri requested review from alonre24 and lerman25 December 16, 2025 05:24

alonre24 added 2 commits December 16, 2025 14:54

add versions and format disclaimer + Shard ID field

f1f2b52

fix version disclaimer

d36ef27

meiravgri requested a review from dwdougherty December 16, 2025 13:00

DWD copy-edit pass

f886a42

dwdougherty approved these changes Dec 16, 2025

View reviewed changes

dwdougherty merged commit 18feb83 into main Dec 16, 2025
7 checks passed

dwdougherty deleted the meiravgri-ft-profile-docs branch December 16, 2025 15:25

	\| `Total` `GIL` `time` \| The total time (ms) the query held the Global Interpreter Lock (GIL) during execution. Relevant for multi-threaded deployments where queries run in background threads. \|
	\| `Total` `GIL` `time` \| The total time (ms) the query held the Global Interpreter Lock (GIL) during execution. This field is only applicable for shards in clustered environments and may present a value on the coordinator that does not reflect actual GIL usage.\|

[MOD-6057] Improve and elaborate FT.PROFILE documentation #2532

[MOD-6057] Improve and elaborate FT.PROFILE documentation #2532

Uh oh!

Conversation

meiravgri commented Dec 11, 2025 • edited by atlassian bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CLAassistant commented Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 11, 2025

Uh oh!

dwdougherty commented Dec 11, 2025

Uh oh!

Uh oh!

alonre24 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lerman25 Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dwdougherty left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

[MOD-6057] Improve and elaborate `FT.PROFILE` documentation #2532

[MOD-6057] Improve and elaborate `FT.PROFILE` documentation #2532

meiravgri commented Dec 11, 2025 •

edited by atlassian bot

Loading

CLAassistant commented Dec 11, 2025 •

edited

Loading

lerman25 Dec 15, 2025 •

edited

Loading