[Bug]: [benchmark][cluster] Show partition failed when drop partition in concurrent DQL & multi-partition scene #38372
Assignees
Labels
kind/bug
Issues or changes related a bug
test/benchmark
benchmark test
triage/accepted
Indicates an issue or PR is ready to be actively worked on.
Milestone
Comments
wangting0128
added
kind/bug
wangting0128
changed the title
yanliang567
added
triage/accepted
congqixia
added a commit
to congqixia/milvus
that referenced
this issue
Dec 11, 2024
Related to milvus-io#38372 This PR make drop partition only check target partition load states only in case of concurrent releasing other partition in same collection. Signed-off-by: Congqi Xia <[email protected]>
|
The problem here was concurrent releasing & drop partition on same collection: Drop Partition task tried to check partition loaded state while concurrent release partition make querycoord show partition return error |
sre-ci-robot
pushed a commit
that referenced
this issue
Dec 11, 2024
…38375) Related to #38372 This PR make drop partition only check target partition load states only in case of concurrent releasing other partition in same collection. --------- Signed-off-by: Congqi Xia <[email protected]>
congqixia
added a commit
to congqixia/milvus
that referenced
this issue
Dec 12, 2024
…ilvus-io#38375) Related to milvus-io#38372 This PR make drop partition only check target partition load states only in case of concurrent releasing other partition in same collection. --------- Signed-off-by: Congqi Xia <[email protected]>
|
This is a rare problem, I'll observe the test results for a few days |
not reproduceargo task：bitmap-corn-1733972400 |
sre-ci-robot
pushed a commit
that referenced
this issue
Dec 12, 2024
…tion (#38375) (#38394) Cherry pick from master pr: #38375 Related to #38372 This PR make drop partition only check target partition load states only in case of concurrent releasing other partition in same collection. --------- Signed-off-by: Congqi Xia <[email protected]>
After testing more than 20 times, this problem did not recur, close it |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is there an existing issue for this?
Environment
Current Behavior
argo task: fouramf-bitmap-scenes-tw86w
test case name: test_bitmap_locust_hybrid_index_cluster
server:
partitionID=454527759048375665 -> partition name="scene_test_partition_dx2Qffte"
{pod=~"fouramf-bitmap-scenes-tw86w-10-milvus-.*"} |~ "efbd5b23076d27d6f774ce63685f1d99"| [2024/12/11 06:26:47.078 +00:00] [WARN] [proxy/impl.go:1421] ["DropPartition failed to WaitToFinish"] [traceID=efbd5b23076d27d6f774ce63685f1d99] [role=proxy] [db=default] [collection=fouram_IPSnBSas] [partition=scene_test_partition_hybrid_search_BwV46GtI] [error="partition not loaded[partition=454527759048375665]"] [BeginTS=454531064006443009] [EndTS=454531064006443009]
client log:
Expected Behavior
No response
Steps To Reproduce
concurrent test and calculation of RT and QPS :purpose: `primary key: INT64` 1. building default index on all supported 16 scalar fields 2. load and search partial partitions & DQL requests :test steps: 1. create collection with fields: 'float_vector': 128dim 'id': primary key type is INT64 all scalar fields: varchar max_length=100, array max_capacity=9 2. build indexes: IVF_SQ8: 'float_vector' default scalar index: all scalar fields 3. insert 6 million data 4. flush collection 5. build indexes again using the same params 6. load collection 7. concurrent request: - scene_test_partition (partition: create->insert->flush->index again->load->search->release->search failed->drop) - scene_test_partition_hybrid_search (partition: create->insert->flush->index again->load->hybrid_search->release->hybrid_search failed->drop) - search - query - hybrid_searchMilvus Log
No response
Anything else?
test result:
The text was updated successfully, but these errors were encountered: