[Bug]: Milvus insertion time getting slow after 3rd loop #38768
Labels
kind/bug
Issues or changes related a bug
needs-triage
Indicates an issue or PR lacks a `triage/foo` label and requires one.
Is there an existing issue for this?
Environment
Current Behavior
I encountered a strange problem: the insertion of Milvus data takes 10 seconds after the third loop, seemingly regardless of the data volume. However, when I terminate the loop, the first insertion only takes 3 seconds.
My code is roughly as follows:
python
embedding = Embeddings('BAAI/bge-m3', use_gpu=True)
vdb = Vector(embedding, db_name) # Connect to Milvus
for file in files:
chunks = convert_raw_2_vdb(vdb, file) # Start chunking into Milvus here
The following is the printed log:
plaintext
file_path = dataset/1.md
total_count = 18
time costs for embedding = 0.4152650833129883
time costs for insert milvus = 3.0823850631713867
file_path = dataset/2.md
total_count = 5
time costs for embedding = 0.10074329376220703
time costs for insert milvus = 7.72633171081543
file_path = dataset/3.md
total_count = 12
time costs for embedding = 0.15440130233764648
time costs for insert milvus = 10.726555109024048
file_path = dataset/4.md
total_count = 12
time costs for embedding = 0.15440130233764648
time costs for insert milvus = 10.776616109025023
…
Expected Behavior
No response
Steps To Reproduce
No response
Milvus Log
No response
Anything else?
No response
The text was updated successfully, but these errors were encountered: