You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am using hnswlib for online indexing, where I continuously add vectors to my vector index library from a message queue using multithreading. Additionally, I perform multithreaded queries and execute a save operation after writing a certain number of items.
Question
As a user of hnswlib, do I need to implement locking to handle these operations effectively?
Operations:
Adding vectors from a message queue (multithreaded)
Performing queries (multithreaded)
Periodic save operations after writing a certain number of items
Additional Information
Any recommendations on how to manage concurrency and ensure data integrity would be greatly appreciated.
The text was updated successfully, but these errors were encountered:
I was also looking into the concurrent read/write in HNSWLib. From my understanding, locking is performed during write (addPoint/deletePoint), but not read (searchKnn). Can someone confirm whether this is true? And is non-locking read done purposely for performance consideration? @yurymalkov Thanks.
Sorry for a late reply. Yes, it is purely performance-based. That is something we wanted to fix for a while (making a separate method which is thread-safe), but didn't prioritize.
Description
I am using hnswlib for online indexing, where I continuously add vectors to my vector index library from a message queue using multithreading. Additionally, I perform multithreaded queries and execute a save operation after writing a certain number of items.
Question
As a user of hnswlib, do I need to implement locking to handle these operations effectively?
Operations:
Additional Information
Any recommendations on how to manage concurrency and ensure data integrity would be greatly appreciated.
The text was updated successfully, but these errors were encountered: