docs: adding OpenMP conflicts resolution to FAISSDocumentStore documents#10740
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
|
||
| #### Symptoms | ||
|
|
||
| You may encounter one or both of the following errors at runtime: |
There was a problem hiding this comment.
📝 [vale] reported by reviewdog 🐶
[MyStyle.TheFollowing] Avoid 'the following'—it's implied by the colon. Rephrase or remove.
| resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown | ||
| ``` | ||
|
|
||
| If setting `OMP_NUM_THREADS=1` prevents the crash, the root cause is **multiple OpenMP runtimes loaded simultaneously**. Each runtime maintains its own thread pool and thread-local storage (TLS). When two runtimes spin up worker threads at the same time, they corrupt each other's memory — causing segfaults at `N > 1` threads. |
There was a problem hiding this comment.
📝 [vale] reported by reviewdog 🐶
[Google.Parens] Use parentheses judiciously.
| resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown | ||
| ``` | ||
|
|
||
| If setting `OMP_NUM_THREADS=1` prevents the crash, the root cause is **multiple OpenMP runtimes loaded simultaneously**. Each runtime maintains its own thread pool and thread-local storage (TLS). When two runtimes spin up worker threads at the same time, they corrupt each other's memory — causing segfaults at `N > 1` threads. |
There was a problem hiding this comment.
🚫 [vale] reported by reviewdog 🐶
[Google.EmDash] Don't put a space before or after a dash.
|
|
||
| #### Fix | ||
|
|
||
| The solution is to pick one canonical `libomp.dylib` (torch's is a good choice) and replace all other copies with symlinks pointing to it. |
There was a problem hiding this comment.
📝 [vale] reported by reviewdog 🐶
[Google.Parens] Use parentheses judiciously.
|
|
||
| The solution is to pick one canonical `libomp.dylib` (torch's is a good choice) and replace all other copies with symlinks pointing to it. | ||
|
|
||
| For each duplicate, delete the copy and replace it with a symlink: |
There was a problem hiding this comment.
[MyStyle.KeyboardKeys] Keyboard keys should be capitalized ('Delete', 'Enter', 'F2').
| /path/to/.venv/lib/pythonX.Y/site-packages/<package>/.dylibs/libomp.dylib | ||
| ``` | ||
|
|
||
| Repeat for every duplicate found. Because these packages use `@loader_path`-relative references to load `libomp.dylib`, the symlink will be transparently resolved to the single canonical runtime at load time. |
There was a problem hiding this comment.
📝 [vale] reported by reviewdog 🐶
[MyStyle.WeakWords] Avoid weak filler words like 'very'.
| /path/to/.venv/lib/pythonX.Y/site-packages/<package>/.dylibs/libomp.dylib | ||
| ``` | ||
|
|
||
| Repeat for every duplicate found. Because these packages use `@loader_path`-relative references to load `libomp.dylib`, the symlink will be transparently resolved to the single canonical runtime at load time. |
There was a problem hiding this comment.
[Google.Will] Avoid using 'will'.
davidsbatista
deleted the
docs/adding-FAISSDocumentStore-OpenMP-issue-fix
branch
2 participants
Notes for the reviewer
see: deepset-ai/haystack-integrations#410 (comment)
Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:and added!in case the PR includes breaking changes.