You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Over in numpy/numpy#27786 we had a NumPy user report a parallel scaling issue and I went through some effort to debug it.
The debugging section in the guide should explain some techniques for identifying scaling issues and then diagnosing their source. We should explain how to run native python extensions under native performance profiling tools like perf to generate a flame graph. I didn't do this over in NumPy, but we should also explore other tooling like Intel's VTune and valgrind that can detect multithreaded scaling issues. samply might also be worth looking into, since it works on both MacOS and Linux.
Longer term, it would be nice to have tooling to help with this. This is simpler than other kinds of benchmarking since you don't need to compare across commits - just detect that the parallel scaling is bad in a single commit.
The text was updated successfully, but these errors were encountered:
Over in numpy/numpy#27786 we had a NumPy user report a parallel scaling issue and I went through some effort to debug it.
The debugging section in the guide should explain some techniques for identifying scaling issues and then diagnosing their source. We should explain how to run native python extensions under native performance profiling tools like
perf
to generate a flame graph. I didn't do this over in NumPy, but we should also explore other tooling like Intel's VTune and valgrind that can detect multithreaded scaling issues. samply might also be worth looking into, since it works on both MacOS and Linux.Longer term, it would be nice to have tooling to help with this. This is simpler than other kinds of benchmarking since you don't need to compare across commits - just detect that the parallel scaling is bad in a single commit.
The text was updated successfully, but these errors were encountered: