You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When generating tasks for large datasets this can be quite slow and maybe it would be best to chuck up the dataset and call compute using threading this would allow us to submit the datasets faster.
The text was updated successfully, but these errors were encountered:
Submission threading is now possible and for a 3000 entry optimization dataset we see around a 1.4x speed up during the submission process. We might be able to gain more speedups using some combination of threading and multiprocessing, Ill leave this open to collect ideas.
When generating tasks for large datasets this can be quite slow and maybe it would be best to chuck up the dataset and call compute using threading this would allow us to submit the datasets faster.
The text was updated successfully, but these errors were encountered: