Fix kahns algo sparse vertex performance - #15072
Conversation
for more information, see https://pre-commit.ci
|
@cclauss kindly review |
1 similar comment
|
@cclauss kindly review |
|
@mindaugl |
cclauss
left a comment
There was a problem hiding this comment.
Please add a benchmark that proves that the proposed modification could save significant runtime.
Sure I will do that |
|
What are the benchmark results on your machine? |
Here are the benchmark results on my machine comparing the pre-optimization implementation (list.pop(0)) against the current implementation (deque.popleft()) for topological_sort() on a 30,000-vertex DAG across 5 runs: Benchmark results for topological_sort with 30000 vertices over 5 runs: Pre-optimization (list.pop(0)): 0.29597 seconds Speedup ratio: 6.00x faster The benchmark directly measures topological_sort() on the exact same graph input and programmatically asserts that both implementations return valid, complete topological sorts. |
|
@priya-sundaram-dev, please review. |
priya-sundaram-dev
left a comment
There was a problem hiding this comment.
Thanks for the work here, and for caring about the pop(0) → deque.popleft() win.
I checked out refs/pull/15072/head and ran it: doctests pass (8/8), and benchmark() reports ~6.2x on my machine (list.pop(0) 0.343s vs deque.popleft 0.055s over 5 runs on 30k zero-indegree vertices), so the numbers hold up.
One thing worth flagging before this lands, though: the actual optimization (dict.fromkeys indegree + deque.popleft) is already on master. Diffing this branch against master, the only thing 15072 still adds is:
_topological_sort_list_queue()— a ~30-line copy of the pre-optimization code, kept solely as a benchmark baseline, andbenchmark()+ abenchmark()call under__main__.
My concern is that this permanently keeps a deliberately-slower duplicate implementation in a learning-resource file. Anyone reading kahns_algorithm_topo.py now sees two topological sorts and has to figure out that one exists only to be beaten in a microbenchmark. TA/Python files are meant to be read as clean references, and the speedup is already captured by the merged change.
Suggestion: drop _topological_sort_list_queue and benchmark() from the committed file. If you'd like the benchmark preserved, it reads better as a note in the PR description (the 6.2x number you measured) than as code that ships. That leaves master with the fast version and no dead baseline.
Happy to re-review quickly whichever direction the maintainers prefer — this is a scope call, not a correctness problem.
Sure I will do that |
…ahns_algorithm_topo
priya-sundaram-dev
left a comment
There was a problem hiding this comment.
Thanks for the quick turnaround, @Kanika0306 — the shipped file is now clean and focused.
Re-reviewed refs/pull/15072/head:
- The dead
list.pop(0)baseline and the microbenchmark are gone, so the module stays lean — the 6.2x number lives in the PR description where it belongs. 👍 - The real substance holds up nicely: switching
indegreefrom a positional[0]*len(graph)list todict.fromkeys(graph, 0)is a genuine correctness fix, not just perf — the old code silently broke on non-contiguous / sparse vertex IDs. Verified:topological_sort({10:[20], 20:[]})→[10, 20],{10:[20], 20:[10]}→None(cycle),{0:[1,2],1:[3],2:[3],3:[]}→[0,1,2,3]. deque.popleft()removes the O(n) shift on the BFS frontier.- Doctests: 8/8 pass, including the new
sparse_graph/sparse_cycleguards.
LGTM. 🚀
cclauss
left a comment
There was a problem hiding this comment.
Nice teamwork! given enough eyeballs, all bugs are shallow
Describe your change:
Fixes the
O(N)queue operation caused bylist.pop(0)by usingcollections.dequeandqueue.popleft(), ensuring constant-time dequeue operations and preserving the expectedO(V + E)time complexity of Kahn's topological sort algorithm.Fixes an
IndexErrorfor sparse/non-contiguous integer vertex IDs by changingindegreefrom a list to a dictionary keyed by the graph vertices.Adds doctests covering topological sorting with sparse/non-contiguous integer vertex IDs.
Adds doctests covering cycle detection with non-contiguous vertex IDs.
Add an algorithm?
Fix a bug or typo in an existing algorithm?
Add or change doctests?
Documentation change?
Checklist:
Fixes #15071