Skip to content

[Repo Assist] perf(falsify): replace N×nx.ancestors calls with transitive_closure_dag in validate_pd#1606

Draft
github-actions[bot] wants to merge 1 commit into
mainfrom
repo-assist/perf-falsify-pairs-20260623-02a06a4ef68bd2f6
Draft

[Repo Assist] perf(falsify): replace N×nx.ancestors calls with transitive_closure_dag in validate_pd#1606
github-actions[bot] wants to merge 1 commit into
mainfrom
repo-assist/perf-falsify-pairs-20260623-02a06a4ef68bd2f6

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

🤖 This PR was created by Repo Assist, an automated AI assistant.

Summary

validate_pd in dowhy/gcm/falsify.py was building the list of (ancestor, node) pairs by calling nx.ancestors() once per graph node:

# Before — O(N·(V+E)): N separate BFS/DFS traversals
pairs = [(ancestor, node) for node in causal_graph.nodes for ancestor in nx.ancestors(causal_graph, node)]
if adjacent_only:
    pairs = [(ancestor, node) for (ancestor, node) in pairs
             if ancestor in get_ordered_predecessors(causal_graph, node)]

For a graph with N=100 nodes that is called repeatedly (e.g., during falsification), this meant 100 Python function calls and 100 separate graph traversals before any independence tests even started.

Changes

adjacent_only=True — skip ancestor enumeration entirely; direct edges are already available as causal_graph.edges():

if adjacent_only:
    pairs = list(causal_graph.edges())   # O(E) vs O(N·(V+E))

adjacent_only=False (default) — use nx.transitive_closure_dag(), which adds edge (u, v) iff u can reach v (i.e. u is an ancestor of v). A single optimised graph traversal replaces N separate nx.ancestors() calls:

else:
    tc = nx.transitive_closure_dag(causal_graph)
    pairs = list(tc.edges())

nx.transitive_closure_dag has been available since NetworkX 2.7, which is below the project's minimum of >=2.8.5.

Impact

Scenario Before After
100-node graph, adjacent_only=False 100 BFS/DFS calls 1 optimised traversal
100-node graph, adjacent_only=True 100 BFS/DFS calls + filter 1 edge iteration
Correctness ✓ (proven: sorted(old) == sorted(new) on test graphs)

Test Status

Two new unit tests added in tests/gcm/test_falsify.py:

  • test_given_chain_dag_when_validate_pd_without_adjacent_only_then_tests_all_ancestor_pairs — verifies N_TESTS=3 for a 3-node chain (all ancestor pairs)
  • test_given_chain_dag_when_validate_pd_with_adjacent_only_then_tests_only_direct_edges — verifies N_TESTS=2 for a 3-node chain (only direct edges)

Full CI suite requires Poetry with all optional extras (not available in this environment). The optimisation is semantically equivalent: sorted(old_pairs) == sorted(new_pairs) was verified with a standalone NetworkX 3.4.2 script.

Generated by 🌈 Repo Assist, see workflow run. Learn more.

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repo-assist.md@11c9a2c442e519ff2b427bf58679f5a525353f76

…ag in validate_pd

For a graph with N nodes and E edges, the old implementation called
nx.ancestors() N times — one BFS/DFS per node, totalling O(N·(V+E))
Python function calls.

This commit replaces both paths:

  adjacent_only=True:
    Use list(causal_graph.edges()) directly — no ancestor enumeration
    needed at all. O(E) instead of O(N·(V+E)).

  adjacent_only=False (default):
    Use nx.transitive_closure_dag(), which adds an edge (u,v) iff u
    can reach v (u is an ancestor of v). A single optimised graph
    traversal replaces N separate nx.ancestors() calls, reducing
    Python-function-call overhead from O(N) to O(1). Correctness is
    unchanged: the set of (ancestor, descendant) pairs is identical.

Also adds two lightweight unit tests that assert the correct N_TESTS
count for both code paths on a three-node chain DAG.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants