fix: Memory leak on reactive pipelines

[hoxbro]

Description

Derived rx nodes registered their invalidation callbacks (_invalidate_obj/_invalidate_current) on their source parameter as strong bound-method references. Since the source is typically long-lived, this pinned every derived node — and anything it captured, such as large arrays — alive indefinitely, with no way to reclaim it.

Wrap the invalidation callbacks in a weakref.WeakMethod (_WeakInvalidator) and register a finalizer to prune the watcher once the node is collected. The node now becomes garbage collectable as soon as it is no longer referenced, and the source's watcher list no longer grows without bound.

This changes no observable public API behavior: while an expression is referenced it behaves exactly as before, and watch()/_watch() are untouched. Only a node that was previously unreachable-but-uncollectable (pinned solely by the source's internal watcher) is now reclaimed.

import gc
import weakref

import numpy as np
import param
import psutil

ARRAY_ELEMENTS = 10_000_000  # ~80 MB per array
N_ITER = 20


def rss_mb():
    return psutil.Process().memory_info().rss / 1024 / 1024


def watcher_count(source):
    watchers = source._internal_params[0].owner._param__private.watchers
    return sum(len(lst) for what in watchers.values() for lst in what.values())


def run():
    a = param.rx(np.zeros(ARRAY_ELEMENTS))

    print(f"{'iter':>5} | {'watchers on a':>13} | {'b GC-able?':>10} | {'RSS (MB)':>9}")
    print("-" * 46)

    baseline = rss_mb()
    for i in range(N_ITER):
        payload = np.full(ARRAY_ELEMENTS, float(i))
        b = a + payload

        ref = weakref.ref(b)
        del b, payload
        gc.collect()
        leaked = ref() is not None  # alive => leaked, pinned by `a`

        print(
            f"{i:>5} | {watcher_count(a):>13} | "
            f"{'NO (leak)' if leaked else 'yes':>10} | "
            f"{rss_mb() - baseline:>+9.1f}"
        )


if __name__ == "__main__":
    run()

main branch:

 iter | watchers on a | b GC-able? |  RSS (MB)
----------------------------------------------
    0 |             6 |  NO (leak) |     +76.3
    1 |            10 |  NO (leak) |    +152.6
    2 |            14 |  NO (leak) |    +228.9
    3 |            18 |  NO (leak) |    +305.2
    4 |            22 |  NO (leak) |    +381.5
    5 |            26 |  NO (leak) |    +457.8
    6 |            30 |  NO (leak) |    +534.1
    7 |            34 |  NO (leak) |    +610.4
    8 |            38 |  NO (leak) |    +686.7
    9 |            42 |  NO (leak) |    +763.0
   10 |            46 |  NO (leak) |    +839.3
   11 |            50 |  NO (leak) |    +915.6
   12 |            54 |  NO (leak) |    +991.9
   13 |            58 |  NO (leak) |   +1068.2
   14 |            62 |  NO (leak) |   +1144.5
   15 |            66 |  NO (leak) |   +1220.8
   16 |            70 |  NO (leak) |   +1297.1
   17 |            74 |  NO (leak) |   +1373.4
   18 |            78 |  NO (leak) |   +1449.7
   19 |            82 |  NO (leak) |   +1526.0

This branch:

 iter | watchers on a | b GC-able? |  RSS (MB)
----------------------------------------------
    0 |             2 |        yes |      +0.0
    1 |             2 |        yes |      +0.0
    2 |             2 |        yes |      +0.0
    3 |             2 |        yes |      +0.0
    4 |             2 |        yes |      +0.0
    5 |             2 |        yes |      +0.0
    6 |             2 |        yes |      +0.0
    7 |             2 |        yes |      +0.0
    8 |             2 |        yes |      +0.0
    9 |             2 |        yes |      +0.0
   10 |             2 |        yes |      +0.0
   11 |             2 |        yes |      +0.0
   12 |             2 |        yes |      +0.0
   13 |             2 |        yes |      +0.0
   14 |             2 |        yes |      +0.0
   15 |             2 |        yes |      +0.0
   16 |             2 |        yes |      +0.0
   17 |             2 |        yes |      +0.0
   18 |             2 |        yes |      +0.0
   19 |             2 |        yes |      +0.0

AI Disclosure

Assisted-by: Claude:claude-opus-4-8

• I have tested all AI-generated content in my PR.
• I take responsibility for all AI-generated content in my PR.

Used AI to find the problem and the solution.

Checklist

• Tests added and are passing

[codecov]

Codecov Report

❌ Patch coverage is 89.47368% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.73%. Comparing base (1052848) to head (4d3bd45).

Files with missing lines	Patch %	Lines
param/reactive.py	89.47%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1156   +/-   ##
=======================================
  Coverage   86.72%   86.73%           
=======================================
  Files           9        9           
  Lines        5304     5321   +17     
=======================================
+ Hits         4600     4615   +15     
- Misses        704      706    +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.