Skip to content

Rewrite the Mahler & Yum example in modern pylcm style#363

Open
hmgaudecker wants to merge 8 commits into
refactor/phase-2-api-reorganisationfrom
example/mahler-yum-from-replication
Open

Rewrite the Mahler & Yum example in modern pylcm style#363
hmgaudecker wants to merge 8 commits into
refactor/phase-2-api-reorganisationfrom
example/mahler-yum-from-replication

Conversation

@hmgaudecker
Copy link
Copy Markdown
Member

@hmgaudecker hmgaudecker commented May 22, 2026

Summary

Replaces the precomputed-grid lcm_examples/mahler_yum_2024/_model.py with an
in-DAG model:

  • In-DAG computation — the health-transition logit and income are computed
    inside the regime DAG, not precomputed into grids. Removes the import-time
    tr2yp_grid tensor and the from _lcm.utils.dispatchers import productmap
    private-API import.
  • Public lcm API only, real @categorical for the 40-level Effort,
    labeled pd.Series grids, fixed_params/START_PARAMS split, and a
    create_inputs that returns a (model_params, initial_conditions_DataFrame)
    pair. Dead ProductivityShock categorical dropped.
  • Package collapsed into __init__.py (the _model.py indirection is gone).
  • Behavioral-moment tests are carried as tests/test_mahler_yum_2024.py;
    the opaque pickle regression (test_regression_mahler_yum) and its fixtures
    are dropped. Benchmark, docs, and generate_benchmark_data.py updated
    to the new create_inputs signature.

Notes for review

  • @mj023 — the model carries utility_constant = 11.0 from
    the other repo's START_PARAMS. The previous standalone example used
    13.108 (which looks like a paper estimate).
  • The Mahler & Yum solve is GPU-scale and float32-unstable; the example tests
    are GPU-only and run at float64.

Test plan

  • test_mahler_yum_2024.py — 29 passed on GPU at float64
  • ty, prek clean
  • CI on the 361 branch

🤖 Generated with Claude Code

hmgaudecker and others added 2 commits May 22, 2026 19:34
Derive the example from the standalone Replication_MY2024 project: an
in-DAG model (health-transition logit and income computed inside the DAG,
not precomputed grids), public `lcm` API only, labeled pandas grids, and a
`create_inputs` that returns a DataFrame of initial conditions. Collapse the
package into `__init__.py`.

Carry the replication's behavioral-moment tests as tests/test_mahler_yum_2024.py
and drop the opaque pickle regression; update the benchmark, the example docs,
and generate_benchmark_data.py to the new create_inputs signature.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The Mahler & Yum solve is GPU-scale and float32-unstable (XLA's float32 fused
kernels accumulate ~1e-3 errors across the large state space). Mark the example
tests GPU-only and pin the behavioral moments to their float64 values.
Regenerate pixi.lock onto current jax/numpy/pandas.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@hmgaudecker hmgaudecker changed the title Rewrite the Mahler & Yum example as a boiled-down replication model Rewrite the Mahler & Yum example in modern pylcm style May 22, 2026
@hmgaudecker hmgaudecker requested a review from mj023 May 22, 2026 17:43
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 22, 2026

Benchmark comparison (main → HEAD)

Comparing 629ac442 (main) → 6392dfd9 (HEAD)

Benchmark Statistic before after Ratio Alert
aca-baseline execution time 26.281 s 15.222 s 0.58
peak GPU mem 639 MB 581 MB 0.91
compilation time 297.53 s 321.01 s 1.08
peak CPU mem 7.40 GB 7.95 GB 1.07
aca-baseline-debug execution time 74.031 s
peak GPU mem 581 MB
compilation time 421.92 s
peak CPU mem 7.58 GB
Mahler-Yum execution time 4.679 s 4.703 s 1.01
peak GPU mem 529 MB 520 MB 0.98
compilation time 14.15 s 11.87 s 0.84
peak CPU mem 1.71 GB 1.57 GB 0.92
Precautionary Savings - Solve execution time 46.3 ms 26.9 ms 0.58
peak GPU mem 101 MB 8 MB 0.08
compilation time 2.66 s 1.57 s 0.59
peak CPU mem 1.14 GB 1.12 GB 0.99
Precautionary Savings - Simulate execution time 120.2 ms 104.9 ms 0.87
peak GPU mem 349 MB 162 MB 0.46
compilation time 5.01 s 4.23 s 0.85
peak CPU mem 1.34 GB 1.33 GB 0.99
Precautionary Savings - Solve & Simulate execution time 159.7 ms 130.4 ms 0.82
peak GPU mem 586 MB 586 MB 1.00
compilation time 7.22 s 5.49 s 0.76
peak CPU mem 1.30 GB 1.30 GB 1.00
Precautionary Savings - Solve & Simulate (irreg) execution time 288.0 ms 292.2 ms 1.01
peak GPU mem 2.20 GB 2.20 GB 1.00
compilation time 7.53 s 5.77 s 0.77
peak CPU mem 1.36 GB 1.37 GB 1.01

@read-the-docs-community
Copy link
Copy Markdown

read-the-docs-community Bot commented May 23, 2026

hmgaudecker and others added 5 commits May 23, 2026 07:00
The behavioral-moment bands (`abs(count - expected) <= 5`,
`atol=0.01` on means, `atol=0.005` on fractions) are calibrated to
the paper's 64-bit replication. Under 32-bit precision the
simulation drifts past every band — counts off by 16, 30, 45 against
±5 bands; mean wealth off by enough to miss `atol=0.01`. Skip the
module at 32-bit so the GPU 32-bit-precision job is not blocked by
drift in a test whose role is paper replication, which has a single
canonical precision.

The dtype-correctness side of 32-bit is covered by the per-precision
regression pickles in `tests/data/regression_tests/{f32,f64}/` for
the mortality and precautionary-savings models.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant