fix(denoise): learn the signal amplitude, normalize targets, and match Xresampled by name

[M-Colley]

Summary

With denoise=True, the Gaussian process produced denoised targets that were much worse than the raw data whenever y was not on unit scale. The kernel was RBF + WhiteKernel + ConstantKernel, with no amplitude on the RBF term and no normalize_y. The smooth component therefore had a fixed prior variance of 1, and for larger-scale targets the fit attributed most of the signal to noise.

For y = A·sin(2x) + noise, with noise s.d. = 0.1·A and n = 200, here is the RMSE of the denoised y against the true signal, as a multiple of the noise level:

A	master	this PR	raw y
1	0.25	0.24	0.97
100	7.33	0.25	1.03
1e4	7.35	0.23	1.02

The PR has three commits:

1. Fix the kernel. Use ConstantKernel() * RBF(...) + WhiteKernel(...) with normalize_y=True, and standardize X (applying the same transform to Xresampled) so the initial length scales suit any units.
2. Use 5 optimizer restarts instead of 50. Each restart re-runs the O(n³)-per-step marginal-likelihood optimization, which put denoising at minutes for a few hundred rows: about 55-63 s at n=250 and about 10 min at n=500, with 3 features. With standardized inputs and targets, the fitted RMSE was identical for 0, 2, 10 and 50 restarts on every case I tried, so 5 is kept as a guard against poor local optima. This commit is separate in case you'd rather keep more.
3. Match Xresampled to X by column name. check_array(Xresampled) dropped the column names, so DataFrames with columns in a different order silently swapped columns. The test uses a temperature column in [300, 400] and a fraction in [0, 1]. The PR also raises a clear error when columns are missing, and when weights are combined with Xresampled under denoising: the targets then have one row per row of Xresampled, so the weights no longer line up, which previously failed with a bare AssertionError.

Tests

• TestDenoising.test_denoise_learns_signal_amplitude: at amplitude 100 the RMSE is 73 on master against a noise level of 10.
• TestDenoising.test_xresampled_columns_are_matched_by_name

Both fail on master and pass here. test_pandas_resample_with_nested_constraints also passes.

🤖 Generated with Claude Code

[MilesCranmer]

This is a behavior change, correct?

[M-Colley]

Yes, for anyone using denoise=True. It's opt-in, so nothing changes otherwise:

• Denoised targets differ. The GP now learns the signal amplitude (ConstantKernel() * RBF) with normalize_y=True, and standardizes X first. For targets near unit scale the result is close to before (0.24 vs 0.25 × the noise level in my benchmark) but not identical. For larger-scale targets this is the intended fix: 7.3× → 0.25× the noise level at amplitude 100.
• 5 optimizer restarts instead of 50 (separate commit). This can only change the fit if one of the dropped restarts would have found a better optimum; in my tests the fits were identical for 0 to 50 restarts. Happy to drop the commit if you'd rather keep 50.
• Xresampled DataFrames are matched to X by column name instead of position. A differently ordered Xresampled now gets the correct columns, and missing columns raise a ValueError.
• weights together with Xresampled under denoise=True now raises a clear ValueError. It previously failed with a bare AssertionError, so no working code path changes.

With a fixed random_state, results are still reproducible, just not identical to earlier versions.

[MilesCranmer]

Sorry to ask but could you be clear which part is you and which part is suggested by Claude? I am trying to understand the need better. PySR is quite a mature library at this point so any behavior changes need a strong motivation.

[M-Colley]

Fair question. The PR came from Claude Code: I asked it to review PySR, and it found this by reading denoising.py and wrote the fix, tests and description; I opened it. I haven't hit it in practice myself, but driving data (speeds in km/h) and questionnaire scores like SUS (0–100) are the kind of data I'd use it on. Re-running the benchmark on data shaped like that (n=200, 3 runs, RMSE against the true signal):

	raw	current	this PR
SUS 0–100 (noise 10)	10.4	5.8	3.1
speed km/h (noise 5)	4.8	24.9	1.1
speed km/h, inputs in m and s	5.0	10.0	1.5

So for speeds, denoising currently makes the data about 5× worse than not denoising at all. One caveat: for targets near unit scale (e.g. 1–7 Likert), it makes no difference with strong effects, and normalize_y was somewhat worse with weak effects (1.00 vs 0.65), so it isn't a free win everywhere.