obs_nodding and obs_nodding_irregular do not seem to behave well with multiple exposures per nod

[Hoeijmakers]

I have been trying to reduce an ABBA sequence where each A and B nod consists of 5 exposures. So the sequence goes:
A,A,A,A,A,B,B,B,B,B, B,B,B,B,B,A,A,A,A,A, etc.

When using pipeline.obs_nodding, it takes exposure 5 (A) and exposure 6 (B) as a nodding pair. Exposures 1 to 4 and 7-10 are ignored entirely.

When using pipeline.obs_nodding_irregular, it starts by taking exposure 1 (A) and exposure 6 (B), which I think is the desired behaviour, and then continues with exposure 2 (A) and exposure 6 (B) again, and the remaining B exposures of the A-B pair are subsequently ignored.

I believe that a solution should ideally be at the level of obs_nodding (because this sequence is regular), in that it recognises the number of A and B frames in each nod pair correctly.

The desired behaviour is (probably) that it uses exposures 1 and 6, 2 and 7, 3 and 8, 4 and 9, 5 and 10, 11 and 16, 12 and 17, etc. as nod pairs. In the case of a time-series (which is my use case), the user can then use the individually extracted A and B exposures.

[tomasstolker]

Hi Jens,

Thanks for opening this issue! Currently it is indeed not the desired behavior of those routines. I had some discussion about this recently with Rico, but we hadn't found the time yet to implement this and deprecate obs_nodding_irregular.

The obs_nodding recipe from EsoRex can only do a pair-wise subtraction if I am not mistaken. Since we want indeed to maintain the individual integrations, we were thinking that we should first collapse several integrations to build up the background image that will be subtracted, in order to limit the noise in the individual nod-subtracted images? So e.g. collapse 6 to 10 and subtract this mean image from 1 to 5.

[Hoeijmakers]

I agree that merging frames is probably the best, most generic way to deal with this. That may be a bit easier said than done, if all those frames need to be individually pipeline reduced before being co-added. The most logical place where this should ultimately be addressed is probably the cr2res_obs_nodding pipeline recipe itself.

For now I have created a PR with a temporary fix based on obs_nodding_irregular, adding an optional keyword to pair A's and B's one to one in sequence.

[tomasstolker]

That's true, although util_calib can be used with calib_type='nodding' for calibrating the science frames. The output could then be collapsed by subsets and used for the pair-wise subtraction with obs_nodding. But this may not be something that can be quickly implemented indeed.

Thanks a lot for creating the PR with a temporary fix! That seem indeed a good solution 👍.