Adding BPOLY calibration option to CARACal

[rubyvanrooyen]

Updates and discussion as per Issue #1419

• Add schema for BPOLY calibration parameters to crosscal_schema.yml
• Update crosscal_worker.py to allow CASA BPOLY bandpass calibration option and sidestep issues due to change in cal table format

A sample configuration will be added later as part of a maser processing application example

[ratt-priv-ci]

Can one of the admins verify this patch?

[bennahugo]

Do you want to initiate testing?

[paoloserra]

@SpheMakh this is still work in progress as some more development is needed to be able to apply the BPOLY solutions to the target in the current Caracal workflow. However, we've just agreed that it would be good if you could review the new bits of code already now. Thanks!

[paoloserra]

@SpheMakh kind reminder, we'd still a preliminary code review from you

[SpheMakh]

Ready to test

[SpheMakh]

OK to test

[paoloserra]

@rubyvanrooyen can this be merged once updated to current master?

[Athanaseus]

2025-01-29 12:54:21     INFO    applycal::::+   ##########################################
2025-01-29 12:54:21     INFO    applycal::::+   ##### Begin Task: applycal           #####
2025-01-29 12:54:21     INFO    applycal::::    applycal(vis="/stimela_mount/msdir/1630886477_sdp_l0-cal.ms",field="J0408-6545",spw="",intent="",selectdata=True,
2025-01-29 12:54:21     INFO    applycal::::+           timerange="",uvrange="",antenna="",scan="",observation="",
2025-01-29 12:54:21     INFO    applycal::::+           msselect="",docallib=False,callib="",gaintable=['/stimela_mount/output/67p-1630886477_sdp_l0-1gc_primary.G1', '/stimela_mount/output/67p-1630886477_sdp_l0-1gc_primary.K1', '/stimela_mount/output/67p-1630886477_sdp_l0-1gc_primary.P1'],gainfield=['nearest', 'nearest', 'nearest'],
2025-01-29 12:54:21     INFO    applycal::::+           interp=['linear', 'linear', 'linear'],spwmap=[],calwt=[False],parang=False,applymode="calflag",
2025-01-29 12:54:21     INFO    applycal::::+           flagbackup=False)
2025-01-29 12:54:21     INFO    applycal::calibrater::open      ****Using NEW VI2-driven calibrater tool****
2025-01-29 12:54:21     INFO    applycal::calibrater::open      Opening MS: /stimela_mount/msdir/1630886477_sdp_l0-cal.ms for calibration.
2025-01-29 12:54:21     INFO    applycal::Calibrater::  Initializing nominal selection to the whole MS.
2025-01-29 12:54:21     INFO    applycal::calibrater::setdata   Beginning selectvis--(MSSelection version)-------
2025-01-29 12:54:21     INFO    applycal::calibrater::reset     Reseting solve/apply state
2025-01-29 12:54:21     INFO    applycal::Calibrater::selectvis Performing selection on MeasurementSet
2025-01-29 12:54:21     INFO    applycal::Calibrater::selectvis+         Selecting on field: 'J0408-6545'
2025-01-29 12:54:21     INFO    applycal::Calibrater::selectvis By selection 215940 rows are reduced to 124440
2025-01-29 12:54:21     INFO    applycal::Calibrater::selectvis Frequency selection: Selecting all channels in all spws.
2025-01-29 12:54:21     INFO    applycal::calibrater::setdata   chanmode=none nchan=1 start=0 step=1 mStart='0km/s' mStep='0km/s' msSelect=''
2025-01-29 12:54:21     INFO    applycal::calibrater::setapply  Beginning setapply--(MSSelection version)-------
2025-01-29 12:54:21     INFO    applycal::Calibrater::setapply(type, applypar)  Arranging to APPLY:
2025-01-29 12:54:21     INFO    applycal::::    Calibration field mapping for 67p-1630886477_sdp_l0-1gc_primary.G1 (via gainfield='nearest'): [0, 0]
2025-01-29 12:54:21     INFO    applycal::::+    Separations (deg): []
2025-01-29 12:54:21     INFO    applycal::Calibrater::setapply(type, applypar)  .   G Jones: table=/stimela_mount/output/67p-1630886477_sdp_l0-1gc_primary.G1 select= interp=linear spwmap=[-1] calWt=false
2025-01-29 12:54:21     INFO    applycal::calibrater::setapply  Beginning setapply--(MSSelection version)-------
2025-01-29 12:54:21     INFO    applycal::Calibrater::setapply(type, applypar)  Arranging to APPLY:
2025-01-29 12:54:22     INFO    applycal::::    Calibration field mapping for 67p-1630886477_sdp_l0-1gc_primary.K1 (via gainfield='nearest'): [0, 0]
2025-01-29 12:54:22     INFO    applycal::::+    Separations (deg): []
2025-01-29 12:54:22     INFO    applycal::Calibrater::setapply(type, applypar)  .   K Jones: table=/stimela_mount/output/67p-1630886477_sdp_l0-1gc_primary.K1 select= interp=linear spwmap=[-1] calWt=false
2025-01-29 12:54:22     INFO    applycal::calibrater::setapply  Beginning setapply--(MSSelection version)-------
2025-01-29 12:54:22     INFO    applycal::Calibrater::setapply(type, applypar)  Arranging to APPLY:
2025-01-29 12:54:23     INFO    applycal::::    Calibration field mapping for BpolyAsB.temp (via gainfield='nearest'): [-1, -1]
2025-01-29 12:54:23     INFO    applycal::::+    Separations (deg): []
2025-01-29 12:54:23     INFO    applycal::Calibrater::setapply(type, applypar)  .   BPOLY: table=/stimela_mount/output/67p-1630886477_sdp_l0-1gc_primary.P1 select= interp=linear,linear spwmap=[-1] calWt=false
2025-01-29 12:54:23     INFO    applycal::calibrater::correct   Beginning correct---------------------------
2025-01-29 12:54:23     INFO    applycal::Calibrater::correct2 (VI2/VB2)        The following calibration terms are arranged for apply:
2025-01-29 12:54:23     INFO    applycal::Calibrater::correct2 (VI2/VB2)        .   BPOLY: table=/stimela_mount/output/67p-1630886477_sdp_l0-1gc_primary.P1 select= interp=linear,linear spwmap=[-1] calWt=false
2025-01-29 12:54:23     INFO    applycal::Calibrater::correct2 (VI2/VB2)        .   K Jones: table=/stimela_mount/output/67p-1630886477_sdp_l0-1gc_primary.K1 select= interp=linear spwmap=[-1] calWt=false
2025-01-29 12:54:23     INFO    applycal::Calibrater::correct2 (VI2/VB2)        .   G Jones: table=/stimela_mount/output/67p-1630886477_sdp_l0-1gc_primary.G1 select= interp=linear spwmap=[-1] calWt=false
2025-01-29 13:00:05     INFO    applycal::calibrater::correct   Finished correcting.
2025-01-29 13:00:05     INFO    applycal::::    Calibration apply flagging statistics (among calibrateable spws):
2025-01-29 13:00:05     INFO    applycal::::      Total visibilities selected for correction (ncorr x nchan x nrow summed over spws) = 2038824960
2025-01-29 13:00:05     INFO    applycal::::      Flags:
2025-01-29 13:00:05     INFO    applycal::::       BPOLY: In: 1600430500 / 2038824960   (78.4976901597%) --> Out: 1600430500 / 2038824960   (78.4976901597%) (/stimela_mount/output/67p-1630886477_sdp_l0-1gc_primary.P1)
2025-01-29 13:00:05     INFO    applycal::::       K Jones: In: 1600430500 / 2038824960   (78.4976901597%) --> Out: 1605955152 / 2038824960   (78.7686625143%) (/stimela_mount/output/67p-1630886477_sdp_l0-1gc_primary.K1)
2025-01-29 13:00:05     INFO    applycal::::       G Jones: In: 1605955152 / 2038824960   (78.7686625143%) --> Out: 1605955152 / 2038824960   (78.7686625143%) (/stimela_mount/output/67p-1630886477_sdp_l0-1gc_primary.G1)
2025-01-29 13:00:05     INFO    applycal::::    CASA Version 5.8.0-109
2025-01-29 13:00:05     INFO    applycal::::
double free or corruption (out)