Primary beam considerations

[SpheMakh]

The primary beam (PB) has be normalised as a function of frequency. The current thinking is that the centre pixel should be unity for all frequency, we could do this by

J_normed = J x (J_o)^-1

where J is the Jones matrix, and J_o only has the centre pixel of the for each component of J.

It also came up in a discussion with @o-smirnov that the small scale ripple in the bandpass might be due to the PB ripple as a function of frequency. At least some of it, since substructure in the bandpass calibrator field can also cause this ripple.

We can test this by solving for the bandpass using different uv-ranges. If the ripple is dominated by substructure, then the ripple should be faster on long baselines compared to shorter ones.

[o-smirnov]

To clarify on the "beam ripple": since the "ripple" manifests itself as an up-down nodding of the beam as a function of frequency, even a source at the centre will experience a beam gain that is "ripply" in freqeuncy, even if only slightly.

Khan knows the period and scale of the "nodding", from this we can predict how much of a ripple we should be seeing. Somebody needs to check this.

[SpheMakh]

Adding @kmbasad to the discussion

[KshitijT]

I remember that from the simulations, we saw that at this point instrumental variation seemed to completely overpower the bandpass rippling from substructure (responsible for ~2-3% variation or so?), @SpheMakh , @bennahugo please correct if wrong. At what level do we expect the beam ripple to contribute to this? Probably the biggest contributor would be the variation from the central source, right?

[bennahugo]

@KshitijT @SpheMakh @o-smirnov The field substructure will introduce ~ -25dB ripple. Currently the bandpass noise floor is around -27dB to -28dB after taking the substructure into account. It is the same in the late afternoon early evening stretch and early morning, although we haven't tested severe temperature differentials - waiting till the summer kicks in. It is not elevation dependent either.

So you could be right - this could be a beam ripple -- I thought this could be strong RFI leaking into the rest of the band since the band is so contaminated. I don't know the suppression levels we expect with the polyphase filter they use in the frontend. Need to simulate and see if the beam rippling can account for the kind of bandpass variations we've seen in observations:

[bennahugo]

Interrestingly the variation is worse in the lower end of the band as you can see from m020 for example. @KshitijT @SpheMakh @o-smirnov.

[bennahugo]

Just to be clear the variation shape stays like this irrespective of integration time of the solutions, so the higher variation in the lower part of the band is not solely SEFD-related.

[SpheMakh]

There is something happening around 1.3GHz. The noise statistics change, and @kmbasad noted a change in the PB behaviour at the same frequency. These must be related somehow

[bennahugo]

@o-smirnov facepalm: I realized why my DEEP2 beam corrected images look so shit - the powerbeam is not normalized per frequency, raising the entire image mean. When I apply a first order correction of max(apparent_without_beam)/max(apparent_withbeam) * apparent_withbeam, and then scale things to the same levels things improve significantly - the overall noise is much better as I expected from simulation!
Without:

With:

[bennahugo]

(these are just 100MHz 1GC maps, so there is a lot still to be done)

[bennahugo]

@paoloserra I forgot to mention this gives me at least a 2x improvement in noise: Noise of 48.26uJy vs 25.7uJy. The second is an upper bound because the normalization is first order. It might be the missing factor you're looking for (wink wink)

[paoloserra]

Could you make a natural spectral line cube from the same data? Just a few channels would be sufficient, and no deconvolution. I'd be curious to know whether the noise goes down there too.

[gigjozsa]

Not clear to me how this could have an effect in 1GC. The ripple - would be great if this would explain the bandpass variation, but the rms?

[bennahugo]

@paoloserra currently we don't have a line deconvolution mode in DDFacet - unfortunately, on my list of TODOs :)

[gigjozsa]

Line imaging is enough, no deconvolution, if everything fails, flag all channels bar one.

[gigjozsa]

Btw, if that is missing, where can we place very urgent requests?

[bennahugo]

cyriltasse/DDFacet - it is already there as urgent.

[paoloserra]

currently we don't have a line deconvolution mode in DDFacet - unfortunately, on my list of TODOs :)

@bennahugo I meant even a single pointing, single channel natural dirty image from your data. You can use the HI imaging worker (WSClean or CASA CLEAN)

[bennahugo]

@paoloserra can't apply the beam with that. I might be able to check using josh's suggestion. Lets give this a try next week

[o-smirnov]

@bennahugo I meant even a single pointing, single channel natural dirty image from your data. You can use the HI imaging worker (WSClean or CASA CLEAN)

Deconvolve with DDF, have it write out the model into MODEL_DATA, write DATA minus MODEL_DATA into CORRECTED_DATA, and make a dirty image of a few channels of CORRECTED_DATA. Yes, this "dirty residual" will not have the beam corrections in it, but it will show if the deconvolved sources are subtracting better. I believe that's what @paoloserra is asking for.

[gigjozsa]

From Gosh: The rms reduction makes sense in mosaicing after primary-beam correciton, but how does the beam shape affect rms before primary-beam correction. Hopefully be in the office this afternoon to discuss this.

[gigjozsa]

OK, understand now. Yes, that's what Paolo's asking for.

[paoloserra]

Yes, that's what I'm asking for

[gigjozsa]

This effect should also be visible as ripple at the position of bright sources in a line data cube.

[gigjozsa]

Is this correct?

[bennahugo]

@o-smirnov Ah yes ok. Yes should be amplitude errors around the sources mostly

[bennahugo]

Let me get a good continuum map first