Adapt HGCal rechit producer for HGCal testbeam data

[stahlleiton]

PR description:

So far there were two HGCal rechit data format directories: HGCalRecHit (including HGCalRecHitHost and HGCalRecHitSoA) and HGCalReco (including HGCalSoARecHits). The HGCalRecHit data formats were used in test beam workflows while HGCalReco formats are used by the TICL reconstruction.

This PR adapts the HGCalSoARecHits format to support the processing of testbeam data (modifying accordingly the HGCalRecHitCalibrationAlgorithms and HGCalRecHitProducers to use HGCalReco formats instead of the HGCalRecHit formats) and allows to run offline reconstruction (up to layer clusters at the moment). Thus, the HGCalRecHit data formats becomes obsolete by this PR.

Moreover this PR now apply energy loss and thickness corrections to the RecHits to convert charge (in fC) to energy (in GeV). See slide 9–12: https://indico.cern.ch/event/1554444/contributions/6545659/attachments/3078603/5448738/HGCal_rawdata_config_20250602.pptx-2.pdf#page=9

And finally it sets up a new relval using as input 2024 HGCal test beam data, which requires the following cms-data PRs: cms-data/Geometry-HGCalMapping#4 and cms-data/RecoLocalCalo-HGCalRecProducers#1

@pfs @IzaakWN

PR validation:

This PR can be tested using relval 77 with cms-data PRs cms-data/Geometry-HGCalMapping#4 and cms-data/RecoLocalCalo-HGCalRecProducers#1

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48606/45572

• There are other open Pull requests which might conflict with changes you have proposed:
  • File CondFormats/HGCalObjects/interface/HGCalCalibParamSoA.h modified in PR(s): [HGCAL] Add energy loss to Calibration SoA #48187, Updated SoA View accessors from raw pointers to span #48377
  • File CondFormats/HGCalObjects/interface/HGCalMappingParameterSoA.h modified in PR(s): Updated SoA View accessors from raw pointers to span #48377
  • File Configuration/PyReleaseValidation/python/relval_standard.py modified in PR(s): Restructure and comment the tau embedding method #48408, Fixed Number of Events Data Wfs for Testing #48490
  • File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Restructure and comment the tau embedding method #48408, Fixed Number of Events Data Wfs for Testing #48490, Implement template helper functions to reduce macro depencencies in SoA backend #48605
  • File DataFormats/HGCalRecHit/interface/HGCalRecHitSoA.h modified in PR(s): Updated SoA View accessors from raw pointers to span #48377
  • File DataFormats/HGCalReco/interface/HGCalSoARecHits.h modified in PR(s): Updated SoA View accessors from raw pointers to span #48377
  • File Geometry/HGCalMapping/test/testMappingModuleIndexer_cfg.py modified in PR(s): [HGCAL] Add energy loss to Calibration SoA #48187, Updates for new HGCal SiPM channel mapping #48571
  • File RecoLocalCalo/HGCalRecAlgos/interface/HGCalESProducerTools.h modified in PR(s): [HGCAL] Add energy loss to Calibration SoA #48187
  • File RecoLocalCalo/HGCalRecAlgos/plugins/alpaka/HGCalRecHitCalibrationAlgorithms.dev.cc modified in PR(s): [HGCAL] Add energy loss to Calibration SoA #48187
  • File RecoLocalCalo/HGCalRecAlgos/plugins/alpaka/HGCalRecHitCalibrationESProducer.cc modified in PR(s): [HGCAL] Add energy loss to Calibration SoA #48187, Updated SoA View accessors from raw pointers to span #48377
  • File RecoLocalCalo/HGCalRecAlgos/src/HGCalESProducerTools.cc modified in PR(s): [HGCAL] Add energy loss to Calibration SoA #48187
  • File RecoLocalCalo/HGCalRecAlgos/test/alpaka/HGCalRecHitESProducersTest.cc modified in PR(s): [HGCAL] Add energy loss to Calibration SoA #48187
  • File RecoLocalCalo/HGCalRecAlgos/test/testHGCalRecHitESProducers_cfg.py modified in PR(s): [HGCAL] Add energy loss to Calibration SoA #48187
  • File RecoLocalCalo/HGCalRecProducers/plugins/alpaka/HGCalLayerClustersAlgoWrapper.dev.cc modified in PR(s): Updated SoA View accessors from raw pointers to span #48377
  • File RecoLocalCalo/HGCalRecProducers/plugins/alpaka/HGCalLayerClustersSoAAlgoWrapper.dev.cc modified in PR(s): Updated SoA View accessors from raw pointers to span #48377

[cmsbuild]

A new Pull Request was created by @stahlleiton for master.

It involves the following packages:

• CondFormats/HGCalObjects (alca, upgrade, db)
• Configuration/PyReleaseValidation (upgrade, pdmv)
• DataFormats/HGCalRecHit (reconstruction, upgrade)
• DataFormats/HGCalReco (reconstruction, upgrade)
• Geometry/HGCalMapping (geometry, upgrade)
• RecoLocalCalo/Configuration (reconstruction)
• RecoLocalCalo/HGCalRecAlgos (reconstruction, upgrade)
• RecoLocalCalo/HGCalRecProducers (reconstruction, upgrade)

@AdrianoDee, @Dr15Jones, @Moanwar, @atpathak, @bsunanda, @civanch, @cmsbuild, @DickyChant, @francescobrivio, @jfernan2, @kpedro88, @makortel, @mandrenguyen, @mdhildreth, @miquork, @perrotta, @srimanob, @subirsarkar can you please review it and eventually sign? Thanks.
@JanChyczynski, @Martin-Grunewald, @PonIlya, @ReyerBand, @apsallid, @argiro, @bsunanda, @cseez, @denizsun, @edjtscott, @fabiocos, @felicepantaleo, @hatakeyamak, @lecriste, @lgray, @makortel, @martinamalberti, @missirol, @mmusich, @pfs, @rchatter, @rovere, @rsreds, @salimcerci, @sameasy, @seemasharmafnal, @sethzenz, @slomeo, @thomreis, @tocheng, @vandreev11, @wang0jin, @youyingli, @yuanchao this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[kpedro88]

This PR deletes the entire DataFormats/HGCalRecHit package. That seems like a mistake.

[kpedro88]

hold

[cmsbuild]

Pull request has been put on hold by @kpedro88
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

[IzaakWN]

This PR deletes the entire DataFormats/HGCalRecHit package. That seems like a mistake.

Hi @kpedro88, this was on purpose. As you can see in RecoLocalCalo/HGCalRecAlgos/interface/alpaka/HGCalRecHitCalibrationAlgorithms.h, it now adopts the same RecHit data format as is used by CLUE, so data from cassette testing and test beams this year will be able to be reconstructed.

[kpedro88]

unhold

[kpedro88]

please update the PR with a more thorough description of what was changed and why.

[kpedro88]

please test

[cmsbuild]

+1

Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ed9397/47711/summary.html
COMMIT: 8ff805e
CMSSW: CMSSW_15_1_X_2025-08-12-2300/el8_amd64_gcc12
Additional Tests: CUDA,ROCM
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48606/47711/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @leobeltra Updated README #48711
• @Dr15Jones Additional debugging for testFWCoreConcurrency #48726
• @slava77 LST: add trackToTrack comparisons for GPU vs CPU tracking #48508
• @makortel Always initialize PortableHostObject::product_ #48668
• @stahlleiton Fix relvals 138.1 and 138.2 #48697

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ed9397/47711/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ed9397/47711/git-merge-result

Comparison Summary

Summary:

• You potentially added 20 lines to the logs
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 4076152
• DQMHistoTests: Total failures: 21
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 4076111
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
• Checked 215 log files, 184 edm output root files, 50 DQM output files
• TriggerResults: no differences found

CUDA Comparison Summary

Summary:

• You potentially removed 4 lines from the logs
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 7
• DQMHistoTests: Total histograms compared: 53444
• DQMHistoTests: Total failures: 8197
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 45247
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
• Checked 24 log files, 30 edm output root files, 7 DQM output files
• TriggerResults: no differences found

ROCM Comparison Summary

Summary:

• You potentially removed 1 lines from the logs
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 7
• DQMHistoTests: Total histograms compared: 53444
• DQMHistoTests: Total failures: 7692
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 45752
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
• Checked 24 log files, 30 edm output root files, 7 DQM output files
• TriggerResults: no differences found

[jfernan2]

+1

[AdrianoDee]

+pdmv

[Moanwar]

+Upgrade

[civanch]

+1

[stahlleiton]

@cms-sw/alca-l2 and @cms-sw/db-l2 : It would be greatly appreciated if you could review this PR and eventually sign, thanks

[perrotta]

+1

• For the updated SOA structures in HGCalCalibParamSoA and HGCalMappingParameterSoA

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @rappoccio, @sextonkennedy, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)
Notice This PR was tested with additional Pull Request(s), please also merge them if necessary: cms-data/RecoLocalCalo-HGCalRecProducers#1

[mandrenguyen]

+1