PerfTools/AllocMonitor: add c++ types for EDModules and ESModules to json output of scripts/edmModuleAllocMonitorAnalyze.py #48874

gartung · 2025-09-08T16:10:17Z

closes cms-sw/framework-team#1550

cmsbuild · 2025-09-08T16:10:45Z

cms-bot internal usage

cmsbuild · 2025-09-08T16:12:01Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48874/46029

cmsbuild · 2025-09-08T16:12:23Z

A new Pull Request was created by @gartung for master.

It involves the following packages:

PerfTools/AllocMonitor (core)

@Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please review it and eventually sign? Thanks.
@felicepantaleo, @makortel this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

PerfTools/AllocMonitor/scripts/edmModuleAllocMonitorAnalyze.py

cmsbuild · 2025-09-08T17:51:56Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48874/46030

cmsbuild · 2025-09-08T17:52:19Z

Pull request #48874 was updated. @Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please check and sign again.

cmsbuild · 2025-09-08T18:08:56Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48874/46031

cmsbuild · 2025-09-08T18:09:14Z

Pull request #48874 was updated. @Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please check and sign again.

PerfTools/AllocMonitor/scripts/edmModuleAllocMonitorAnalyze.py

gartung · 2025-09-10T15:09:19Z

@cmsbuild, please test

makortel · 2025-09-11T16:17:37Z

@cmsbuild, please test

cmsbuild · 2025-09-11T17:10:31Z

-1

Failed Tests: RelVals
Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-56f988/48070/summary.html
COMMIT: 3087361
CMSSW: CMSSW_16_0_X_2025-09-11-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48874/48070/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

29634.75

----- Begin Fatal Exception 11-Sep-2025 18:51:14 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_DIGI_L1TrackTrigger_L1_L1P2GT_DIGI2RAW_HLT_VALIDATION.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file L1Trigger/Phase2L1ParticleFlow/data/met/l1met_ptphi2pxpy_poly2_v1.json anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48874/48070/CMSSW_16_0_X_2025-09-11-1100/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48874/48070/CMSSW_16_0_X_2025-09-11-1100/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48874/48070/CMSSW_16_0_X_2025-09-11-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_16_0_X_2025-09-11-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_16_0_X_2025-09-11-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_16_0_X_2025-09-11-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/29634.75_TTbar_14TeV+Run4D110_HLT75e33Timing


At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Types.py(1050): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Mixins.py(398): insertContentsInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Mixins.py(533): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Modules.py(160): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Config.py(1170): _insertManyInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Config.py(1394): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

29834.999

----- Begin Fatal Exception 11-Sep-2025 18:55:45 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step3_DIGI_DATAMIX_L1TrackTrigger_L1_L1P2GT_DIGI2RAW_HLT.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file L1Trigger/Phase2L1ParticleFlow/data/met/l1met_ptphi2pxpy_poly2_v1.json anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48874/48070/CMSSW_16_0_X_2025-09-11-1100/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48874/48070/CMSSW_16_0_X_2025-09-11-1100/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48874/48070/CMSSW_16_0_X_2025-09-11-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_16_0_X_2025-09-11-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_16_0_X_2025-09-11-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_16_0_X_2025-09-11-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/29834.999_TTbar_14TeV+Run4D110PU_PMXS1S2PR


At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Types.py(1050): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Mixins.py(398): insertContentsInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Mixins.py(533): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Modules.py(160): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Config.py(1170): _insertManyInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Config.py(1394): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

24834.911

----- Begin Fatal Exception 11-Sep-2025 18:55:04 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_DIGI_L1TrackTrigger_L1_L1P2GT_DIGI2RAW_HLT.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file L1Trigger/Phase2L1ParticleFlow/data/met/l1met_ptphi2pxpy_poly2_v1.json anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48874/48070/CMSSW_16_0_X_2025-09-11-1100/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48874/48070/CMSSW_16_0_X_2025-09-11-1100/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48874/48070/CMSSW_16_0_X_2025-09-11-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_16_0_X_2025-09-11-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_16_0_X_2025-09-11-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_16_0_X_2025-09-11-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/24834.911_TTbar_14TeV+Run4D98_DD4hep


At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Types.py(1050): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Mixins.py(398): insertContentsInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Mixins.py(533): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Modules.py(160): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Config.py(1170): _insertManyInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02906/el8_amd64_gcc12/cms/cmssw/CMSSW_16_0_X_2025-09-10-1100/src/FWCore/ParameterSet/python/Config.py(1394): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

gartung · 2025-09-11T17:24:38Z

Relval errors appear to be unrelated to change.

makortel · 2025-09-11T17:34:31Z

Yeah, those are related to #48897

makortel · 2025-09-11T17:34:53Z

+core

cmsbuild · 2025-09-11T17:35:13Z

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @mandrenguyen, @ftenchini (and backports should be raised in the release meeting by the corresponding L2)

ftenchini · 2025-09-12T07:27:11Z

@cmsbuild please test

cmsbuild · 2025-09-12T14:07:18Z

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-56f988/48077/summary.html
COMMIT: 3087361
CMSSW: CMSSW_16_0_X_2025-09-11-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48874/48077/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 1 lines from the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 4113751
DQMHistoTests: Total failures: 20
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 4113711
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 215 log files, 184 edm output root files, 50 DQM output files
TriggerResults: no differences found

makortel · 2025-09-12T14:12:39Z

Comparison differences are related to #47071

makortel · 2025-09-12T14:12:42Z

+core

ftenchini · 2025-09-12T16:19:33Z

+1

gartung added 3 commits September 8, 2025 11:05

Initial changes to add cpptype to parsed info

ecb1cee

Add cpptypes dictionary to json output

bffc832

Fix bugs found with testing

6cc7c14

cmsbuild added this to the CMSSW_15_1_X milestone Sep 8, 2025

cmsbuild added core-pending pending-signatures tests-pending orp-pending code-checks-pending labels Sep 8, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 8, 2025

makortel reviewed Sep 8, 2025

View reviewed changes

Fix bug where null reported for cpptype

5528849

cmsbuild added code-checks-pending and removed code-checks-approved labels Sep 8, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 8, 2025

Revert changes made to accomodate previous changes

7bd6616

cmsbuild added code-checks-pending and removed code-checks-approved labels Sep 8, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 8, 2025

Dr15Jones reviewed Sep 8, 2025

View reviewed changes

PerfTools/AllocMonitor/scripts/edmModuleAllocMonitorAnalyze.py Outdated Show resolved Hide resolved

cmsbuild added tests-pending and removed tests-started labels Sep 10, 2025

cmsbuild added tests-started and removed tests-pending labels Sep 11, 2025

cmsbuild added tests-rejected and removed tests-started labels Sep 11, 2025

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Sep 11, 2025

cmsbuild added tests-started and removed tests-rejected labels Sep 12, 2025

cmsbuild added tests-approved and removed tests-started labels Sep 12, 2025

cmsbuild added orp-approved and removed orp-pending labels Sep 12, 2025

cmsbuild merged commit abf3d78 into cms-sw:master Sep 12, 2025
10 checks passed

gartung deleted the gartung-PerfTools-AllocMonitor branch September 24, 2025 16:43

PerfTools/AllocMonitor: add c++ types for EDModules and ESModules to json output of scripts/edmModuleAllocMonitorAnalyze.py #48874

PerfTools/AllocMonitor: add c++ types for EDModules and ESModules to json output of scripts/edmModuleAllocMonitorAnalyze.py #48874

Uh oh!

Conversation

gartung commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmsbuild commented Sep 8, 2025

Uh oh!

cmsbuild commented Sep 8, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cmsbuild commented Sep 8, 2025

Uh oh!

cmsbuild commented Sep 8, 2025

Uh oh!

cmsbuild commented Sep 8, 2025

Uh oh!

cmsbuild commented Sep 8, 2025

Uh oh!

Uh oh!

gartung commented Sep 10, 2025

Uh oh!

makortel commented Sep 11, 2025

Uh oh!

cmsbuild commented Sep 11, 2025

RelVals

Uh oh!

gartung commented Sep 11, 2025

Uh oh!

makortel commented Sep 11, 2025

Uh oh!

makortel commented Sep 11, 2025

Uh oh!

cmsbuild commented Sep 11, 2025

Uh oh!

ftenchini commented Sep 12, 2025

Uh oh!

cmsbuild commented Sep 12, 2025

Comparison Summary

Uh oh!

makortel commented Sep 12, 2025

Uh oh!

makortel commented Sep 12, 2025

Uh oh!

ftenchini commented Sep 12, 2025

Uh oh!

Uh oh!

Uh oh!

gartung commented Sep 8, 2025 •

edited

Loading

cmsbuild commented Sep 8, 2025 •

edited

Loading