Add Process name, reduced configuration ID, and ParameterSet ID to the JobReport #47548

makortel · 2025-03-10T16:40:11Z

PR description:

In the context of dmwm/CRABServer#8953 this PR adds

<Process>
  <Name>TEST</Name>
  <ReducedConfigurationID> 783c5ab60bfe18ac97262801adef9de4 </ReducedConfigurationID>
  <ParameterSetID> 1f94f8e1ac0ae9d7dade1c99cc5862b3 </ParameterSetID>
</Process>

block to the framework job report. This information allows CRAB to stop relying on edmProvDump, after which we'll have more freedom to improve the edmProvDump output.

Resolves cms-sw/framework-team#1273

PR validation:

Framework unit tests pass

cmsbuild · 2025-03-10T16:40:39Z

cms-bot internal usage

cmsbuild · 2025-03-10T16:43:37Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47548/44021

There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Framework/src/EventProcessor.cc modified in PR(s): Access PathsAndConsumesOfModules from new signal #47467, Compartmentalize how ProductRegistry is filled #47533
- File FWCore/MessageLogger/src/JobReport.cc modified in PR(s): Removed -Wconversion warnings in part of framework #47339

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47548/44021/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47548/44021/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2025-03-10T20:49:13Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47548/44024

There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Framework/src/EventProcessor.cc modified in PR(s): Access PathsAndConsumesOfModules from new signal #47467, Compartmentalize how ProductRegistry is filled #47533
- File FWCore/MessageLogger/src/JobReport.cc modified in PR(s): Removed -Wconversion warnings in part of framework #47339

cmsbuild · 2025-03-10T20:49:37Z

A new Pull Request was created by @makortel for master.

It involves the following packages:

FWCore/Framework (core)
FWCore/MessageLogger (core)
FWCore/MessageService (core)
IOPool/Common (core)

@Dr15Jones, @cmsbuild, @makortel, @smuzaffar can you please review it and eventually sign? Thanks.
@missirol, @wddgit this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

makortel · 2025-03-10T20:51:31Z

@cmsbuild, please test

makortel · 2025-03-10T20:51:41Z

@Dr15Jones Could you take a look?

Dr15Jones

Any thoughts about how you could test that the proper IDs are being output? Maybe a module that writes them to output and then the test looks at that output and compares to the FJR part?

FWCore/MessageLogger/interface/JobReport.h

cmsbuild · 2025-03-10T22:55:14Z

+1

Size: This PR adds an extra 68KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd76d5/44898/summary.html
COMMIT: 5468ed5
CMSSW: CMSSW_15_1_X_2025-03-10-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/47548/44898/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 1 lines to the logs
Reco comparison results: 5 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3920300
DQMHistoTests: Total failures: 64
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3920216
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 214 log files, 184 edm output root files, 49 DQM output files
TriggerResults: no differences found

cmsbuild · 2025-03-11T23:03:35Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47548/44051

There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Framework/src/EventProcessor.cc modified in PR(s): Access PathsAndConsumesOfModules from new signal #47467, Compartmentalize how ProductRegistry is filled #47533
- File FWCore/MessageLogger/src/JobReport.cc modified in PR(s): Removed -Wconversion warnings in part of framework #47339

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47548/44051/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47548/44051/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

makortel · 2025-03-11T23:03:57Z

Any thoughts about how you could test that the proper IDs are being output? Maybe a module that writes them to output and then the test looks at that output and compares to the FJR part?

I added a test using a module that prints the 3 values and a script that compares the job output to the XML file.

…e JobReport Main motivation is to give CRAB a more stable way to retrieve the ParameterSetID than edmProvDump.

makortel · 2025-03-11T23:05:38Z

@cmsbuild, please test

cmsbuild · 2025-03-11T23:07:13Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47548/44052

There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Framework/src/EventProcessor.cc modified in PR(s): Access PathsAndConsumesOfModules from new signal #47467, Compartmentalize how ProductRegistry is filled #47533
- File FWCore/MessageLogger/src/JobReport.cc modified in PR(s): Removed -Wconversion warnings in part of framework #47339

cmsbuild · 2025-03-11T23:07:40Z

Pull request #47548 was updated. @Dr15Jones, @makortel, @smuzaffar can you please check and sign again.

cmsbuild · 2025-03-12T01:11:24Z

+1

Size: This PR adds an extra 36KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd76d5/44930/summary.html
COMMIT: d78cbce
CMSSW: CMSSW_15_1_X_2025-03-11-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/47548/44930/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 2 lines to the logs
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3920300
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3920274
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 214 log files, 184 edm output root files, 49 DQM output files
TriggerResults: no differences found

makortel · 2025-03-12T19:08:12Z

+core

cmsbuild · 2025-03-12T19:08:38Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @rappoccio, @antoniovilela, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2025-03-14T06:18:33Z

+1

cmsbuild added this to the CMSSW_15_1_X milestone Mar 10, 2025

cmsbuild added core-pending pending-signatures tests-pending orp-pending code-checks-pending labels Mar 10, 2025

makortel mentioned this pull request Mar 10, 2025

Adding PsetHash from Edmprovdump fails with CMSSW_15_1 dmwm/CRABServer#8953

Open

cmsbuild added code-checks-rejected and removed code-checks-pending labels Mar 10, 2025

makortel force-pushed the fwjrPSetID branch from 3082288 to 5468ed5 Compare March 10, 2025 20:47

cmsbuild added code-checks-pending and removed code-checks-rejected labels Mar 10, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 10, 2025

cmsbuild added tests-started and removed tests-pending labels Mar 10, 2025

Dr15Jones suggested changes Mar 10, 2025

View reviewed changes

FWCore/MessageLogger/interface/JobReport.h Outdated Show resolved Hide resolved

cmsbuild added tests-approved and removed tests-started labels Mar 10, 2025

makortel force-pushed the fwjrPSetID branch from 5468ed5 to fec77f0 Compare March 11, 2025 23:02

cmsbuild added tests-pending and removed tests-approved code-checks-approved labels Mar 11, 2025

cmsbuild added code-checks-rejected and removed code-checks-pending labels Mar 11, 2025

Add Process name, reduced configuration ID, and ParameterSet ID to th…

d78cbce

…e JobReport Main motivation is to give CRAB a more stable way to retrieve the ParameterSetID than edmProvDump.

makortel force-pushed the fwjrPSetID branch from fec77f0 to d78cbce Compare March 11, 2025 23:05

cmsbuild added code-checks-pending and removed code-checks-rejected labels Mar 11, 2025

cmsbuild added tests-started and removed tests-pending labels Mar 11, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 11, 2025

cmsbuild added tests-approved and removed tests-started labels Mar 12, 2025

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Mar 12, 2025

cmsbuild mentioned this pull request Mar 12, 2025

Create new main ProductRegistry if input products change #47584

Merged

cmsbuild added orp-approved and removed orp-pending labels Mar 14, 2025

cmsbuild merged commit adb40c1 into cms-sw:master Mar 14, 2025
11 checks passed

makortel deleted the fwjrPSetID branch March 14, 2025 13:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Process name, reduced configuration ID, and ParameterSet ID to the JobReport #47548

Add Process name, reduced configuration ID, and ParameterSet ID to the JobReport #47548

makortel commented Mar 10, 2025

cmsbuild commented Mar 10, 2025 •

edited

Loading

cmsbuild commented Mar 10, 2025

cmsbuild commented Mar 10, 2025

cmsbuild commented Mar 10, 2025

makortel commented Mar 10, 2025

makortel commented Mar 10, 2025

Dr15Jones left a comment

cmsbuild commented Mar 10, 2025

cmsbuild commented Mar 11, 2025

makortel commented Mar 11, 2025

makortel commented Mar 11, 2025

cmsbuild commented Mar 11, 2025

cmsbuild commented Mar 11, 2025

cmsbuild commented Mar 12, 2025

makortel commented Mar 12, 2025

cmsbuild commented Mar 12, 2025

mandrenguyen commented Mar 14, 2025

Add Process name, reduced configuration ID, and ParameterSet ID to the JobReport #47548

Add Process name, reduced configuration ID, and ParameterSet ID to the JobReport #47548

Conversation

makortel commented Mar 10, 2025

PR description:

PR validation:

cmsbuild commented Mar 10, 2025 • edited Loading

cmsbuild commented Mar 10, 2025

cmsbuild commented Mar 10, 2025

cmsbuild commented Mar 10, 2025

makortel commented Mar 10, 2025

makortel commented Mar 10, 2025

Dr15Jones left a comment

Choose a reason for hiding this comment

cmsbuild commented Mar 10, 2025

Comparison Summary

cmsbuild commented Mar 11, 2025

makortel commented Mar 11, 2025

makortel commented Mar 11, 2025

cmsbuild commented Mar 11, 2025

cmsbuild commented Mar 11, 2025

cmsbuild commented Mar 12, 2025

Comparison Summary

makortel commented Mar 12, 2025

cmsbuild commented Mar 12, 2025

mandrenguyen commented Mar 14, 2025

cmsbuild commented Mar 10, 2025 •

edited

Loading