Add WMCMSSWSubprocess and WMTiming metrics to WMArchvie document

[vkuznet]

Fixes #11657
Fixes the second part of #11660 and #11604

Status

ready

Description

Add WMCMSSWSubprocess and WMTiming metrics from FJR to WMArchive document

Is it backward compatible (if not, which system it affects?)

YES

Related PRs

Complement to #11656 and #11665

External dependencies / deployment changes

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: succeeded
  • 1 tests deleted
• Python3 Pylint check: failed
  • 1 warnings and errors that must be fixed
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded
  • 5 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/14433/artifact/artifacts/PullRequestReport.html

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: succeeded
  • 1 changes in unstable tests
• Python3 Pylint check: failed
  • 2 warnings and errors that must be fixed
  • 1 warnings
  • 65 comments to review
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded
  • 56 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/14434/artifact/artifacts/PullRequestReport.html

[vkuznet]

@amaltaro , this PR contains CMSSW subprocess timing metrics. Should I add to this PR CMSSW performance one or would you like to have separate PR. In later case I'll probably need to rebase my second PR since this changes may be merged. While if we'll make appropriate changes here (with proper default) then we can speed-up the process of preparing WMArchive docs for both subprocess timing and performace metrics together. Since we provide defaults we may not need to have them in FJR document per-se. But we'll be ready with both on WMArchive side.

[amaltaro]

Valentin, what is the other PR that you are talking about?

Looking at the changes you provided in this PR, I would say it should be fixing #11657 instead of #11660, as you mentioned in the description.

We have a total of 3 GH issues involving time/cpu metrics, if possible, let's try to end up with 3 PRs closing them (I know we have one extra already with the test/json updates).

[vkuznet]

Alan, I was referring to CMSSW performance metrics which also need propagation to WMArchive, see #11663, or IO/Site metrics #11575 and related WMArchive PRs to change the schema. Since in both cases it requires modification to WMCore/Serivce/WMArchive/DataMap to present them in final JSON that's why I asked should we add them in this PR or via separate PR.

And, thanks for the catch with description, I put wrong issue indeed, now it is fixed.

[amaltaro]

For the CMSSW I/O metrics, I would rather have it in a separate pull request fixing the relevant GH issue (#11538). I guess there is already a pull request for the FJR part.

IMO, you could make a 2nd commit in the same PR providing the relevant changes for ArchiveDataReporter/WMArchive.

[amaltaro]

@vkuznet Valentin, please find a couple of comments along the code.

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: failed
  • 2 new failures
• Python3 Pylint check: failed
  • 2 warnings and errors that must be fixed
  • 1 warnings
  • 65 comments to review
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded
  • 55 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/14435/artifact/artifacts/PullRequestReport.html

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: failed
  • 1 new failures
• Python3 Pylint check: failed
  • 2 warnings and errors that must be fixed
  • 1 warnings
  • 65 comments to review
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded
  • 51 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/14436/artifact/artifacts/PullRequestReport.html

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: failed
  • 2 new failures
  • 2 changes in unstable tests
• Python3 Pylint check: failed
  • 2 warnings and errors that must be fixed
  • 1 warnings
  • 65 comments to review
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded
  • 51 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/14437/artifact/artifacts/PullRequestReport.html

[vkuznet]

Alan, I added WMTiming metrics into this PR and it is ready to review. I also put in separate commit (7d9a152) changes to test json files to include WMTiming and WMCMSSWSubprocess metrics. This will allow to drop this PR #11659 in favor of this one.

[amaltaro]

Having a second look at the DataMap module, I don't really understand what the difference is between TOP_LEVEL_STEP_DEFAULT and STEP_DEFAULT. I would say the former goes at the top level of the json, while the latter goes inside each step sub-dictionary (cmsRun1, cmsRun2, and the unneeded logArch, stageOut).

Said that, the expected behavior for these new metrics is:

• WMTiming: should be provided only once per WMArchive document (at the job top level json report)
• WMCMSSWSubprocess: should be provided within each step sub-dictionary of the json report (ideally only for CMSSW type, thus cmsRun1, cmsRun2, etc). Not at the top level json report, given that it's a cmsRun/CMSSW property.

To avoid bad surprises and further PR with follow up, I would suggest to take one of the FWJR provided in this PR and pass it through the createArchiveDoc method:
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Services/WMArchive/DataMap.py#L397C5-L397C22

to ensure that the final WMArchive document is as expected.

Valentin, can you please do so and share the output file?

[amaltaro]

Nevermind! Let us get this in, run a test and then follow up with potential further fixes.