Publish pipeline-steps-doc-generator and backend-extension-indexer artifacts to some kind of storage

[timja]

Service(s)

infra.ci.jenkins.io

Summary

https://github.com/jenkins-infra/jenkins.io/blob/9addc32ac8d366e02a9ec49e07e81ab527547cf4/scripts/fetch-external-resources#L36
and
https://github.com/jenkins-infra/jenkins.io/blob/9addc32ac8d366e02a9ec49e07e81ab527547cf4/scripts/fetch-external-resources#L66

cause site publishing to be reliant on ci.jenkins.io being up.

Which makes ci.jenkins.io a critical path for publishing security advisories and for contributors developing local site builds.

We should decouple these by publishing to some kind of storage.
Either reports.jenkins.io or an Azure storage, these need to be published publicly somewhere that any contributors machine can download from

see also #3086 (comment)

Reproduction steps

No response

[dduportal]

@timja @daniel-beck do you agree with the following proposal?

1. Migrating the job from ci.jenkins.io to infra.ci.jenkins.io, with the github checks enabled (e.g. build log shown in GitHub publicly)
2. Adding a deployment to reports.jenkins.io (like the other reports) with the pipeline method publishReports()
3. Update the github.com/jenkins-infra/jenkins.io's shell script fetch-external-resources.sh to use reports.jenkins.io instead of ci.j
4. Once validated, then remove the job from ci.jenkins.io

[timja]

sounds good

[daniel-beck]

do you agree with the following proposal?

Yes, absolutely, this has been terrible for a long time. We didn't have reports.j.io when this was set up, so this was a pragmatic way to do this.

Which makes ci.jenkins.io a critical path for publishing security advisories

This has bitten us once or twice; rather than ignoring some bullshit error on startup we have to urgently figure out what was wrong. Every time I complain about the 8 minute startup time of ci.j.io, this dependency from jenkins.io builds is why 😉

[dduportal]

• chore(helpdesk-3087) switch URL of the pipeline generated steps to reports.jenkins.io jenkins.io#5667 for the switch of jenkins.io.

next step will be to remove the 2 jobs from ci.jenkins.io.

[dduportal]

Both pipeline marked as archived in ci.jenkins.io (removed from the "include list" trait of the GH org scanning named Infra).

[daniel-beck]

Both pipeline marked as archived in ci.jenkins.io

To clarify, there is no value in having a public CI there?

[timja]

They definitely should have public CI

[dduportal]

ack, reopening to enable again public CI then

[smerle33]

in the process of merging the pipeline files from ci and infra.ci (to provide a public output), I will try to take the process for : pipeline-steps-doc-generator and leave backend-extension-indexer to @dduportal

[dduportal]

Just added back the 2 repositories to ci.jenkins.io in https://ci.jenkins.io/job/Infra/ (updated the GH organization scanning job). Scanning in progress, that should add build checks for jenkins-infra/backend-extension-indexer#46 and jenkins-infra/pipeline-steps-doc-generator#241

[dduportal]

• The 2 pipelines had been unified succesfully (thanks @smerle33 !)
• 2 cleanup tasks left before closing:
  • Reconfigure infra.ci jobs to use Jenkinsfile - feat(infra.ci) use 'Jenkinsfile' filename pattern for jobs backend-extension-indexer and pipeline-steps-doc-generator kubernetes-management#3281
  • Remove the Jenkinsfile.infra.ci.jenkins.io symlinks from the 2 repos
    • jenkins-infra/backend-extension-indexer@8be4c8d
    • jenkins-infra/pipeline-steps-doc-generator@8a39776