Cut release 1.4.0

[smolkaj]

We discussed in the P4 API WG meeting that it would be a good time to cut a new release. Opening this as an umbrella bug to track that.

• label open PRs and issues with https://github.com/p4lang/p4runtime/labels/1.4.0 if they should block the release.
  • ideally include all PRs/issue with p4-language-compatibility An issue related to compatibility between P4_16 language spec and P4Runtime API spec label
• close all PRs/issue with label https://github.com/p4lang/p4runtime/labels/1.4.0
• make sure the changelog entry in the spec is up to date. #491
• Update the Bazel example to use the new release
  • Merge Update Bazel example to point to 1.4.0 #493
• All references in the P4Runtime spec should then be updated to point to the latest released version of the P4 language spec.
• Update references to the "latest" P4Runtime version in documentation.
• Add comments "{Changed,Deprecated} in 1.4.0" to all protobuf changes - PR 490
• Update p4.org spec page links (Andy)
• After publishing the release on Github, make sure that the PyPi package was uploaded successfully. This happens automatically from Github CI. There will be a CI error if it fails.
• Update the version string in the spec .mdk source document. After the release, the version string should be set to 1.5.0-dev on the main branch.
• Added - Change P4RT spec version to 1.4.1, 1.4.0 tag was "consumed"
• Added - Update the Bazel example to use the new release (final step to avoid tag cyclic dependency)

...

@antoninbas do you have any notes/suggestion on the process you have used in the past?

cc @chrispsommers @jonathan-dilorenzo @jafingerhut

EDIT: The cutoff date for changes going into 1.4.0 is September 13, the date of our next P4 API working group meeting.

[antoninbas]

This is the PR for the 1.3.0 release: #327

A few things to look out for:

• Update the version string in the spec .mdk source document. After the release, the version string should be set to 1.5.0-dev on the main branch.
• Make sure the changelog entry in the spec is up to date.
• Update the Bazel example to use the new release?
• Update references to the "latest" P4Runtime version in documentation.

After publishing the release on Github, make sure that the PyPi package was uploaded successfully. This happens automatically from Github CI. There will be a CI error if it fails.

I would also recommend making sure that there is full "compatibility" with the latest P4 version. This is a bit tedious, even though in general no change is required. This is also why the 1.4.0 release has been postponed so much (but we have had a few release candidate tags so that people could point to something more recent than 1.3.0). There are still quite a few open issues with the language compatibility label: p4-language-compatibility An issue related to compatibility between P4_16 language spec and P4Runtime API spec . All references in the P4Runtime spec should then be updated to point to the latest released version of the P4 spec.

[smolkaj]

Thanks for the guidance, that is very helpful.

[chrispsommers]

@smolkaj I propose (and volunteer) to create a version-1.4.0-rc branch to which we (anyone in the WG) can collaboratively and incrementally contribute PRs; then we can make PR from this working branch to main. This will allow us to divide and conquer. If this sounds good, I'll get the process started.

[smolkaj]

Just for my understanding -- why create an extra branch instead of working directly on main?

[chrispsommers]

Maybe I'm overthinking the amount of work to make a "release." Ideally there would be a single PR repreasenting the changes to main. In the past this came from one author (e.g. Antonin. If we have one author this time then we could review the one PR. On the other hand, assuming there were a few tasks we could parcel out, we could apply these to an interim branchm then prepare a final consolidated PR with all the changes, then merge that into main.

[smolkaj]

What would we do with PRs like #473? IIUC, those PRs would still go into main directly? And the version-1.4.0-rc branch would be used only for staging the various changes needed for actually cutting the release?

Ideally there would be a single PR repreasenting the changes to main.

I'm open to trying this. But admittedly, I'm a bit skeptical. Google wisdom argues exactly the opposite: it is better to break things up into small, incremental PRs that get submitted to a single main branch one at a time.
This is assuming that it is possible to break things up into small incremental PRs without leaving the main branch in a bad/inconsistent state, which is not allowed.

Perhaps a compromise could be to try to submit to main directly and only revert to using a release branch if submitting directly to main would bring us into a bad state?

[chrispsommers]

Any pending PRs should be done before, and independent of, the new release PR. If you look at #327 the changes are pretty minor, and they don't include any content like would be in pending item #473, they are just updates to the spec version etc. If we had a working branch for this ver 1.4.0 release it would have to sync to any changes made meanwhile to main. Changing all these version numbers throughout the spec to signify a new release should be done atomically.

[smolkaj]

Ok, agreed. Staging all the version changes on a shared release branch SGTM.

[chrispsommers]

Done: https://github.com/p4lang/p4runtime/tree/version-1.4.0-rc

[smolkaj]

Shall we set ourselves a hard deadline before the 2024 P4 Workshop, so we have the release ready in time for the workshop and the roadmap retreat? Wdyt @chrispsommers? cc @jonathan-dilorenzo

EDIT: The date for the workshop will likely be October 3rd.

[chrispsommers]

@smolkaj @jafingerhut With 1.5 weeks before the P4 workshop, do you think we have a chance of wrapping this up? I think the final TODO list is fairly small, the one wrinkle being we need to make the release 1.4.1 due to the immutable tag 1.4.0 which was already taken.

Any volunteers to help finish the list?

BTW the changelog looks good to me if @jafingerhut needs it for a WG summary.

[jafingerhut]

@chrispsommers I am happy to review any PRs you think are needed to get the release out. I am probably not as much help on the code aspects of the P4Runtime release, moreso the .mdk/HTML/PDF documentation parts. I can help with the updates on the p4.org specs page, in that I know who at LF to ask to update the specs page :-)

Regarding the item "All references in the P4Runtime spec should then be updated to point to the latest released version of the P4 spec.", if "P4 spec" means the language spec, it seems to me that pointing to the specs page, rather than to a particular version, requires the least ongoing maintenance and changes:

• https://p4.org/specs/

If you want a link directly to the latest released version of the P4 language spec, it seems questionable to me that they will cut a new release of the language spec before the workshop. If they do not, then links directly to v1.2.4 are here:

• https://p4.org/p4-spec/docs/P4-16-v1.2.4.html
• https://p4.org/p4-spec/docs/P4-16-v1.2.4.pdf

[chrispsommers]

@jafingerhut Thanks, I'll take you up on the offer to loing LF contact to update the link back to latest P4RT spec. Bonus opints if you want to update some .mdk per the list above, that'd be great.

@antoninbas @smolkaj any opinion on linking generically to https://p4.org/specs/ vs the versioned links for v1.2.4?

@smolkaj do you want to do a PR to update Bazel example to 1.4.1?

[antoninbas]

@antoninbas @smolkaj any opinion on linking generically to https://p4.org/specs/ vs the versioned links for v1.2.4?

I would argue that it's better to reference an actual version of the language, given that language changes can affect P4Runtime. But maybe I am missing some context?

[chrispsommers]

@antoninbas I think you've made a good case to use links to specific versions, thanks.

[smolkaj]

Reminder to not tag the release until the very end, see discussion: #499

[smolkaj]

I searched for "https://p4.org/" in P4Runtime-Spec.mdk and finding 0 hits, so I'm not sure what

All references in the P4Runtime spec should then be updated to point to the latest released version of the P4 spec.

is referring to. Could you clarify, @antoninbas?

[antoninbas]

@smolkaj all the links should be in references.bib

[smolkaj]

Thanks for the clarification

[chrispsommers]

Hi @smolkaj , I don't think this can be close until we "check a few more boxes."

[smolkaj]

Yup, thanks, this happened automatically, by accident.

[chrispsommers]

Yup, thanks, this happened automatically, by accident.

That phrase should be canonized.

[smolkaj]

I went ahead and cut the release: https://github.com/p4lang/p4runtime/releases/tag/v1.4.1

[chrispsommers]

Nice, thanks!

[smolkaj]

I don't see the release showing up on PyPi quite yet: https://pypi.org/search/?q=p4runtime&o=-created
I suspect it will take some time.

[smolkaj]

Looks like the CI scripts for publishing to PyPi failed: http://screen/6f5CynU266BXUXg Need to take a closer look.

[antoninbas]

@smolkaj it's saying I don't have a verified email address. Let me see if I can fix that and trigger the workflow again.

[smolkaj]

@antoninbas, could you take a look?

ERROR    HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/        
         User 'AntoninBas' does not have a verified primary email address.      
         Please add a verified primary email before attempting to upload to     
         PyPI. See https://pypi.org/help/#verified-email for more information.  

[smolkaj]

You are operating at the speed of light! Thanks :)

[antoninbas]

@smolkaj I verified my account in PyPi. Unfortunately for some reason I don't have write access to this Github repo anymore (p4lang/p4runtime), so I cannot trigger the job again. Which is weird because I still have my permissions for other p4lang repos (cc @fruffy). You should be able to trigger the failed job from the UI.

[smolkaj]

One more:

ERROR    HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/        
         User 'AntoninBas' does not have two-factor authentication enabled.     
         Please enable two-factor authentication before attempting to upload to 
         PyPI. See https://pypi.org/help/#two-factor-authentication for more    
         information. 

[smolkaj]

I also added you back to the maintainer list.

[smolkaj]

Alright, great success! Thanks, @antoninbas

[smolkaj]

@jafingerhut, the final step would be to update the links on p4.org, judging from the TODO list at the top. Looks like you volunteered for that one, IIUC?

[antoninbas]

@smolkaj @chrispsommers let me know if you want to be added to the maintainers for the p4runtime project in PyPI. You will need to create a PyPI account first. Note that the Github workflow will keep using my account unless the Github secret is updated with a new token.

[jafingerhut]

@jafingerhut, the final step would be to update the links on p4.org, judging from the TODO list at the top. Looks like you volunteered for that one, IIUC?

I have successfully passed the buck via email, CC'ing you, to the person who can actually do it :-). Feel free to contact Denise for any p4.org web site changes, whether spec-related or not, e.g. new blog posts.

[smolkaj]

@smolkaj @chrispsommers let me know if you want to be added to the maintainers for the p4runtime project in PyPI. You will need to create a PyPI account first. Note that the Github workflow will keep using my account unless the Github secret is updated with a new token.

I created an account, user name is smolkaj. THanks!

[chrispsommers]

Please add me as well, chrispsommers. We're so appreciative that you're still helping us assume the many duties for this WG!

[smolkaj]

@chrispsommers pointed out there is a final step remaining: "Update the version string in the spec .mdk source document. After the release, the version string should be set to 1.5.0-dev on the main branch."

[chrispsommers]

Fixed by #507 and #509