-
Notifications
You must be signed in to change notification settings - Fork 16
feat: datatracker event notification via celery #423
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: datatracker event notification via celery #423
Conversation
Uses prototype API call from purple-publish-api branch in my fork of datatracker feat/rpc-api
# Conflicts: # docker-compose.yml # purple/settings/base.py # purple/settings/development.py # requirements.txt
Does not upload contents yet
datatracker/utils/publication.py
Outdated
| rpcapi.notify_rfc_published( | ||
| RfcPubRequest( | ||
| published=datetime.datetime.now(tz=datetime.UTC), # todo real pub date | ||
| draft_name=rfctobe.draft.name, # todo non-draft RFCs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would expect this to be None, which means no became_rfc relationship gets created
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok - I've adjusted this to handle rfctobe.draft is None. Purple overall will need some adjustment to deal with that I think (immediately adjacent issue is that purple doesn't actually know the title except through the draft). The updated code here should just work when we straighten that out, though.
More work is needed to actually support such an rfctobe in the system
This adds celery infrastructure and a framework for notifying datatracker of events. Attempts redelivery with exponential(ish) backoff out to once every 15 minutes with a suggestion of letting it retry delivery for a week at that terminal rate. After retries are exhausted, reports failure to admins via email that can be reviewed in mailpit for dev/staging.
This requires the datatracker API in ietf-tools/datatracker#9975. So far it only uses the metadata / Document-creation API. It does not yet upload contents but has a stub in place for that. This also does not connect the publication task to events or the UI. For now, it can be manually run from the shell.
Example using a random RfcToBe:
In an app shell:
In the celery log:
If I turn set the max_retries to 2 and raise a RuntimeError in the task, this shows up in the celery log
and an email is sent to ADMINS with the task name/ID/args/kwargs and a traceback if an exception occurred.
This does not yet add k8s infrastructure for celery.