Skip to content

Conversation

@jennifer-richards
Copy link
Member

@jennifer-richards jennifer-richards commented Aug 12, 2025

This adds celery infrastructure and a framework for notifying datatracker of events. Attempts redelivery with exponential(ish) backoff out to once every 15 minutes with a suggestion of letting it retry delivery for a week at that terminal rate. After retries are exhausted, reports failure to admins via email that can be reviewed in mailpit for dev/staging.

This requires the datatracker API in ietf-tools/datatracker#9975. So far it only uses the metadata / Document-creation API. It does not yet upload contents but has a stub in place for that. This also does not connect the publication task to events or the UI. For now, it can be manually run from the shell.

Example using a random RfcToBe:

In an app shell:

>>> from datatracker.tasks import *
>>> notify_rfc_published_task.delay(rfctobe_id=RfcToBe.objects.last().pk)
<AsyncResult: ba375382-32f4-4794-b7db-763356aa76be>```

In the celery log:

celery-1  | [2025-08-13 20:40:39.591] (INFO) Task datatracker.tasks.notify_rfc_published_task[73b82d17-0c61-4ed8-9832-a9675f188aa0] received (celery.worker.strategy)
celery-1  | [2025-08-13 20:40:39.646] (INFO) Task datatracker.tasks.notify_rfc_published_task[73b82d17-0c61-4ed8-9832-a9675f188aa0] succeeded in 0.05237758300791029s: None (celery.app.trace)

If I turn set the max_retries to 2 and raise a RuntimeError in the task, this shows up in the celery log

[2025-11-21 19:45:25.119] (INFO) Task datatracker.tasks.notify_rfc_published_task[4a649274-aaa6-4ac0-8258-7df26c0b2a9a] received (celery.worker.strategy)

[2025-11-21 19:45:25.233] (INFO) Task datatracker.tasks.notify_rfc_published_task[4a649274-aaa6-4ac0-8258-7df26c0b2a9a] received (celery.worker.strategy)

[2025-11-21 19:45:25.236] (INFO) Task datatracker.tasks.notify_rfc_published_task[4a649274-aaa6-4ac0-8258-7df26c0b2a9a] retry: Retry in 3s: RuntimeError() (celery.app.trace)

[2025-11-21 19:45:28.228] (ERROR) Task datatracker.tasks.notify_rfc_published_task[4a649274-aaa6-4ac0-8258-7df26c0b2a9a] log: Emailing admins to report failure: datatracker.tasks.notify_rfc_published_task[4a649274-aaa6-4ac0-8258-7df26c0b2a9a] with args=[1] and kwargs={}. Giving up after 1 retries.

[2025-11-21 19:45:28.239] (ERROR) Task datatracker.tasks.notify_rfc_published_task[4a649274-aaa6-4ac0-8258-7df26c0b2a9a] raised unexpected: RuntimeError() (celery.app.trace)

and an email is sent to ADMINS with the task name/ID/args/kwargs and a traceback if an exception occurred.

This does not yet add k8s infrastructure for celery.

@jennifer-richards jennifer-richards marked this pull request as ready for review November 21, 2025 20:03
rpcapi.notify_rfc_published(
RfcPubRequest(
published=datetime.datetime.now(tz=datetime.UTC), # todo real pub date
draft_name=rfctobe.draft.name, # todo non-draft RFCs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would expect this to be None, which means no became_rfc relationship gets created

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok - I've adjusted this to handle rfctobe.draft is None. Purple overall will need some adjustment to deal with that I think (immediately adjacent issue is that purple doesn't actually know the title except through the draft). The updated code here should just work when we straighten that out, though.

More work is needed to actually support such an rfctobe in the system
@jennifer-richards jennifer-richards merged commit 3ac0d3c into ietf-tools:main Nov 24, 2025
2 checks passed
@jennifer-richards jennifer-richards deleted the rfc-pub-notification branch November 24, 2025 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants