Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(celery): stop closing prerun_span too soon to account for Celery chains scenario [backport 2.18] #11806

Merged
merged 3 commits into from
Dec 19, 2024

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Dec 19, 2024

Backport e8aab65 from #11498 to 2.18.

We've made a few changes to handle celery context recently, including: #10676

In particular the goal of #10676 was to handle a scenario where a long running task may run into an exception, preventing it from closing.

Unfortunately, this scenario did not account for cases where tasks are chained and may not close until later.

See: #11479 and #11624

With this PR, the sample app in #11479 would attach the celery specific span back to the root span.

I also need to add tests for the chains scenario.

Related to AIDM-494

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

…chains scenario (#11498)

We've made a few changes to handle celery context recently, including:
#10676

In particular the goal of
#10676 was to handle a
scenario where a long running task may run into an exception, preventing
it from closing.

Unfortunately, this scenario did not account for cases where tasks are
chained and may not close until later.

See: #11479 and
#11624

With this PR, the sample app in
#11479 would attach the
celery specific span back to the root span.

I also need to add tests for the chains scenario.

Related to AIDM-494

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

(cherry picked from commit e8aab65)
@pr-commenter
Copy link

pr-commenter bot commented Dec 19, 2024

Benchmarks

Benchmark execution time: 2024-12-19 23:39:49

Comparing candidate commit 3136928 in PR branch backport-11498-to-2.18 with baseline commit 45c64c6 in branch 2.18.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 393 metrics, 2 unstable metrics.

scenario:iast_aspects-ospathdirname_aspect

  • 🟩 execution_time [-376.983ns; -300.929ns] or [-9.380%; -7.487%]

@quinna-h quinna-h closed this Dec 19, 2024
@quinna-h quinna-h reopened this Dec 19, 2024
Copy link
Contributor Author

CODEOWNERS have been resolved as:

releasenotes/notes/fix-celery-closed-spans-34ff43868c1e33b8.yaml        @DataDog/apm-python
tests/contrib/celery/run_tasks.py                                       @DataDog/apm-core-python @DataDog/apm-idm-python
tests/contrib/celery/tasks.py                                           @DataDog/apm-core-python @DataDog/apm-idm-python
tests/contrib/celery/test_chained_task.py                               @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/celery/app.py                                  @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/celery/signals.py                              @DataDog/apm-core-python @DataDog/apm-idm-python

@wantsui wantsui enabled auto-merge (squash) December 19, 2024 22:02
@wantsui wantsui disabled auto-merge December 19, 2024 22:44
@wantsui wantsui enabled auto-merge (squash) December 19, 2024 22:44
@wantsui wantsui merged commit 0d6c79b into 2.18 Dec 19, 2024
560 checks passed
@wantsui wantsui deleted the backport-11498-to-2.18 branch December 19, 2024 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants