Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEBUG-2334 remaining dynamic instrumentation code #4098

Open
wants to merge 39 commits into
base: master
Choose a base branch
from
Open

Conversation

p-datadog
Copy link
Contributor

@p-datadog p-datadog commented Nov 11, 2024

What does this PR do?

  • Adds DI Remote and integrates DI into remote config processing
  • Integration test starting from remote config parsing, through instrumentation to snapshot sending to backend
  • DI is now enabled by default
  • DI code tracking is now enabled by default (script_compiled trace point) if DD_DYNAMIC_INSTRUMENTATION_ENABLED is set in environment

Motivation:

Initial DI implementation in Ruby

Change log entry
Dynamic instrumentation is now available in Ruby. Currently only log probes are implemented, but they can be set on both methods and lines.

Additional Notes:

Replaces #4063

How to test the change?
Additional unit and integration tests are included + system tests in DataDog/system-tests#3516

@p-datadog p-datadog requested a review from a team as a code owner November 11, 2024 13:16
@github-actions github-actions bot added the core Involves Datadog core libraries label Nov 11, 2024
@p-datadog p-datadog marked this pull request as draft November 11, 2024 13:16
lib/datadog/di/serializer.rb Outdated Show resolved Hide resolved
spec/datadog/di/serializer_helper.rb Outdated Show resolved Hide resolved
spec/datadog/di/integration/instrumentation_spec.rb Outdated Show resolved Hide resolved
spec/datadog/di/serializer_spec.rb Outdated Show resolved Hide resolved
spec/datadog/di/serializer_helper.rb Outdated Show resolved Hide resolved
lib/datadog/di/probe_manager.rb Outdated Show resolved Hide resolved
lib/datadog/di/serializer.rb Outdated Show resolved Hide resolved
lib/datadog/di/code_tracker.rb Outdated Show resolved Hide resolved
spec/datadog/di/serializer_spec.rb Outdated Show resolved Hide resolved
@pr-commenter
Copy link

pr-commenter bot commented Nov 11, 2024

Benchmarks

Benchmark execution time: 2024-11-25 00:05:16

Comparing candidate commit 4683a48 in PR branch di-rest-2 with baseline commit e703220 in branch master.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 29 metrics, 2 unstable metrics.

scenario:profiler - sample timeline=false

  • 🟥 throughput [-0.516op/s; -0.506op/s] or [-7.876%; -7.729%]

scenario:tracing - Propagation - Trace Context

  • 🟥 throughput [-3262.367op/s; -3155.087op/s] or [-8.614%; -8.331%]

@p-datadog p-datadog force-pushed the di-rest-2 branch 2 times, most recently from b7e93a0 to 288d1d8 Compare November 12, 2024 15:57
@p-datadog p-datadog force-pushed the di-rest-2 branch 5 times, most recently from 79e1ff8 to 7d4aedf Compare November 13, 2024 02:52
lib/datadog/di/instrumenter.rb Outdated Show resolved Hide resolved
lib/datadog/di/instrumenter.rb Outdated Show resolved Hide resolved
@p-datadog p-datadog force-pushed the di-rest-2 branch 5 times, most recently from 66546cd to 52034e2 Compare November 18, 2024 15:18
Maximum capture attribute count can be specified in the probe,
implement support for it in DI.
Comment on lines 70 to 84
if RUBY_VERSION >= '2.6'
begin
# Activate code tracking by default because line trace points will not work
# without it.
Datadog::DI.activate_tracking!
rescue => exc
if defined?(Datadog.logger)
Datadog.logger.warn("Failed to activate code tracking for DI: #{exc.class}: #{exc}")
else
# We do not have Datadog logger potentially because DI code tracker is
# being loaded early in application boot process and the rest of datadog
# wasn't loaded yet. Output to standard error.
warn("Failed to activate code tracking for DI: #{exc.class}: #{exc}")
end
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nit:

Suggested change
if RUBY_VERSION >= '2.6'
begin
# Activate code tracking by default because line trace points will not work
# without it.
Datadog::DI.activate_tracking!
rescue => exc
if defined?(Datadog.logger)
Datadog.logger.warn("Failed to activate code tracking for DI: #{exc.class}: #{exc}")
else
# We do not have Datadog logger potentially because DI code tracker is
# being loaded early in application boot process and the rest of datadog
# wasn't loaded yet. Output to standard error.
warn("Failed to activate code tracking for DI: #{exc.class}: #{exc}")
end
end
return if RUBY_VERSION >= '2.6'
# Activate code tracking by default because line trace points will not work
# without it.
Datadog::DI.activate_tracking!
rescue => exc
if defined?(Datadog.logger)
Datadog.logger.warn("Failed to activate code tracking for DI: #{exc.class}: #{exc}")
else
# We do not have Datadog logger potentially because DI code tracker is
# being loaded early in application boot process and the rest of datadog
# wasn't loaded yet. Output to standard error.
warn("Failed to activate code tracking for DI: #{exc.class}: #{exc}")
end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you meant to reverse the sign on comparison which I did.

end
end
=end
Datadog::DI.activate_tracking
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is unusual to see method calls done at the root level, when the file is loaded.
Can you add a short comment here explaining why it's needed at this level? I understand that it's an early activation needed for DI, and that there's a more expansive explanation of DI behavior somewhere else in code, but just leave a short note here so people don't get confused about this call site.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment and this is now conditional on DI environment variable being set.

Comment on lines 35 to 38
# TODO when would this happen? Do we request DI RC when DI is not on?
# Or we always get RC even if DI is not turned on?
# -- in dev. env when component is not built but config is still requested?
# TODO log something?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you should have this remote configuration class receiving data if DI's component was never initialize.
I'd say we can assume that this is an impossible case. It makes sense to check if DI's component is always initialized early (which I think it is).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adjusted the comment and maybe we can inject the DI component into here at some point in the future instead of having to reference a global. I think this is the only place besides code tracker (which necessarily must reference a global, since normally it's created before DI component exists at all) that doesn't have all of its dependencies injected.

@p-datadog p-datadog marked this pull request as ready for review November 22, 2024 03:18
# we will start the new remote component as well.
old_state = {
remote_started: old.remote&.started?,
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marcotc I added a note here per your comment in the other PR, does it sound OK to you?

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 95.32710% with 20 lines in your changes missing coverage. Please review.

Project coverage is 97.77%. Comparing base (e703220) to head (4683a48).

Files with missing lines Patch % Lines
lib/datadog/di.rb 57.14% 9 Missing ⚠️
spec/datadog/di/remote_spec.rb 96.99% 4 Missing ⚠️
lib/datadog/di/code_tracker.rb 0.00% 2 Missing ⚠️
lib/datadog/di/remote.rb 95.74% 2 Missing ⚠️
.../integration/everything_from_remote_config_spec.rb 98.38% 2 Missing ⚠️
lib/datadog/core/configuration/components.rb 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4098      +/-   ##
==========================================
+ Coverage   97.76%   97.77%   +0.01%     
==========================================
  Files        1350     1353       +3     
  Lines       81327    81714     +387     
  Branches     4107     4139      +32     
==========================================
+ Hits        79508    79898     +390     
+ Misses       1819     1816       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Involves Datadog core libraries
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants