[DO NOT MERGE] Implement custom exceptions raised from profiler code

[bouwkast]

What does this PR do?

THIS IS FROM CLAUDE CODE - USE AS AN EXAMPLE ONLY - HASN'T BEEN REVIEWED - DO NOT MERGE

In #4985 we removed pii_safe to get error logs back into compliance that the messages that we send to intake are a set of known strings.

In this PR we implement a suggestion of changing known errors that we raise in the native(?) bits of Ruby to use a custom exception type. We will then take this exception type and we will log these errors to intake (we'd avoid sending any potentially unknown exceptions this way).

Motivation:

#4985 (comment)

So I believe we discussed introducing new exception types to be emitted by profiling with the fixed strings from C that we could use for this, but this PR doesn't do that. Any plans on still doing that?

This was the initial recommendation/suggestion that spawned this

• Remove the pii_safe option. Only invoke telemetry methods with PII safe arguments.
• The Ruby profiler is the current user of pii_safe. We will ensure all its messages contain only know values.
• In today's implementation, the profiler (which is in C, to ensure it can execute in memory-unsafe contexts like during GCs or in between Ruby threads context switching) can only safely communicate its errors to our telemetry code (which is in Ruby) through Ruby exceptions. This means that we populate a Ruby exception message with a string containing the profiler error, then use that exception message when reporting it to telemetry.
• To ensure we are only reporting exception messages that we know for sure were created by us, we will introduce a custom exception class (e.g. DatadogProfilingException), and only report messages from that exception. For example, these exceptions will go from rb_raise(rb_eRuntimeError, ... to rb_raise(rb_eDatadogProfilingError, ...

Change log entry

None.

Additional Notes:

How to test the change?

I don't know, I'd need help here

[datadog-datadog-prod-us1]

⚠️ Tests

⚠️ Warnings

🧪 25 Tests failed

Datadog::AppSec::Component.build_appsec_component when appsec is enabled when ffi is not loaded returns nil, warns and reports telemetry from rspec (Datadog)

    expected: 1 time with any arguments
    received: 2 times with any arguments


      expected: 1 time with any arguments
      received: 2 times with any arguments
./spec/datadog/appsec/component_spec.rb:39:in \`block (5 levels) in <top (required)>'
./spec/spec_helper.rb:272:in \`block (2 levels) in <top (required)>'
./spec/spec_helper.rb:154:in \`block (2 levels) in <top (required)>'
/usr/local/bundle/gems/webmock-3.25.1/lib/webmock/rspec.rb:39:in \`block (2 levels) in <top (required)>'
...

Datadog::AppSec::Component.build_appsec_component when appsec is enabled when using ffi version that is known to leak memory with Ruby >= 3.3.0 returns nil, warns and reports telemetry from rspec (Datadog)

    expected: 1 time with any arguments
    received: 2 times with any arguments


      expected: 1 time with any arguments
      received: 2 times with any arguments
./spec/datadog/appsec/component_spec.rb:26:in \`block (5 levels) in <top (required)>'
./spec/spec_helper.rb:272:in \`block (2 levels) in <top (required)>'
./spec/spec_helper.rb:154:in \`block (2 levels) in <top (required)>'
/usr/local/bundle/gems/webmock-3.25.1/lib/webmock/rspec.rb:39:in \`block (2 levels) in <top (required)>'
...

Datadog::AppSec::SecurityEngine::Engine#add_or_update_config when config loading fails with item-level errors reports item-level errors count through telemetry from rspec (Datadog)

private method \`error' called on #<InstanceDouble(Datadog::Core::Telemetry::Component) (anonymous)>

        raise NoMethodError, "%s method \`%s' called on %s" % [
        ^^^^^

Failure/Error: telemetry.error("#{error}: [#{config_ids.join(",")}]")

NoMethodError:
  private method \`error' called on #<InstanceDouble(Datadog::Core::Telemetry::Component) (anonymous)>

...

View all

ℹ️ Info

❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 8550883 | Docs | Was this helpful? Give us feedback!}