Skip to content

Conversation

@dd-oleksii
Copy link
Contributor

What does this PR do?

Adding a feature-flagging evaluation library for use in Python/Ruby tracers.

Motivation

We're adding feature flags support in a bunch of tracers and want to use the common core in Python and Ruby (and any other tracer that can work with Rust).

Additional Notes

It's being moved over from dd-source. You can review it by commit. The first commit is a copy from dd-source, the next ones are dropping features we don't need / adapting to libdatadog.

How to test the change?

See inline tests.

@dd-oleksii dd-oleksii requested review from a team as code owners October 20, 2025 19:40
@dd-octo-sts
Copy link

dd-octo-sts bot commented Oct 20, 2025

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.25 MB 7.25 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 71.31 MB 71.34 MB +.04% (+30.64 KB) 🔍
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 83.67 MB 83.69 MB +.02% (+19.82 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 9.20 MB 9.20 MB +.03% (+3.32 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 18.58 MB 18.54 MB --.22% (-43.00 KB) 💪
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 65.77 KB 65.77 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 129.37 MB 129.18 MB --.14% (-192.00 KB) 💪
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 675.79 MB 675.38 MB --.06% (-419.15 KB) 💪
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 5.93 MB 5.93 MB --.06% (-4.00 KB) 💪
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 65.77 KB 65.77 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 17.52 MB 17.51 MB --.08% (-16.00 KB) 💪
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 32.48 MB 32.45 MB --.09% (-32.16 KB) 💪
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 15.83 MB 15.80 MB --.24% (-39.50 KB) 💪
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 66.79 KB 66.79 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 131.71 MB 131.34 MB --.27% (-376.00 KB) 💪
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 664.92 MB 663.90 MB --.15% (-1.01 MB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 4.54 MB 4.54 MB +.09% (+4.50 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 66.79 KB 66.79 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 18.65 MB 18.66 MB +.04% (+8.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 30.49 MB 30.51 MB +.07% (+24.04 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 63.93 MB 63.95 MB +.02% (+13.84 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.56 MB 8.56 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 78.39 MB 78.41 MB +.02% (+23.58 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 9.90 MB 9.90 MB +.06% (+6.52 KB) 🔍

@codecov-commenter
Copy link

codecov-commenter commented Oct 20, 2025

Codecov Report

❌ Patch coverage is 78.29699% with 209 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.84%. Comparing base (0272ba4) to head (64681df).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1271      +/-   ##
==========================================
+ Coverage   71.74%   71.84%   +0.10%     
==========================================
  Files         356      368      +12     
  Lines       56993    57967     +974     
==========================================
+ Hits        40887    41648     +761     
- Misses      16106    16319     +213     
Components Coverage Δ
datadog-crashtracker 49.05% <ø> (-0.03%) ⬇️
datadog-crashtracker-ffi 5.86% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 87.85% <ø> (+0.01%) ⬆️
data-pipeline-ffi 88.19% <ø> (ø)
ddcommon 84.31% <ø> (ø)
ddcommon-ffi 73.84% <ø> (ø)
ddtelemetry 59.98% <ø> (ø)
ddtelemetry-ffi 21.24% <ø> (ø)
dogstatsd-client 83.26% <ø> (ø)
datadog-ipc 82.39% <ø> (ø)
datadog-profiling 76.90% <ø> (ø)
datadog-profiling-ffi 62.12% <ø> (ø)
datadog-sidecar 36.96% <ø> (+0.29%) ⬆️
datdog-sidecar-ffi 12.37% <ø> (+1.42%) ⬆️
spawn-worker 55.18% <ø> (ø)
tinybytes 92.22% <ø> (ø)
datadog-trace-normalization 98.24% <ø> (ø)
datadog-trace-obfuscation 94.17% <ø> (ø)
datadog-trace-protobuf 59.65% <ø> (ø)
datadog-trace-utils 89.84% <ø> (-0.16%) ⬇️
datadog-tracer-flare 62.42% <ø> (ø)
datadog-log 76.31% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pr-commenter
Copy link

pr-commenter bot commented Oct 21, 2025

Benchmarks

Comparison

Benchmark execution time: 2025-10-21 10:30:23

Comparing candidate commit 64681df in PR branch FFL-1255-feature-flagging with baseline commit 0272ba4 in branch main.

Found 7 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 2 unstable metrics.

scenario:normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...

  • 🟩 execution_time [-19.952µs; -19.790µs] or [-9.692%; -9.613%]
  • 🟩 throughput [+516910.860op/s; +520972.088op/s] or [+10.642%; +10.725%]

scenario:normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...

  • 🟩 execution_time [-38.284µs; -38.109µs] or [-7.156%; -7.123%]
  • 🟩 throughput [+143363.511op/s; +144022.010op/s] or [+7.670%; +7.706%]

scenario:normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters

  • 🟩 execution_time [-26.917µs; -26.834µs] or [-13.752%; -13.710%]
  • 🟩 throughput [+812022.130op/s; +814400.456op/s] or [+15.893%; +15.940%]

scenario:redis/obfuscate_redis_string

  • 🟩 execution_time [-2.001µs; -1.475µs] or [-5.697%; -4.200%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 60.263ms 60.864ms ± 2.298ms 60.581ms ± 0.051ms 60.644ms 60.897ms 78.561ms 80.215ms 32.41% 7.946 61.495 3.77% 0.163ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [60.546ms; 61.183ms] or [-0.523%; +0.523%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.191µs 3.226µs ± 1.455µs 3.018µs ± 0.025µs 3.039µs 3.676µs 14.007µs 15.080µs 399.76% 7.403 55.775 44.99% 0.103µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.025µs; 3.428µs] or [-6.251%; +6.251%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 86.223µs 86.478µs ± 0.159µs 86.456µs ± 0.047µs 86.515µs 86.604µs 86.910µs 88.320µs 2.16% 8.080 89.774 0.18% 0.011µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [86.456µs; 86.500µs] or [-0.025%; +0.025%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 185.528µs 185.997µs ± 0.316µs 185.983µs ± 0.169µs 186.130µs 186.369µs 187.647µs 187.949µs 1.06% 2.900 14.536 0.17% 0.022µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 5320591.581op/s 5376439.137op/s ± 9077.027op/s 5376846.721op/s ± 4886.694op/s 5382213.524op/s 5386943.513op/s 5388676.928op/s 5390023.444op/s 0.25% -2.858 14.234 0.17% 641.843op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 17.771µs 17.842µs ± 0.033µs 17.837µs ± 0.017µs 17.856µs 17.895µs 17.939µs 18.028µs 1.07% 1.269 4.738 0.18% 0.002µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 55468313.376op/s 56046165.255op/s ± 102726.112op/s 56061879.138op/s ± 53575.547op/s 56113223.296op/s 56181402.479op/s 56247608.437op/s 56272917.027op/s 0.38% -1.241 4.581 0.18% 7263.833op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.504µs 10.548µs ± 0.020µs 10.546µs ± 0.013µs 10.562µs 10.582µs 10.599µs 10.629µs 0.78% 0.551 0.757 0.19% 0.001µs 1 200
normalization/normalize_name/normalize_name/good throughput 94085849.061op/s 94806173.818op/s ± 183064.622op/s 94818322.415op/s ± 117157.423op/s 94924710.144op/s 95081489.164op/s 95159933.655op/s 95206291.116op/s 0.41% -0.537 0.725 0.19% 12944.624op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [185.954µs; 186.041µs] or [-0.024%; +0.024%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [5375181.148op/s; 5377697.125op/s] or [-0.023%; +0.023%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [17.838µs; 17.847µs] or [-0.025%; +0.025%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [56031928.404op/s; 56060402.106op/s] or [-0.025%; +0.025%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.545µs; 10.551µs] or [-0.027%; +0.027%] None None None
normalization/normalize_name/normalize_name/good throughput [94780802.823op/s; 94831544.814op/s] or [-0.027%; +0.027%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 10.700ms 10.719ms ± 0.014ms 10.717ms ± 0.007ms 10.724ms 10.738ms 10.759ms 10.817ms 0.94% 2.871 14.844 0.13% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [10.717ms; 10.721ms] or [-0.018%; +0.018%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 238.416ns 250.891ns ± 13.705ns 245.902ns ± 4.325ns 251.402ns 285.478ns 291.198ns 297.131ns 20.83% 1.677 1.818 5.45% 0.969ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [248.991ns; 252.790ns] or [-0.757%; +0.757%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 5.011µs 5.084µs ± 0.038µs 5.088µs ± 0.022µs 5.103µs 5.163µs 5.166µs 5.167µs 1.57% 0.474 -0.041 0.75% 0.003µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.079µs; 5.089µs] or [-0.104%; +0.104%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 32.622µs 33.387µs ± 1.383µs 32.735µs ± 0.037µs 32.878µs 36.386µs 36.424µs 36.607µs 11.83% 1.695 0.903 4.13% 0.098µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [33.195µs; 33.578µs] or [-0.574%; +0.574%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.332µs 2.393µs ± 0.019µs 2.390µs ± 0.010µs 2.408µs 2.420µs 2.426µs 2.433µs 1.81% -0.738 1.241 0.80% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.391µs; 2.396µs] or [-0.111%; +0.111%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching serializing traces from their internal representation to msgpack execution_time 14.566ms 14.700ms ± 0.092ms 14.651ms ± 0.069ms 14.787ms 14.826ms 14.864ms 14.911ms 1.78% 0.209 -1.549 0.63% 0.007ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching serializing traces from their internal representation to msgpack execution_time [14.687ms; 14.712ms] or [-0.087%; +0.087%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
single_flag_killswitch/rules-based execution_time 172.735ns 177.112ns ± 3.330ns 176.810ns ± 1.820ns 178.352ns 183.597ns 186.940ns 187.885ns 6.26% 0.976 0.790 1.88% 0.235ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
single_flag_killswitch/rules-based execution_time [176.651ns; 177.573ns] or [-0.261%; +0.261%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.896µs 3.913µs ± 0.003µs 3.912µs ± 0.002µs 3.914µs 3.917µs 3.919µs 3.920µs 0.19% -0.690 5.127 0.07% 0.000µs 1 200
credit_card/is_card_number/ throughput 255118075.669op/s 255583588.765op/s ± 185079.596op/s 255600750.229op/s ± 129182.962op/s 255717548.618op/s 255815452.074op/s 255879733.507op/s 256690878.128op/s 0.43% 0.704 5.214 0.07% 13087.104op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 78.722µs 80.943µs ± 1.013µs 80.923µs ± 0.684µs 81.674µs 82.549µs 83.286µs 84.173µs 4.02% 0.149 -0.042 1.25% 0.072µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 11880307.926op/s 12356274.129op/s ± 154414.736op/s 12357380.270op/s ± 104766.772op/s 12455700.934op/s 12622429.183op/s 12678125.968op/s 12702938.479op/s 2.80% -0.077 -0.102 1.25% 10918.771op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 71.463µs 72.298µs ± 0.385µs 72.306µs ± 0.249µs 72.538µs 72.946µs 73.228µs 73.576µs 1.76% 0.333 -0.034 0.53% 0.027µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 13591352.614op/s 13832017.257op/s ± 73540.872op/s 13830143.284op/s ± 47499.053op/s 13882071.655op/s 13942411.072op/s 13976339.272op/s 13993318.643op/s 1.18% -0.304 -0.074 0.53% 5200.125op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.894µs 3.913µs ± 0.003µs 3.913µs ± 0.002µs 3.915µs 3.918µs 3.920µs 3.923µs 0.24% -0.790 6.250 0.08% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 254925652.505op/s 255551209.369op/s ± 202686.285op/s 255544400.840op/s ± 132297.328op/s 255683427.299op/s 255815209.110op/s 255869255.424op/s 256806399.837op/s 0.49% 0.809 6.360 0.08% 14332.085op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 68.388µs 69.192µs ± 0.393µs 69.179µs ± 0.250µs 69.422µs 69.884µs 70.182µs 70.340µs 1.68% 0.353 -0.091 0.57% 0.028µs 1 200
credit_card/is_card_number/378282246310005 throughput 14216581.083op/s 14453064.167op/s ± 81901.071op/s 14455240.073op/s ± 52180.962op/s 14507711.581op/s 14574497.986op/s 14615173.879op/s 14622519.164op/s 1.16% -0.323 -0.126 0.57% 5791.280op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 45.362µs 45.639µs ± 0.152µs 45.635µs ± 0.125µs 45.761µs 45.908µs 45.968µs 45.986µs 0.77% 0.183 -0.775 0.33% 0.011µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 21745697.120op/s 21911111.296op/s ± 73123.568op/s 21913112.336op/s ± 59769.020op/s 21971280.241op/s 22025491.096op/s 22039765.336op/s 22045015.923op/s 0.60% -0.171 -0.782 0.33% 5170.617op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.427µs 6.436µs ± 0.012µs 6.435µs ± 0.002µs 6.437µs 6.442µs 6.447µs 6.546µs 1.73% 8.258 74.800 0.18% 0.001µs 1 200
credit_card/is_card_number/x371413321323331 throughput 152766248.765op/s 155375635.466op/s ± 275048.999op/s 155403636.459op/s ± 57941.873op/s 155461562.779op/s 155549166.207op/s 155577910.965op/s 155588479.885op/s 0.12% -8.212 74.226 0.18% 19448.901op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.896µs 3.912µs ± 0.003µs 3.913µs ± 0.002µs 3.914µs 3.917µs 3.918µs 3.919µs 0.18% -1.049 6.044 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 255136824.609op/s 255594481.669op/s ± 179762.849op/s 255586658.377op/s ± 111431.080op/s 255704538.408op/s 255858369.661op/s 255955007.364op/s 256706456.167op/s 0.44% 1.064 6.139 0.07% 12711.153op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 62.329µs 64.091µs ± 0.545µs 64.062µs ± 0.381µs 64.479µs 64.949µs 65.252µs 65.495µs 2.24% -0.100 -0.103 0.85% 0.039µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15268270.719op/s 15603847.354op/s ± 132724.275op/s 15609980.303op/s ± 92862.856op/s 15695203.498op/s 15807259.850op/s 15899909.218op/s 16044004.567op/s 2.78% 0.149 -0.061 0.85% 9385.024op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 53.910µs 54.233µs ± 0.143µs 54.273µs ± 0.084µs 54.334µs 54.406µs 54.456µs 54.512µs 0.44% -0.636 -0.537 0.26% 0.010µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 18344582.860op/s 18439150.408op/s ± 48818.039op/s 18425513.028op/s ± 28583.663op/s 18465536.912op/s 18533691.007op/s 18546720.339op/s 18549462.141op/s 0.67% 0.644 -0.530 0.26% 3451.957op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.893µs 3.912µs ± 0.003µs 3.912µs ± 0.002µs 3.914µs 3.917µs 3.919µs 3.921µs 0.22% -0.811 7.313 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 255044077.641op/s 255612385.401op/s ± 195182.708op/s 255612031.120op/s ± 123401.024op/s 255738060.288op/s 255862559.726op/s 255920304.661op/s 256854561.253op/s 0.49% 0.830 7.433 0.08% 13801.502op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 50.167µs 50.536µs ± 0.162µs 50.575µs ± 0.101µs 50.647µs 50.751µs 50.860µs 51.180µs 1.20% -0.181 0.417 0.32% 0.011µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 19538867.028op/s 19788148.766op/s ± 63391.477op/s 19772444.018op/s ± 39504.867op/s 19832619.172op/s 19903550.018op/s 19926147.372op/s 19933362.469op/s 0.81% 0.204 0.371 0.32% 4482.454op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 45.348µs 45.688µs ± 0.143µs 45.706µs ± 0.116µs 45.802µs 45.903µs 45.938µs 45.952µs 0.54% -0.221 -0.867 0.31% 0.010µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 21761795.191op/s 21887654.387op/s ± 68622.040op/s 21879071.607op/s ± 55525.833op/s 21943031.447op/s 22004221.307op/s 22029224.379op/s 22051751.332op/s 0.79% 0.231 -0.861 0.31% 4852.311op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.428µs 6.438µs ± 0.004µs 6.438µs ± 0.003µs 6.441µs 6.444µs 6.447µs 6.450µs 0.18% -0.015 -0.185 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 155037011.844op/s 155322881.399op/s ± 97456.749op/s 155316858.887op/s ± 64051.228op/s 155396371.496op/s 155475419.720op/s 155546805.075op/s 155567904.439op/s 0.16% 0.018 -0.186 0.06% 6891.233op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.912µs; 3.913µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ throughput [255557938.513op/s; 255609239.017op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [80.803µs; 81.084µs] or [-0.173%; +0.173%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12334873.732op/s; 12377674.527op/s] or [-0.173%; +0.173%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [72.245µs; 72.351µs] or [-0.074%; +0.074%] None None None
credit_card/is_card_number/ 378282246310005 throughput [13821825.199op/s; 13842209.314op/s] or [-0.074%; +0.074%] None None None
credit_card/is_card_number/37828224631 execution_time [3.913µs; 3.914µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/37828224631 throughput [255523118.999op/s; 255579299.738op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/378282246310005 execution_time [69.137µs; 69.246µs] or [-0.079%; +0.079%] None None None
credit_card/is_card_number/378282246310005 throughput [14441713.467op/s; 14464414.868op/s] or [-0.079%; +0.079%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [45.618µs; 45.661µs] or [-0.046%; +0.046%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [21900977.072op/s; 21921245.519op/s] or [-0.046%; +0.046%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.434µs; 6.438µs] or [-0.025%; +0.025%] None None None
credit_card/is_card_number/x371413321323331 throughput [155337516.320op/s; 155413754.612op/s] or [-0.025%; +0.025%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.912µs; 3.913µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/ throughput [255569568.267op/s; 255619395.071op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [64.016µs; 64.167µs] or [-0.118%; +0.118%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15585453.046op/s; 15622241.662op/s] or [-0.118%; +0.118%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [54.213µs; 54.253µs] or [-0.037%; +0.037%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [18432384.697op/s; 18445916.119op/s] or [-0.037%; +0.037%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.912µs; 3.913µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255585334.955op/s; 255639435.847op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [50.513µs; 50.558µs] or [-0.044%; +0.044%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [19779363.317op/s; 19796934.215op/s] or [-0.044%; +0.044%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [45.668µs; 45.708µs] or [-0.043%; +0.043%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [21878144.033op/s; 21897164.742op/s] or [-0.043%; +0.043%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.438µs; 6.439µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [155309374.831op/s; 155336387.967op/s] or [-0.009%; +0.009%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.996µs 25.656µs ± 9.707µs 18.170µs ± 0.119µs 34.807µs 43.884µs 45.034µs 70.541µs 288.22% 1.052 0.991 37.74% 0.686µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.311µs; 27.002µs] or [-5.244%; +5.244%] None None None

Group 14

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 163.391µs 164.131µs ± 0.372µs 164.037µs ± 0.157µs 164.234µs 164.798µs 165.269µs 166.740µs 1.65% 2.634 12.891 0.23% 0.026µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [164.079µs; 164.182µs] or [-0.031%; +0.031%] None None None

Group 15

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 495.970µs 496.835µs ± 0.428µs 496.773µs ± 0.300µs 497.111µs 497.661µs 497.881µs 498.147µs 0.28% 0.509 -0.232 0.09% 0.030µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 2007438.452op/s 2012741.738op/s ± 1734.005op/s 2012991.737op/s ± 1217.728op/s 2014114.241op/s 2015074.053op/s 2015739.474op/s 2016249.854op/s 0.16% -0.505 -0.238 0.09% 122.613op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 370.091µs 370.804µs ± 0.298µs 370.791µs ± 0.209µs 371.007µs 371.289µs 371.569µs 371.640µs 0.23% 0.329 -0.000 0.08% 0.021µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2690778.223op/s 2696843.866op/s ± 2169.835op/s 2696936.614op/s ± 1518.881op/s 2698379.747op/s 2700147.795op/s 2701277.401op/s 2702039.735op/s 0.19% -0.325 -0.004 0.08% 153.431op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 168.496µs 168.852µs ± 0.151µs 168.838µs ± 0.107µs 168.948µs 169.111µs 169.230µs 169.264µs 0.25% 0.360 -0.099 0.09% 0.011µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5907945.417op/s 5922367.655op/s ± 5302.300op/s 5922840.947op/s ± 3767.381op/s 5926234.029op/s 5930205.931op/s 5931781.776op/s 5934867.179op/s 0.20% -0.355 -0.104 0.09% 374.929op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 36.219µs 36.410µs ± 0.083µs 36.407µs ± 0.059µs 36.470µs 36.534µs 36.601µs 36.670µs 0.72% 0.102 -0.134 0.23% 0.006µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 27270413.419op/s 27465448.442op/s ± 62312.471op/s 27467587.244op/s ± 44668.096op/s 27506948.749op/s 27564629.686op/s 27598128.932op/s 27610147.940op/s 0.52% -0.089 -0.144 0.23% 4406.157op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 46.176µs 46.324µs ± 0.068µs 46.318µs ± 0.043µs 46.364µs 46.444µs 46.513µs 46.563µs 0.53% 0.680 0.758 0.15% 0.005µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 21476453.693op/s 21587261.884op/s ± 31719.424op/s 21590098.781op/s ± 20093.450op/s 21608782.242op/s 21630230.848op/s 21643130.412op/s 21656401.933op/s 0.31% -0.669 0.735 0.15% 2242.902op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [496.776µs; 496.894µs] or [-0.012%; +0.012%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [2012501.422op/s; 2012982.054op/s] or [-0.012%; +0.012%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [370.763µs; 370.845µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2696543.148op/s; 2697144.585op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [168.831µs; 168.872µs] or [-0.012%; +0.012%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5921632.807op/s; 5923102.503op/s] or [-0.012%; +0.012%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [36.398µs; 36.421µs] or [-0.031%; +0.031%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [27456812.532op/s; 27474084.351op/s] or [-0.031%; +0.031%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [46.314µs; 46.333µs] or [-0.020%; +0.020%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [21582865.877op/s; 21591657.891op/s] or [-0.020%; +0.020%] None None None

Group 16

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sdk_test_data/rules-based execution_time 121.714µs 123.416µs ± 1.771µs 123.046µs ± 0.381µs 123.466µs 125.599µs 132.610µs 137.692µs 11.90% 4.966 30.367 1.43% 0.125µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sdk_test_data/rules-based execution_time [123.171µs; 123.661µs] or [-0.199%; +0.199%] None None None

Group 17

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 64681df 1761041811 FFL-1255-feature-flagging
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
receiver_entry_point/report/2597 execution_time 6.070ms 6.111ms ± 0.025ms 6.109ms ± 0.014ms 6.124ms 6.148ms 6.211ms 6.235ms 2.07% 1.486 4.783 0.40% 0.002ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
receiver_entry_point/report/2597 execution_time [6.108ms; 6.115ms] or [-0.056%; +0.056%] None None None

Baseline

Omitted due to size.

@dd-oleksii dd-oleksii changed the title Ffl 1255 feature flagging FFL-1255 Add feature flag evaluation library Oct 21, 2025
@dd-oleksii dd-oleksii requested a review from a team October 21, 2025 10:35
Copy link
Contributor

@bantonsson bantonsson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in general. I can't comment on the correctness, but the tests are passing 👍🏼

}

#[test]
fn evaluation_sdk_test_data() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels a bit weird that all the test cases are run inside a single method. I think it would be better to use a build.rs file to generate them. I'll open a small PR.

impl Split {
/// Return `true` if `targeting_key` matches the given split.
///
/// To match a split, subject must match all underlying shards.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if there was some documentation that describes how the algorithm works. I see that there are shards, but how are they used.

test_cases
}

fn bench_sdk_test_data_rules_based(b: &mut Bencher) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exciting!

@dd-mergequeue dd-mergequeue bot merged commit 7e0063f into main Oct 22, 2025
37 checks passed
@dd-mergequeue dd-mergequeue bot deleted the FFL-1255-feature-flagging branch October 22, 2025 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants