Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Metrics Hook #114

Merged
merged 26 commits into from
Dec 19, 2023
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
402b394
Adding inital Metrics Hook.
askpt Dec 14, 2023
9d56d1b
Add metrics tracking to MetricsHook class
askpt Dec 14, 2023
06ba9d4
Add metrics instrumentation to MetricsHook class
askpt Dec 14, 2023
231ab4d
Add descriptions to metric counters
askpt Dec 14, 2023
c50235f
Update MetricsHook.cs with new constant UNIT
askpt Dec 14, 2023
53d9b0b
Add evaluationSuccessCounter metric tracking
askpt Dec 14, 2023
922fcec
Add error tracking for evaluation errors
askpt Dec 14, 2023
98f69eb
Add tags to evaluationActiveUpDownCounter in Finally method
askpt Dec 14, 2023
5010a93
Remove empty line in MetricsHook.cs
askpt Dec 14, 2023
83da161
Refactor metrics hook class and use constants for metric names and de…
askpt Dec 14, 2023
3654da8
Add documentation.
askpt Dec 14, 2023
c19866b
Merge branch 'main' into askpt/113-add-metrics
askpt Dec 14, 2023
5a0ca25
Refactor metrics constants and attributes
askpt Dec 14, 2023
f3932c2
Update metrics constants names
askpt Dec 14, 2023
e96b2a7
Add meter name to MetricsHook class
askpt Dec 14, 2023
3a1edef
Add MetricsHookTest class for OpenFeature.Contrib.Hooks.Otel.Test
askpt Dec 14, 2023
64e9e61
Add Finally_Test to MetricsHookTest.cs
askpt Dec 14, 2023
fcaea28
Refactor metrics hook tests
askpt Dec 14, 2023
a813239
Refactor metrics assertions in MetricsHookTest.cs
askpt Dec 14, 2023
67eb243
Update README.md with usage examples for Traces and Metrics
askpt Dec 14, 2023
4cb73a5
Update OpenTelemetry hook example to send metrics to the console
askpt Dec 14, 2023
f8e52e6
Update MetricsConstants and MetricsHook to use long instead of double
askpt Dec 15, 2023
24d1682
Remove unused constant and update metric counter parameters
askpt Dec 15, 2023
a46278d
Update metrics documentation in README.md
askpt Dec 18, 2023
ea63383
fixup: add askpt to component_owners
toddbaert Dec 18, 2023
87a59e8
Merge branch 'main' into askpt/113-add-metrics
askpt Dec 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/component_owners.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ components:
src/OpenFeature.Contrib.Hooks.Otel:
- bacherfl
- toddbaert
- askpt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@askpt I hope you don't mind I've added you here. This will add you to PRs for this component automatically, once you're an org member.

src/OpenFeature.Contrib.Providers.Flagd:
- bacherfl
- toddbaert
Expand All @@ -17,6 +18,7 @@ components:
test/OpenFeature.Contrib.Hooks.Otel.Test:
- bacherfl
- toddbaert
- askpt
test/OpenFeature.Contrib.Providers.Flagd.Test:
- bacherfl
- toddbaert
Expand All @@ -27,4 +29,4 @@ components:
- matthewelwell

ignored-authors:
- renovate-bot
- renovate-bot
3 changes: 3 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"dotnet.defaultSolution": "DotnetSdkContrib.sln"
}
21 changes: 21 additions & 0 deletions src/OpenFeature.Contrib.Hooks.Otel/MetricsConstants.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
namespace OpenFeature.Contrib.Hooks.Otel
{
internal static class MetricsConstants
{
internal const string ActiveCountName = "feature_flag.evaluation_active_count";
internal const string RequestsTotalName = "feature_flag.evaluation_requests_total";
internal const string SuccessTotalName = "feature_flag.evaluation_success_total";
internal const string ErrorTotalName = "feature_flag.evaluation_error_total";

internal const string ActiveDescription = "active flag evaluations counter";
internal const string RequestsDescription = "feature flag evaluation request counter";
internal const string SuccessDescription = "feature flag evaluation success counter";
internal const string ErrorDescription = "feature flag evaluation error counter";

internal const string KeyAttr = "key";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if they expect PRs for this or not, but we could open one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the library has not "updated" since 2022: https://www.nuget.org/packages/OpenTelemetry.SemanticConventions/1.0.0-rc9.9

Might need some help to reopen engagement there

internal const string ProviderNameAttr = "provider_name";
internal const string VariantAttr = "variant";
internal const string ReasonAttr = "reason";
internal const string ExceptionAttr = "exception";
}
}
134 changes: 134 additions & 0 deletions src/OpenFeature.Contrib.Hooks.Otel/MetricsHook.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Diagnostics.Metrics;
using System.Reflection;
using System.Threading.Tasks;
using OpenFeature.Model;

namespace OpenFeature.Contrib.Hooks.Otel
{
/// <summary>
/// Represents a hook for capturing metrics related to flag evaluations.
/// The meter name is "OpenFeature.Contrib.Hooks.Otel".
/// </summary>
public class MetricsHook : Hook
{
private static readonly AssemblyName AssemblyName = typeof(MetricsHook).Assembly.GetName();
private static readonly string InstrumentationName = AssemblyName.Name;
private static readonly string InstrumentationVersion = AssemblyName.Version?.ToString();

private readonly UpDownCounter<long> _evaluationActiveUpDownCounter;
private readonly Counter<long> _evaluationRequestCounter;
private readonly Counter<long> _evaluationSuccessCounter;
private readonly Counter<long> _evaluationErrorCounter;

/// <summary>
/// Initializes a new instance of the <see cref="MetricsHook"/> class.
/// </summary>
public MetricsHook()
{
var meter = new Meter(InstrumentationName, InstrumentationVersion);

_evaluationActiveUpDownCounter = meter.CreateUpDownCounter<long>(MetricsConstants.ActiveCountName, description: MetricsConstants.ActiveDescription);
_evaluationRequestCounter = meter.CreateCounter<long>(MetricsConstants.RequestsTotalName, "{request}", MetricsConstants.RequestsDescription);
_evaluationSuccessCounter = meter.CreateCounter<long>(MetricsConstants.SuccessTotalName, "{impression}", MetricsConstants.SuccessDescription);
_evaluationErrorCounter = meter.CreateCounter<long>(MetricsConstants.ErrorTotalName, description: MetricsConstants.ErrorDescription);
}

/// <summary>
/// Executes before the flag evaluation and captures metrics related to the evaluation.
/// The metrics are captured in the following order:
/// 1. The active count is incremented. (feature_flag.evaluation_active_count)
/// 2. The request count is incremented. (feature_flag.evaluation_requests_total)
/// </summary>
/// <typeparam name="T">The type of the flag value.</typeparam>
/// <param name="context">The hook context.</param>
/// <param name="hints">The optional hints.</param>
/// <returns>The evaluation context.</returns>
public override Task<EvaluationContext> Before<T>(HookContext<T> context, IReadOnlyDictionary<string, object> hints = null)
{
var tagList = new TagList
{
{ MetricsConstants.KeyAttr, context.FlagKey },
{ MetricsConstants.ProviderNameAttr, context.ProviderMetadata.Name }
};

_evaluationActiveUpDownCounter.Add(1, tagList);
_evaluationRequestCounter.Add(1, tagList);

return base.Before(context, hints);
}


/// <summary>
/// Executes after the flag evaluation and captures metrics related to the evaluation.
/// The metrics are captured in the following order:
/// 1. The success count is incremented. (feature_flag.evaluation_success_total)
/// </summary>
/// <typeparam name="T">The type of the flag value.</typeparam>
/// <param name="context">The hook context.</param>
/// <param name="details">The flag evaluation details.</param>
/// <param name="hints">The optional hints.</param>
/// <returns>The evaluation context.</returns>
public override Task After<T>(HookContext<T> context, FlagEvaluationDetails<T> details, IReadOnlyDictionary<string, object> hints = null)
{
var tagList = new TagList
{
{ MetricsConstants.KeyAttr, context.FlagKey },
{ MetricsConstants.ProviderNameAttr, context.ProviderMetadata.Name },
{ MetricsConstants.VariantAttr, details.Variant ?? details.Value?.ToString() },
{ MetricsConstants.ReasonAttr, details.Reason ?? "UNKNOWN" }
};

_evaluationSuccessCounter.Add(1, tagList);

return base.After(context, details, hints);
}

/// <summary>
/// Executes when an error occurs during flag evaluation and captures metrics related to the error.
/// The metrics are captured in the following order:
/// 1. The error count is incremented. (feature_flag.evaluation_error_total)
/// </summary>
/// <typeparam name="T">The type of the flag value.</typeparam>
/// <param name="context">The hook context.</param>
/// <param name="error">The exception that occurred.</param>
/// <param name="hints">The optional hints.</param>
/// <returns>The evaluation context.</returns>
public override Task Error<T>(HookContext<T> context, Exception error, IReadOnlyDictionary<string, object> hints = null)
{
var tagList = new TagList
{
{ MetricsConstants.KeyAttr, context.FlagKey },
{ MetricsConstants.ProviderNameAttr, context.ProviderMetadata.Name },
{ MetricsConstants.ExceptionAttr, error?.Message ?? "Unknown error" }
};

_evaluationErrorCounter.Add(1, tagList);

return base.Error(context, error, hints);
}

/// <summary>
/// Executes after the flag evaluation is complete and captures metrics related to the evaluation.
/// The active count is decremented. (feature_flag.evaluation_active_count)
/// </summary>
/// <typeparam name="T">The type of the flag value.</typeparam>
/// <param name="context">The hook context.</param>
/// <param name="hints">The optional hints.</param>
/// <returns>The evaluation context.</returns>
public override Task Finally<T>(HookContext<T> context, IReadOnlyDictionary<string, object> hints = null)
{
var tagList = new TagList
{
{ MetricsConstants.KeyAttr, context.FlagKey },
{ MetricsConstants.ProviderNameAttr, context.ProviderMetadata.Name }
};

_evaluationActiveUpDownCounter.Add(-1, tagList);

return base.Finally(context, hints);
}
}
}
65 changes: 63 additions & 2 deletions src/OpenFeature.Contrib.Hooks.Otel/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,15 @@

- open-feature/dotnet-sdk >= v1.0

## Usage
## Usage - Traces

For this hook to function correctly a global `TracerProvider` must be set, an example of how to do this can be found below.

The `open telemetry hook` taps into the after and error methods of the hook lifecycle to write `events` and `attributes` to an existing `span`.
For this, an active span must be set in the `Tracer`, otherwise the hook will no-op.

### Example

The following example demonstrates the use of the `OpenTelemetry hook` with the `OpenFeature dotnet-sdk`. The traces are sent to a `jaeger` OTLP collector running at `localhost:4317`.

```csharp
Expand All @@ -31,7 +32,7 @@ namespace OpenFeatureTestApp
var tracerProvider = Sdk.CreateTracerProviderBuilder()
.AddSource("my-tracer")
.ConfigureResource(r => r.AddService("jaeger-test"))
.AddOtlpExporter(o =>
.AddOtlpExporter(o =>
{
o.ExportProcessorType = ExportProcessorType.Simple;
})
Expand Down Expand Up @@ -64,6 +65,66 @@ In case something went wrong during a feature flag evaluation, you will see an e

![](./assets/otlp-error.png)

## Usage - Metrics

For this hook to function correctly a global `MeterProvider` must be set.
`MetricsHook` performs metric collection by tapping into various hook stages.

Below are the metrics extracted by this hook and dimensions they carry:

| Metric key | Description | Unit | Dimensions |
| -------------------------------------- | ------------------------------- | ------------ | ----------------------------------- |
| feature_flag.evaluation_requests_total | Number of evaluation requests | {request} | key, provider name |
| feature_flag.evaluation_success_total | Flag evaluation successes | {impression} | key, provider name, reason, variant |
| feature_flag.evaluation_error_total | Flag evaluation errors | Counter | key, provider name |
| feature_flag.evaluation_active_count | Active flag evaluations counter | Counter | key |

Consider the following code example for usage.

toddbaert marked this conversation as resolved.
Show resolved Hide resolved
### Example

The following example demonstrates the use of the `OpenTelemetry hook` with the `OpenFeature dotnet-sdk`. The metrics are sent to the `console`.

```csharp
using OpenFeature.Contrib.Providers.Flagd;
using OpenFeature;
using OpenFeature.Contrib.Hooks.Otel;
using OpenTelemetry;
using OpenTelemetry.Metrics;

namespace OpenFeatureTestApp
{
class Hello {
static void Main(string[] args) {

// set up the OpenTelemetry OTLP exporter
var meterProvider = Sdk.CreateMeterProviderBuilder()
.AddMeter("OpenFeature.Contrib.Hooks.Otel")
.ConfigureResource(r => r.AddService("openfeature-test"))
.AddConsoleExporter()
.Build();

// add the Otel Hook to the OpenFeature instance
OpenFeature.Api.Instance.AddHooks(new MetricsHook());

var flagdProvider = new FlagdProvider(new Uri("http://localhost:8013"));

// Set the flagdProvider as the provider for the OpenFeature SDK
OpenFeature.Api.Instance.SetProvider(flagdProvider);

var client = OpenFeature.Api.Instance.GetClient("my-app");

var val = client.GetBooleanValue("myBoolFlag", false, null);

// Print the value of the 'myBoolFlag' feature flag
System.Console.WriteLine(val.Result.ToString());
}
}
}
```

After running this example, you should be able to see some metrics being generated into the console.

## License

Apache 2.0 - See [LICENSE](./../../LICENSE) for more information.
Loading
Loading