Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/cloud/metrics/openmetrics/api-reference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ The list of metrics and their labels are available via the [List Descriptors](/c
### Labels

A label is a key-value attribute associated with a metric data point. Labels can be used to filter or aggregate metrics.
Temporal SDKs and the Temporal Service call this same concept a [tag](/glossary#tag); it's called a label once scraped in OpenMetrics format.

Common labels include:

Expand Down
6 changes: 3 additions & 3 deletions docs/design-patterns/retry-metrics.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -287,9 +287,9 @@ export async function monitoredRetryWorkflow(endpoint: string): Promise<string>
</TabItem>
</Tabs>

### Add dimension labels to the metric
### Add dimension tags to the metric

Add labels (tags) to the metric to identify which Activity type, endpoint, or Workflow is producing the high attempt counts.
Add tags to the metric to identify which Activity type, endpoint, or Workflow is producing the high attempt counts.
This makes the metric actionable in dashboards and alerts.

<Tabs groupId="language" queryString>
Expand Down Expand Up @@ -356,7 +356,7 @@ if (ctx.info.attempt > ALERT_THRESHOLD) {

- **Choose a threshold above normal transient noise.** If your downstream system occasionally has 1–2 retry attempts under normal conditions, set the threshold at 5 or 10 so the metric only fires for genuinely sustained failures.
- **Emit on every attempt above the threshold, not only once.** Incrementing the counter on each high-attempt invocation allows alerting systems to detect both the onset and the duration of a problem by watching the counter rate.
- **Use the SDK metrics scope, not a third-party library.** The SDK scope integrates with your Worker's existing metrics pipeline and adds default labels such as namespace and task queue automatically.
- **Use the SDK metrics scope, not a third-party library.** The SDK scope integrates with your Worker's existing metrics pipeline and adds default tags such as namespace and task queue automatically.
- **Set up rate-based alerts, not count-based.** A count alert requires resetting or remembering the baseline. A rate alert (e.g., "more than 3 increments per minute") fires when the problem is active and clears when it resolves.
- **Combine with Fast/Slow Retries.** Emit the metric in the slow-phase Activity of a [Fast/Slow Retries](/design-patterns/fast-slow-retries) pattern to alert when the Workflow has been in the slow phase long enough to be a concern.

Expand Down
2 changes: 1 addition & 1 deletion docs/develop/go/workflows/selectors.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ If multiple items are available, the order of matching is not defined.

An important use case of futures is setting up a race between a timer and a pending activity, effectively adding a "soft" timeout that doesn't result in any errors or retries of that activity.

For example, [the Timer sample](https://github.com/temporalio/samples-go/blob/master/timer) shows how you can write a long running order processing operation where:
For example, [the Timer sample](https://github.com/temporalio/samples-go/tree/main/timer) shows how you can write a long running order processing operation where:

- if processing takes too long, we send out a notification email to user about the delay, but we won't cancel the operation
- if the operation finishes before the timer fires, then we want to cancel the timer.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -163,4 +163,4 @@ You create, operate, and manage access to your Codec Server in your own environm
The Temporal CLI and the Web UI in turn provide built-in hooks to call the Codec Server to decode encrypted payloads on demand.
Refer to the [Codec Server](/production-deployment/data-encryption) documentation for information on how to design and deploy a Codec Server.

For reference, see the [Codec server](https://github.com/temporalio/sdk-java/tree/master/temporal-remote-data-encoder) sample.
For reference, see the [Codec server](https://github.com/temporalio/sdk-java/tree/main/temporal-remote-data-encoder) sample.
2 changes: 1 addition & 1 deletion docs/develop/java/client/temporal-client.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -669,7 +669,7 @@ A Workflow Execution can be started either synchronously or asynchronously.

- Asynchronous start initiates a Workflow Execution and immediately returns to the caller. This is the most common way
to start Workflows in production code. The
[`WorkflowClient`](https://github.com/temporalio/sdk-java/blob/master/temporal-sdk/src/main/java/io/temporal/client/WorkflowClient.java)
[`WorkflowClient`](https://github.com/temporalio/sdk-java/blob/main/temporal-sdk/src/main/java/io/temporal/client/WorkflowClient.java)
provides some static methods, such as `start`, `execute`, `signalWithStart` etc., that help with starting your
Workflows asynchronously.

Expand Down
2 changes: 1 addition & 1 deletion docs/develop/java/integrations/spring-ai.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -395,6 +395,6 @@ new McpPlugin();

## Resources

- [`temporal-spring-ai` README](https://github.com/temporalio/sdk-java/blob/master/temporal-spring-ai/README.md) — full reference for the module
- [`temporal-spring-ai` README](https://github.com/temporalio/sdk-java/blob/main/contrib/temporal-spring-ai/README.md) — full reference for the module
- [Spring Boot integration](/develop/java/integrations/spring-boot-integration) — required companion module
- [Plugin system](/develop/plugins-guide) — how integrations are registered with Workers and Clients
2 changes: 1 addition & 1 deletion docs/develop/java/platform/observability.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ WorkerFactoryOptions factoryOptions =
WorkerFactory factory = WorkerFactory.newInstance(client, factoryOptions);
```

For more information, see the Temporal [OpenTracing module](https://github.com/temporalio/sdk-java/blob/master/temporal-opentracing/README.md).
For more information, see the Temporal [OpenTracing module](https://github.com/temporalio/sdk-java/blob/main/contrib/temporal-opentracing/README.md).

### Context Propagation Over Nexus Operation Calls

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [vale] reported by reviewdog 🐶
[Temporal.Headings] 'Context Propagation Over Nexus Operation Calls' should use sentence-style capitalization.


Expand Down
2 changes: 1 addition & 1 deletion docs/develop/rust/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Once your local Temporal Service is set up, continue building with the following

- [Rust SDK Quickstart - Setup Guide](/develop/rust/quickstart)
- [Rust API Documentation](https://docs.rs/temporalio-sdk/latest/temporalio_sdk/)
- [Rust SDK GitHub](https://github.com/temporalio/sdk-core/tree/master/crates/sdk)
- [Rust SDK GitHub](https://github.com/temporalio/sdk-rust/tree/main/crates/sdk)

### Get Connected with the Temporal Rust Community

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [vale] reported by reviewdog 🐶
[Temporal.Headings] 'Get Connected with the Temporal Rust Community' should use sentence-style capitalization.


Expand Down
2 changes: 1 addition & 1 deletion docs/encyclopedia/data-conversion/codec-server.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ Codec Server.
Most Temporal SDKs provide example Codec Server implementations:

- [Go](https://github.com/temporalio/samples-go/tree/main/codec-server)
- [Java](https://github.com/temporalio/sdk-java/tree/master/temporal-remote-data-encoder)
- [Java](https://github.com/temporalio/sdk-java/tree/main/temporal-remote-data-encoder)
- [Python](https://github.com/temporalio/samples-python/blob/main/encryption/codec_server.py) | [Python with External Storage](https://github.com/temporalio/samples-python/tree/main/external_storage)
- [TypeScript](https://github.com/temporalio/samples-typescript/blob/main/encryption/src/codec-server.ts)
- [.NET](https://github.com/temporalio/samples-dotnet/blob/main/src/Encryption/CodecServer/Program.cs)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,6 @@ Evaluate and ensure that your remote data encoder endpoints are secured and only
Samples:

- [Go](https://github.com/temporalio/samples-go/tree/main/codec-server)
- [Java](https://github.com/temporalio/sdk-java/tree/master/temporal-remote-data-encoder)
- [Java](https://github.com/temporalio/sdk-java/tree/main/temporal-remote-data-encoder)
- [Python](https://github.com/temporalio/samples-python/tree/main/encryption)
- [TypeScript](https://github.com/temporalio/samples-typescript/tree/main/encryption)
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Use dynamic configuration keys to fine-tune your self-deployed Temporal Service

For details on dynamic configuration keys, see [Dynamic configuration reference](/references/dynamic-configuration).

For dynamic configuration examples, see [https://github.com/temporalio/temporal/tree/master/config/dynamicconfig](https://github.com/temporalio/temporal/tree/master/config/dynamicconfig).
For dynamic configuration examples, see [https://github.com/temporalio/temporal/tree/main/config/dynamicconfig](https://github.com/temporalio/temporal/tree/main/config/dynamicconfig).

## What is Temporal Service security configuration? {/* #temporal-cluster-security-configuration */}

Expand Down
7 changes: 7 additions & 0 deletions docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -613,6 +613,13 @@ Queue to listen on.

<!-- _Tags: [term](/tags/term), [explanation](/tags/explanation)_ -->

#### [Tag](/references/sdk-metrics)

A Tag is a key-value pair that a Temporal SDK or the Temporal Service attaches to an emitted metric, such as `namespace` or `task_queue`.
Once a metric is scraped by Prometheus or another OpenMetrics-compatible system, the same key-value pair is called a label.

<!-- _Tags: [term](/tags/term), [explanation](/tags/explanation)_ -->

#### [Task](/tasks#task)

A Task is the context needed to make progress with a specific Workflow Execution or Activity Execution.
Expand Down
2 changes: 1 addition & 1 deletion docs/production-deployment/data-encryption.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ The Web UI and CLI send POST requests to the following endpoints on your Codec S
For examples on how to create your Codec Server, see the following Codec Server implementation samples:

- [Go](https://github.com/temporalio/samples-go/tree/main/codec-server)
- [Java](https://github.com/temporalio/sdk-java/tree/master/temporal-remote-data-encoder)
- [Java](https://github.com/temporalio/sdk-java/tree/main/temporal-remote-data-encoder)
- [Python](https://github.com/temporalio/samples-python/blob/main/encryption/codec_server.py)
- [TypeScript](https://github.com/temporalio/samples-typescript/blob/main/encryption/src/codec-server.ts)
- [.NET](https://github.com/temporalio/samples-dotnet/blob/main/src/Encryption/CodecServer/Program.cs)
Expand Down
2 changes: 2 additions & 0 deletions docs/references/cluster-metrics.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ See [Cloud metrics](/cloud/metrics/) for metrics emitted by [Temporal Cloud](/cl

See [SDK metrics](/references/sdk-metrics) for metrics emitted by the [SDKs](/encyclopedia/architecture/temporal-sdks).

Temporal SDKs and the Temporal Service attach these metric dimensions as [tags](/glossary#tag). Once a metric is scraped by Prometheus or another OpenMetrics-compatible system, the same dimensions are called labels.

:::

A Temporal Service emits a range of metrics to help operators get visibility into the Temporal Service's performance and to set up alerts.
Expand Down
28 changes: 11 additions & 17 deletions docs/references/sdk-metrics.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ Some SDKs emit metrics beyond what this reference lists.
Only the metrics on this page have guaranteed, defined behavior.
Treat any others as deprecated, inconsistent, or experimental.

Temporal SDKs and the Temporal Service attach these metric dimensions as [tags](/glossary#tag). Once a metric is scraped by Prometheus or another OpenMetrics-compatible system, the same dimensions are called labels.

:::

The Temporal SDKs emit a set of metrics from Temporal Client usage and Worker Processes.
Expand All @@ -38,19 +40,11 @@ The Temporal SDKs emit a set of metrics from Temporal Client usage and Worker Pr
All metrics are prefixed with `temporal_` before being exported to their configured destination.
Section headings on this page omit the prefix.
Some metrics are specific to certain SDKs.
The Availability column in the table below refers to where a metric is defined:

TypeScript, Python, .NET, and Ruby SDK metrics are defined in the Core SDK.

PHP and Go metrics are defined in the Go SDK.

Java metrics are defined in the Java SDK.
Metrics are defined in the following locations.

- [Core SDK Worker metrics](https://github.com/temporalio/sdk-rust/blob/main/crates/sdk-core/src/telemetry/metrics.rs)
- [Core SDK Client metrics](https://github.com/temporalio/sdk-rust/blob/main/crates/client/src/metrics.rs)
- [Java SDK Worker metrics](https://github.com/temporalio/sdk-java/blob/master/temporal-sdk/src/main/java/io/temporal/worker/MetricsType.java)
- [Java SDK Client metrics](https://github.com/temporalio/sdk-java/blob/master/temporal-serviceclient/src/main/java/io/temporal/serviceclient/MetricsType.java)
- [Go SDK Worker and Client metrics](https://github.com/temporalio/sdk-go/blob/c32b04729cc7691f80c16f80eed7f323ee5ce24f/internal/common/metrics/constants.go)
- **Core**: TypeScript, Python, .NET, and Ruby SDK metrics are defined in the Core SDK, in [Worker metrics](https://github.com/temporalio/sdk-rust/blob/main/crates/sdk-core/src/telemetry/metrics.rs) and [Client metrics](https://github.com/temporalio/sdk-rust/blob/main/crates/client/src/metrics.rs).
- **Go**: Go and PHP SDK metrics are defined in the Go SDK, in [Worker and Client metrics](https://github.com/temporalio/sdk-go/blob/main/internal/common/metrics/constants.go).
- **Java**: Java SDK metrics are defined in the Java SDK, in [Worker metrics](https://github.com/temporalio/sdk-java/blob/main/temporal-sdk/src/main/java/io/temporal/worker/MetricsType.java) and [Client metrics](https://github.com/temporalio/sdk-java/blob/main/temporal-serviceclient/src/main/java/io/temporal/serviceclient/MetricsType.java).

:::note Metric units across SDKs

Expand All @@ -64,7 +58,7 @@ The Core SDK is a shared common core library used by several Temporal SDKs, incl

:::

Each metric may have some combination of the following keys attached to them:
Each metric may have some combination of the following [tags](/glossary#tag) attached to them:

- `task_queue`: Task Queue that the Worker Entity is polling
- `namespace`: Namespace the Worker is bound to
Expand All @@ -82,7 +76,7 @@ Each metric may have some combination of the following keys attached to them:
- `workflow_type`: The name of the Workflow Function the metric is associated with
- `operation`: RPC method name; available for metrics related to Temporal Client gRPC requests

Some keys may not be available in every SDK, and Histogram metrics may have different buckets in each SDK.
Some tags may not be available in every SDK, and Histogram metrics may have different buckets in each SDK.

| Metric name | Emitted by | Metric type | Availability |
| -------------------------------------------------------------------------------------------- | -------------- | ----------- | -------------- |
Expand Down Expand Up @@ -173,7 +167,7 @@ An Activity Worker poll for an Activity Task timed out, and no Activity Task is
The Schedule-To-Start time of an Activity Task.
A [Schedule-To-Start Timeout](/encyclopedia/detecting-activity-failures#schedule-to-start-timeout) can be set when an Activity Execution is spawned.
Use this metric to check that Activity Tasks are being processed from the queue in a timely manner.
Some SDKs include the `activity_type` label, but the metric should not vary by type, because the type does not influence the rate at which tasks are pulled from the queue.
Some SDKs include the `activity_type` tag, but the metric should not vary by type, because the type does not influence the rate at which tasks are pulled from the queue.

- Type: Histogram
- Available in: Core, Go, Java
Expand Down Expand Up @@ -435,7 +429,7 @@ A Worker Entity has been registered, created, or started.
### `worker_task_slots_available`

The total number of Workflow, Activity, Local Activity, or Nexus Task execution slots that are currently available.
Use the `worker_type` key to differentiate execution slots.
Use the `worker_type` tag to differentiate execution slots.
The Worker type specifies an ability to perform certain tasks.
For example, Workflow Workers execute Workflow Tasks, Activity Workers execute Activity Tasks, and so forth.

Expand All @@ -446,7 +440,7 @@ For example, Workflow Workers execute Workflow Tasks, Activity Workers execute A
### `worker_task_slots_used`

The total number of Workflow, Activity, Local Activity, or Nexus Tasks execution slots in current use.
Use the `worker_type` key to differentiate execution slots.
Use the `worker_type` tag to differentiate execution slots.
The Worker type specifies an ability to perform certain tasks.
For example, Workflow Workers execute Workflow Tasks, Activity Workers execute Activity Tasks, and so forth.

Expand Down
Loading