Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Spark Operator Prometheus Metrics Guide #3983

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
+19 −19
Open

Update Spark Operator Prometheus Metrics Guide #3983

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 19 additions & 19 deletions content/en/docs/components/spark-operator/getting-started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,30 +228,30 @@ If enabled, the operator generates the following metrics:

| Metric | Description |
| ------------- | ------------- |
| `spark_app_count` | Total number of SparkApplication handled by the Operator.|
| `spark_app_submit_count` | Total number of SparkApplication spark-submitted by the Operator.|
| `spark_app_success_count` | Total number of SparkApplication which completed successfully.|
| `spark_app_failure_count` | Total number of SparkApplication which failed to complete. |
| `spark_app_running_count` | Total number of SparkApplication which are currently running.|
| `spark_app_success_execution_time_microseconds` | Execution time for applications which succeeded.|
| `spark_app_failure_execution_time_microseconds` | Execution time for applications which failed. |
| `spark_app_start_latency_microseconds` | Start latency of SparkApplication as type of [Prometheus Summary](https://prometheus.io/docs/concepts/metric_types/#summary). |
| `spark_app_start_latency_seconds` | Start latency of SparkApplication as type of [Prometheus Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram). |
| `spark_app_executor_success_count` | Total number of Spark Executors which completed successfully. |
| `spark_app_executor_failure_count` | Total number of Spark Executors which failed. |
| `spark_app_executor_running_count` | Total number of Spark Executors which are currently running. |
| `spark_application_count` | Total number of SparkApplication handled by the Operator.|
| `spark_application_submit_count` | Total number of SparkApplication spark-submitted by the Operator.|
| `spark_application_success_count` | Total number of SparkApplication which completed successfully.|
| `spark_application_failure_count` | Total number of SparkApplication which failed to complete. |
| `spark_application_running_count` | Total number of SparkApplication which are currently running.|
| `spark_application_success_execution_time_seconds` | Execution time for applications which succeeded.|
| `spark_application_failure_execution_time_seconds` | Execution time for applications which failed. |
| `spark_application_start_latency_seconds` | Start latency of SparkApplication as type of [Prometheus Summary](https://prometheus.io/docs/concepts/metric_types/#summary). |
| `spark_application_start_latency_seconds` | Start latency of SparkApplication as type of [Prometheus Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram). |
| `spark_executor_success_count` | Total number of Spark Executors which completed successfully. |
| `spark_executor_failure_count` | Total number of Spark Executors which failed. |
| `spark_executor_running_count` | Total number of Spark Executors which are currently running. |

#### Work Queue Metrics

| Metric | Description |
| ------------- | ------------- |
| `spark_application_controller_depth` | Current depth of workqueue |
| `spark_application_controller_adds` | Total number of adds handled by workqueue |
| `spark_application_controller_latency` | Latency for workqueue |
| `spark_application_controller_work_duration` | How long processing an item from workqueue takes |
| `spark_application_controller_retries` | Total number of retries handled by workqueue |
| `spark_application_controller_unfinished_work_seconds` | Unfinished work in seconds |
| `spark_application_controller_longest_running_processor_microseconds` | Longest running processor in microseconds |
| `workqueue_depth` | Current depth of workqueue |
| `workqueue_adds_total` | Total number of adds handled by workqueue |
| `workqueue_queue_duration_seconds_bucket` | How long in seconds an item stays in workqueue before being requested |
| `workqueue_work_duration_seconds_bucket` | How long in seconds processing an item from workqueue takes |
| `workqueue_retries_total` | Total number of retries handled by workqueue |
| `workqueue_unfinished_work_seconds` | Unfinished work in seconds |
| `workqueue_longest_running_processor_seconds` | Longest running processor in seconds |

The following is a list of all the configurations the operators supports for metrics:

Expand Down