Skip to content

Commit 4bb67bd

Browse files
zhaohehuhupan3793
zhaohehuhu
and
pan3793
committed
[KYUUBI #5381] Change the default metrics reporter to Prometheus
### _Why are the changes needed?_ Close #5381 change default metrics reporter to prometheus since Kyuubi 1.8 ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request ### _Was this patch authored or co-authored using generative AI tooling?_ Closes #5344 from zhaohehuhu/Improvement-0928. Closes #5381 84f4c82 [hezhao2] reset METRICS_REPORTERS for test case b9ee5f7 [Cheng Pan] Update kyuubi-server/src/test/scala/org/apache/kyuubi/events/handler/ServerJsonLoggingEventHandlerSuite.scala 86165a6 [Cheng Pan] Update kyuubi-server/src/test/scala/org/apache/kyuubi/events/handler/ServerJsonLoggingEventHandlerSuite.scala a3605b6 [hezhao2] set METRICS_PROMETHEUS_PORT to 0 for test cases f1a4d28 [hezhao2] restore version number for kyuubi.metrics.reporters in doc dae40e1 [hezhao2] change default metrics reporter to prometheus Lead-authored-by: hezhao2 <[email protected]> Co-authored-by: Cheng Pan <[email protected]> Signed-off-by: Cheng Pan <[email protected]>
1 parent 1b229b6 commit 4bb67bd

File tree

4 files changed

+16
-11
lines changed

4 files changed

+16
-11
lines changed

‎docs/configuration/settings.md‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -352,16 +352,16 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
352352

353353
### Metrics
354354

355-
| Key | Default | Meaning | Type | Since |
356-
|---------------------------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------|
357-
| kyuubi.metrics.console.interval | PT5S | How often should report metrics to console | duration | 1.2.0 |
358-
| kyuubi.metrics.enabled | true | Set to true to enable kyuubi metrics system | boolean | 1.2.0 |
359-
| kyuubi.metrics.json.interval | PT5S | How often should report metrics to JSON file | duration | 1.2.0 |
360-
| kyuubi.metrics.json.location | metrics | Where the JSON metrics file located | string | 1.2.0 |
361-
| kyuubi.metrics.prometheus.path | /metrics | URI context path of prometheus metrics HTTP server | string | 1.2.0 |
362-
| kyuubi.metrics.prometheus.port | 10019 | Prometheus metrics HTTP server port | int | 1.2.0 |
363-
| kyuubi.metrics.reporters | JSON | A comma-separated list for all metrics reporters<ul> <li>CONSOLE - ConsoleReporter which outputs measurements to CONSOLE periodically.</li> <li>JMX - JmxReporter which listens for new metrics and exposes them as MBeans.</li> <li>JSON - JsonReporter which outputs measurements to json file periodically.</li> <li>PROMETHEUS - PrometheusReporter which exposes metrics in Prometheus format.</li> <li>SLF4J - Slf4jReporter which outputs measurements to system log periodically.</li></ul> | set | 1.2.0 |
364-
| kyuubi.metrics.slf4j.interval | PT5S | How often should report metrics to SLF4J logger | duration | 1.2.0 |
355+
| Key | Default | Meaning | Type | Since |
356+
|---------------------------------|------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------|
357+
| kyuubi.metrics.console.interval | PT5S | How often should report metrics to console | duration | 1.2.0 |
358+
| kyuubi.metrics.enabled | true | Set to true to enable kyuubi metrics system | boolean | 1.2.0 |
359+
| kyuubi.metrics.json.interval | PT5S | How often should report metrics to JSON file | duration | 1.2.0 |
360+
| kyuubi.metrics.json.location | metrics | Where the JSON metrics file located | string | 1.2.0 |
361+
| kyuubi.metrics.prometheus.path | /metrics | URI context path of prometheus metrics HTTP server | string | 1.2.0 |
362+
| kyuubi.metrics.prometheus.port | 10019 | Prometheus metrics HTTP server port | int | 1.2.0 |
363+
| kyuubi.metrics.reporters | PROMETHEUS | A comma-separated list for all metrics reporters<ul> <li>CONSOLE - ConsoleReporter which outputs measurements to CONSOLE periodically.</li> <li>JMX - JmxReporter which listens for new metrics and exposes them as MBeans.</li> <li>JSON - JsonReporter which outputs measurements to json file periodically.</li> <li>PROMETHEUS - PrometheusReporter which exposes metrics in Prometheus format.</li> <li>SLF4J - Slf4jReporter which outputs measurements to system log periodically.</li></ul> | set | 1.2.0 |
364+
| kyuubi.metrics.slf4j.interval | PT5S | How often should report metrics to SLF4J logger | duration | 1.2.0 |
365365

366366
### Operation
367367

‎docs/deployment/migration-guide.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,9 @@
2424
To restore previous behavior, set `kyuubi.metadata.store.jdbc.database.type=DERBY` and
2525
`kyuubi.metadata.store.jdbc.url=jdbc:derby:memory:kyuubi_state_store_db;create=true`.
2626

27+
* Since Kyuubi 1.8, PROMETHEUS is changed as the default metrics reporter. To restore previous behavior,
28+
set `kyuubi.metrics.reporters=JSON`.
29+
2730
## Upgrading from Kyuubi 1.7.1 to 1.7.2
2831

2932
* Since Kyuubi 1.7.2, for Kyuubi BeeLine, please use `--python-mode` option to run python code or script.

‎kyuubi-metrics/src/main/scala/org/apache/kyuubi/metrics/MetricsConf.scala‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ object MetricsConf {
4646
.transformToUpperCase
4747
.toSet()
4848
.checkValues(ReporterType)
49-
.createWithDefault(Set(JSON.toString))
49+
.createWithDefault(Set(PROMETHEUS.toString))
5050

5151
val METRICS_CONSOLE_INTERVAL: ConfigEntry[Long] = buildConf("kyuubi.metrics.console.interval")
5252
.doc("How often should report metrics to console")

‎kyuubi-server/src/test/scala/org/apache/kyuubi/events/handler/ServerJsonLoggingEventHandlerSuite.scala‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ import org.apache.kyuubi._
3434
import org.apache.kyuubi.client.util.BatchUtils._
3535
import org.apache.kyuubi.config.KyuubiConf
3636
import org.apache.kyuubi.events.ServerEventHandlerRegister
37+
import org.apache.kyuubi.metrics.MetricsConf
3738
import org.apache.kyuubi.operation.HiveJDBCTestHelper
3839
import org.apache.kyuubi.operation.OperationState._
3940
import org.apache.kyuubi.server.KyuubiServer
@@ -56,6 +57,7 @@ class ServerJsonLoggingEventHandlerSuite extends WithKyuubiServer with HiveJDBCT
5657
.set(KyuubiConf.SERVER_EVENT_JSON_LOG_PATH, serverLogRoot)
5758
.set(KyuubiConf.ENGINE_SPARK_EVENT_LOGGERS, Seq("JSON"))
5859
.set(KyuubiConf.ENGINE_EVENT_JSON_LOG_PATH, engineLogRoot)
60+
.set(MetricsConf.METRICS_REPORTERS, Set.empty[String])
5961
}
6062

6163
override protected def jdbcUrl: String = getJdbcUrl

0 commit comments

Comments
 (0)