[LIVY-1070] Add LivySessionMetrics Codahale gauges for session monitoring

[samuhasavak5]

What changes were proposed in this pull request?

This PR adds server-side session monitoring gauges for LIVY-1070.

Problem: Livy exposes session state via the REST API (/sessions, /batches) but does not publish session counts as Codahale metrics on the existing /metrics endpoint. External monitoring systems must poll the REST API to observe session distribution.

Solution: Introduce LivySessionMetrics, which registers session count gauges into Livy's MetricRegistry at server startup. Gauges are derived from InteractiveSessionManager and BatchSessionManager and exposed alongside existing Livy metrics via the AdminServlet at /metrics.

Changes

File | Change -- | -- LivySessionMetrics.scala | New class registering 18 Codahale gauges LivyServer.scala | Wire LivySessionMetrics after mountMetricsAdminServlet("/metrics") LivySessionMetricsSpec.scala | Unit tests (8 cases)

Metrics registered (18 gauges)

Overall (3)

• livy.sessions.total
• livy.sessions.active.total
• livy.sessions.terminal.total

Interactive (8)

• livy.sessions.interactive.total
• livy.sessions.interactive.{idle,busy,starting,shutting_down,dead,error,killed}

Batch (7)

• livy.sessions.batch.total
• livy.sessions.batch.{starting,running,success,dead,error,killed}

Design notes

• Additive only — no REST API or session lifecycle behavior changes
• Idempotent registration — skips gauge names already present in the registry
• Error-safe callbacks — gauge getValue returns 0 on exception
• State normalization — handles case and hyphen/underscore variants (e.g. shuttingdown, succeeded)
• HA note — gauge values reflect the local Livy server instance; in HA deployments only the leader holds active sessions

Compatibility

• No new endpoints; uses existing /metrics AdminServlet
• Backward compatible — new gauges appear alongside existing metrics

JIRA: https://issues.apache.org/jira/browse/LIVY-1070

How was this patch tested?

Build

mvn package -Pspark3 -Pscala-2.12 -pl server -am \

-s /tmp/livy-mvn-central-settings.xml -DskipTests

Result: BUILD SUCCESS

Unit tests

mvn test -Pspark3 -Pscala-2.12 -pl server \

-s /tmp/livy-mvn-central-settings.xml \

-Dsuites=org.apache.livy.server.LivySessionMetricsSpec

Result: 8/8 tests passed

Test | Coverage -- | -- All 18 gauge registrations | Registration Duplicate registration guard | Idempotency Zero sessions | Empty state Interactive sessions by state | State counting Batch sessions by state (incl. succeeded alias) | State counting Batch succeeded alias | Edge case Overall totals (total, active, terminal) | Aggregation Exception fallback returns 0 | Error handling

Code coverage

JaCoCo agent enabled during test run (server/target/jacoco/main.exec generated).

No UI changes in this PR.

Was this patch authored or co-authored using generative AI tooling?

Yes, this was co-authored using Cursor to help generate the new test cases.

[samuhasavak5]

@roczei , @gyogal , @nileshrathi345 and @ArnavBalyan
could you please take a look when possible and review

Thanks..!!!!

[roczei]

@samuhasavak5

Thanks for the PR! Tested it locally and was able to query the endpoint successfully at http://localhost:8998/metrics/metrics?pretty=true.

Could you also update docs/rest-api.md to cover the new metrics? It looks like the /metrics endpoint and the new gauge names aren't documented yet.

[samuhasavak5]

@samuhasavak5

Thanks for the PR! Tested it locally and was able to query the endpoint successfully at http://localhost:8998/metrics/metrics?pretty=true.

Could you also update docs/rest-api.md to cover the new metrics? It looks like the /metrics endpoint and the new gauge names aren't documented yet.

Hi @roczei
Thank you for testing and for the feedback.

I have updated docs/rest-api.md to document the /metrics/metrics endpoint
and the 18 livy.sessions.* session gauges (commit f6cdc95).

Please take another look when you have a moment.

[roczei]

Thanks for adding the documentation! LGTM