[OSPP] Observability optimization in Admin

[robocanic]

基于Grafana，探索metric，trace，log的联动。

1. metric 面板是基于dubbo-java的metric label，需要兼容dubbo-go的metric label
2. log面板在admin控制台的可视化
3. 三个指标的联动

更多要求参照：https://summer.ospp.ac.cn/org/prodetail/26cf40056?list=org&navpage=org

English version

Explore the correlation of metrics, traces, and logs using Grafana.
The metrics dashboard currently uses metric labels from Dubbo Java and should also support those from Dubbo Go.
Visualize logs in the Dubbo Admin console.
Enable correlation among metrics, traces, and logs.
For additional requirements, please refer to:
https://summer.ospp.ac.cn/org/prodetail/26cf40056?list=org&navpage=org

[dodjdnh]

Hi @robocanic @chickenlj @nzomkxia，

针对本 Issue 里基于 Grafana 做 metric / trace / log 联动、补齐 log 面板，以及兼容 dubbo-go metric label 这几项要求，我在 #1537 写过一版较完整的提案。落地时我准备按下面五层来做增量扩展，而不是另起一套可观测平台。

┌─────────────────────────────────────────────────────────────┐
│ UI Layer (Vue3)                                             │
│ GrafanaPage / LogPanel / CorrelationContext / Metric Panel  │
└────────────────────────────┬────────────────────────────────┘
                             │ /api/v1
┌────────────────────────────▼────────────────────────────────┐
│ API Layer (Gin)                                             │
│ /logs/query  /logs/health  /log-dashboard  /metric-profiles │
└────────────────────────────┬────────────────────────────────┘
                             │
┌────────────────────────────▼────────────────────────────────┐
│ Application Layer                                           │
│ LogService / CorrelationService / MetricCompatibilityService│
└────────────────────────────┬────────────────────────────────┘
                             │
┌────────────────────────────▼────────────────────────────────┐
│ Adapter Layer                                               │
│ Loki Adapter / ES Adapter / Metric Label Mapper             │
└────────────────────────────┬────────────────────────────────┘
                             │
┌────────────────────────────▼────────────────────────────────┐
│ External Observability Stack                                │
│ Grafana / Prometheus / Loki / ElasticSearch                 │
└─────────────────────────────────────────────────────────────┘

1. UI Layer（Vue3）

• 继续复用 GrafanaPage.vue 承载 metric / trace / log 的 Grafana iframe
• 新增 LogPanel.vue：过滤条件、时间窗、日志列表、跳转入口
• 引入共享 CorrelationContext（可放 composable / provide-inject），在资源页各 tab 间传递：
  • mesh / application / service / instance / method
  • traceId / spanId?
  • start / end
• 应用、实例、服务详情增加 Logging tab：
  • views/resources/applications/tabs/logging.vue
  • views/resources/instances/tabs/logging.vue
  • views/resources/services/tabs/logging.vue
• Logging tab 内两块能力：
  • Grafana log dashboard（主可视化）
  • Admin 代理查询结果列表（便于按 traceId 过滤和一键跳转）
• Metric 兼容不新开独立产品页，优先在现有监控面板上消化 label 差异；必要时加轻量 profile 切换
• 前端 API 封装：ui-vue3/src/api/service/log.ts（queryLogs / getLogDashboard / getMetricProfiles）
• 前端不直连 Loki / ES / Prometheus，不持有下游凭据

2. API Layer（Gin）

在现有 /api/v1 上扩展，风格对齐已有 metric-dashboard / trace-dashboard / promQL：

• GET /api/v1/observability/logs/query
  • 入参对齐 LogQuery：provider、mesh、application、service、instance、method、traceId、keywords、start、end、limit、cursor
  • 出参走现有 CommonResp，data 为统一 LogResult
• GET /api/v1/observability/logs/health
  • 检查当前日志 provider 配置与连通性
• 维度 dashboard 路由（复用 GetGrafanaDashboard 的维度分发，扩展 DashboardType=log）：
  • GET /api/v1/application/log-dashboard
  • GET /api/v1/instance/log-dashboard
  • GET /api/v1/service/log-dashboard
• GET /api/v1/observability/metrics/profiles
  • 返回 dubbo-java / dubbo-go 的 label profile 与映射说明，供前端或 dashboard 变量选择
• 错误处理对齐现有 console：超时、下游 4xx/5xx、配置缺失分别转换；token / password / authorization 等字段返回前脱敏

3. Application Layer

把业务编排放在 console service，不让 handler 直接感知 LogQL / ES DSL：

• LogService
  • 读取 observability 配置，选择 Loki / ES / mock provider
  • 做查询参数校验、默认时间窗、limit 限流
  • 调用 adapter，并把结果归一成 LogResult
• CorrelationService
  • 维护关联上下文到 dashboard 变量、日志过滤条件的映射
  • 主路径：traceId + 时间窗 + 资源标识 强关联
  • 兜底：无 traceId 时按 application/service/instance + 时间窗弱关联
• MetricCompatibilityService
  • 基于现有 instance Prometheus exposition 解析能力做 profile 选择与 label 归一
  • 尽量让 Go / Java 走同一套 Grafana metric dashboard，而不是维护两套面板

对应落点：

• pkg/console/service/log.go
• pkg/console/service/correlation.go（或先内聚在 log/observability service 中，视体积再拆）
• pkg/console/service/metric_compatibility.go
• handler 只做绑定参数与 CommonResp 包装：pkg/console/handler/log.go 等

4. Adapter Layer

独立出适配层，便于单测和后续扩展：

pkg/observability/log/
  provider.go         # LogProvider / LogQuery / LogResult / LogEntry
  loki.go             # LogQL + Loki query_range
  elasticsearch.go    # Query DSL + _search
  normalize.go        # 字段归一化
  mock.go             # fixture

pkg/observability/metric/
  profile.go          # dubbojava / dubbogo profile
  mapper.go           # label mapping

• LokiAdapter：按 label selector 与时间窗构造 LogQL，解析 query_range
• ElasticsearchAdapter：按 bool filter/must 构造 Query DSL，解析 _search
• normalize：把两边差异收敛为统一 LogEntry（timestamp / level / message / application / service / instance / method / traceId / spanId / raw fields）
• Metric Label Mapper：归一常见差异（application/app、interface/service、method、instance/ip/pod 等），再用 fixture 回归，避免改坏现有 Java 面板

配置侧同步扩展：

• pkg/config/observability：新增 logs.loki / logs.elasticsearch（地址、超时、鉴权、默认索引或 label、启用开关）
• pkg/config/console：在 metricDashboards / traceDashboards 旁增加 logDashboards（application/instance/service）

5. External Observability Stack

外部系统保持职责不变，Admin 只做接入与关联：

• Grafana：metric / trace / log 面板可视化主入口
• Prometheus：现有 metric 查询与 instance exposition
• Loki / ElasticSearch：日志存储与检索后端
• Admin 不替换、不内嵌这些系统；也不在第一版做独立日志存储、告警或异常检测平台

提交计划

由于 OSPP 官方要求，以及这组改动涉及面较大，方向和社区确认前不会提前大规模改主干。若整体方向没有问题，按依赖拆成 4 个 PR：

1. PR-1 feat: add log providers and query API配置扩展 + Adapter 层（Loki / ES / mock LogProvider）+ /logs/query、/logs/health + 单测
2. PR-2 feat: add log dashboards for app/instance/service``log-dashboard 路由 + Logging tab + Grafana log 面板接入（应用 / 实例 / 服务）
3. PR-3 feat: correlate metric, trace and log panels``CorrelationContext 贯通 + Trace → Log → Metric 跳转
4. PR-4 feat: support dubbogo metric label profiles
   dubbo-go / dubbo-java metric profile 与 label mapping + fixture + 文档

已有交互原型用于验证跳转和面板形态；如需要可再补链接或录屏。更完整方案见 #1537。

补充两点准备情况：

• 已充分阅读社区开发规范与相关贡献文档，后续会按规范组织提交、测试与文档
• 10 月中旬才开学上课，前期可投入较完整的连续开发时间，保证按上述 PR 节奏推进

沟通现状

近期提案审阅较集中，也看到 #1537 里老师回复需要时间，完全理解。此前 19 日和 24 日通过邮件同步过方案和进展，也尝试加入社区讨论群，但目前还没有进一步反馈，所以把实现思路放到本 Issue 公开同步。若整体方向没有大问题，下一步就按上述拆分准备第一个 draft PR。

Related: #1537

感谢审阅。

---

English version

Hi @robocanic @chickenlj @nzomkxia,

For the requirements in this issue — correlating metrics, traces, and logs on top of Grafana, adding log panels in Admin, and supporting dubbo-go metric labels — I already posted a fuller proposal in #1537. For implementation, I plan to extend the current stack in five layers instead of introducing a separate observability platform.

┌─────────────────────────────────────────────────────────────┐
│ UI Layer (Vue3)                                             │
│ GrafanaPage / LogPanel / CorrelationContext / Metric Panel  │
└────────────────────────────┬────────────────────────────────┘
                             │ /api/v1
┌────────────────────────────▼────────────────────────────────┐
│ API Layer (Gin)                                             │
│ /logs/query  /logs/health  /log-dashboard  /metric-profiles │
└────────────────────────────┬────────────────────────────────┘
                             │
┌────────────────────────────▼────────────────────────────────┐
│ Application Layer                                           │
│ LogService / CorrelationService / MetricCompatibilityService│
└────────────────────────────┬────────────────────────────────┘
                             │
┌────────────────────────────▼────────────────────────────────┐
│ Adapter Layer                                               │
│ Loki Adapter / ES Adapter / Metric Label Mapper             │
└────────────────────────────┬────────────────────────────────┘
                             │
┌────────────────────────────▼────────────────────────────────┐
│ External Observability Stack                                │
│ Grafana / Prometheus / Loki / ElasticSearch                 │
└─────────────────────────────────────────────────────────────┘

1. UI Layer (Vue3)

• Keep using GrafanaPage.vue for metric / trace / log Grafana iframes
• Add LogPanel.vue for filters, time range, log list, and jump actions
• Introduce a shared CorrelationContext (composable / provide-inject) across resource tabs:
  • mesh / application / service / instance / method
  • traceId / spanId?
  • start / end
• Add Logging tabs on application / instance / service details:
  • views/resources/applications/tabs/logging.vue
  • views/resources/instances/tabs/logging.vue
  • views/resources/services/tabs/logging.vue
• Each Logging tab covers:
  • Grafana log dashboard as the main visualization
  • Admin-proxied query results for traceId filtering and one-click jumps
• Metric compatibility should stay on the existing monitor surface; add a light profile switch only if needed
• Frontend API wrapper: ui-vue3/src/api/service/log.ts (queryLogs / getLogDashboard / getMetricProfiles)
• The frontend will not talk to Loki / ES / Prometheus directly, and will not hold downstream credentials

2. API Layer (Gin)

Extend the existing /api/v1 surface in the same style as metric-dashboard / trace-dashboard / promQL:

• GET /api/v1/observability/logs/query
  • Input aligned with LogQuery: provider, mesh, application, service, instance, method, traceId, keywords, start, end, limit, cursor
  • Output uses the existing CommonResp, with data as a unified LogResult
• GET /api/v1/observability/logs/health
  • Check current log provider configuration and connectivity
• Dimension dashboard routes (reuse GetGrafanaDashboard dispatch, extend DashboardType=log):
  • GET /api/v1/application/log-dashboard
  • GET /api/v1/instance/log-dashboard
  • GET /api/v1/service/log-dashboard
• GET /api/v1/observability/metrics/profiles
  • Return dubbo-java / dubbo-go label profiles and mapping notes for frontend or dashboard variable selection
• Error handling stays consistent with the current console: timeouts, downstream 4xx/5xx, and missing config are mapped cleanly; token / password / authorization fields are redacted before response

3. Application Layer

Keep orchestration in console services so handlers do not construct LogQL / ES DSL directly:

• LogService
  • Read observability config and select Loki / ES / mock provider
  • Validate query params, default time window, and limit
  • Call adapters and normalize results into LogResult
• CorrelationService
  • Map correlation context to dashboard variables and log filters
  • Primary path: strong correlation by traceId + time window + resource identity
  • Fallback: weak correlation by application/service/instance + time window when traceId is absent
• MetricCompatibilityService
  • Build on the existing instance Prometheus exposition parsing for profile selection and label normalization
  • Prefer one shared Grafana metric dashboard for Go / Java instead of maintaining two panels

Likely code locations:

• pkg/console/service/log.go
• pkg/console/service/correlation.go (or keep it inside log/observability service first and split later if needed)
• pkg/console/service/metric_compatibility.go
• Handlers only bind params and wrap CommonResp, e.g. pkg/console/handler/log.go

4. Adapter Layer

Keep adapters isolated for unit testing and future providers:

pkg/observability/log/
  provider.go         # LogProvider / LogQuery / LogResult / LogEntry
  loki.go             # LogQL + Loki query_range
  elasticsearch.go    # Query DSL + _search
  normalize.go        # field normalization
  mock.go             # fixtures

pkg/observability/metric/
  profile.go          # dubbojava / dubbogo profiles
  mapper.go           # label mapping

• LokiAdapter: build LogQL from label selectors and time range, parse query_range
• ElasticsearchAdapter: build Query DSL with bool filter/must, parse _search
• normalize: converge both sides into a unified LogEntry (timestamp / level / message / application / service / instance / method / traceId / spanId / raw fields)
• Metric Label Mapper: normalize common differences (application/app, interface/service, method, instance/ip/pod, etc.), then regress with fixtures so existing Java panels stay intact

Config extensions:

• pkg/config/observability: add logs.loki / logs.elasticsearch (endpoint, timeout, auth, default index/labels, enable switch)
• pkg/config/console: add logDashboards next to metricDashboards / traceDashboards (application/instance/service)

5. External Observability Stack

External systems keep their current roles; Admin only integrates and correlates:

• Grafana: main visualization entry for metric / trace / log panels
• Prometheus: existing metric queries and instance exposition
• Loki / ElasticSearch: log storage and retrieval backends
• Admin will not replace or embed these systems, and the first version will not build a separate log storage, alerting, or anomaly-detection platform

PR plan

Because of OSPP process constraints and the size of this change, I will not start large trunk commits before the direction is confirmed with the community. If the overall approach looks fine, I propose splitting the work into 4 PRs:

1. PR-1 feat: add log providers and query APIConfig extension + adapter layer (Loki / ES / mock LogProvider) + /logs/query, /logs/health + unit tests
2. PR-2 feat: add log dashboards for app/instance/service``log-dashboard routes + Logging tabs + Grafana log dashboard integration (application / instance / service)
3. PR-3 feat: correlate metric, trace and log panelsWire CorrelationContext + Trace → Log → Metric navigation
4. PR-4 feat: support dubbogo metric label profiles
   dubbo-go / dubbo-java metric profiles and label mapping + fixtures + docs

I already have an interactive prototype for validating the jump flow and panel shape; I can share a link or recording if useful. The fuller write-up is in #1537.

Two additional readiness notes:

• I have gone through the community contribution guidelines and related docs, and will follow them for commits, tests, and documentation
• Classes do not start until mid-October, so I can commit continuous development time early on and keep the PR cadence above

Communication note

I understand proposal review is busy right now, and I also saw the note on #1537 that more time is needed. That is completely fine. I previously followed up by email on the 19th and 24th, and also tried joining the community discussion group, but have not received further feedback so far. I am posting the implementation plan here so we can align in public. If the direction looks good overall, I will prepare the first draft PR next.

Related: #1537

Thanks for reviewing.