Create saved object for prometheus data-connection #10968

joshuali925 · 2025-11-24T19:14:55Z

Description

Prometheus will be stored as data-connection saved objects. Only connection id and MDS datasource id is saved.

        {
          "data-connection": {
            "connectionId": "my_prometheus",
            "type": "Prometheus"
          },
          "type": "data-connection",
          "references": [
            {
              "id": "02ced1b0-c95f-11f0-a5f2-dfc60f85a7e3",
              "type": "data-source",
              "name": "dataSource"
            }
          ],
          "updated_at": "2025-11-24T18:06:37.234Z"
        }

Underlying they are still in SQL, but OSD will use saved objects API to get Prometheus metadata. Users should only use OSD to create/delete prometheus connections if they use UI.

Additionally, this PR supports "No Auth" option when creating prometheus connections.

Issues Resolved

Screenshot

Testing the changes

Changelog

feat: Create saved object for prometheus data-connection

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: Joshua Li <[email protected]>

codecov · 2025-11-24T19:29:23Z

Codecov Report

❌ Patch coverage is 58.82353% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.77%. Comparing base (1929553) to head (6cbecce).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
...anagement/server/routes/data_connections_router.ts	61.76%	7 Missing and 6 partials ⚠️
...figuration/configure_direct_query_data_sources.tsx	57.14%	2 Missing and 4 partials ⚠️
.../data_source_management/public/components/utils.ts	50.00%	6 Missing ⚠️
...ation/prometheus/review_prometheus_data_source.tsx	0.00%	0 Missing and 2 partials ⚠️
...on/prometheus/configure_prometheus_data_source.tsx	75.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10968      +/-   ##
==========================================
- Coverage   60.79%   60.77%   -0.03%     
==========================================
  Files        4531     4532       +1     
  Lines      122238   122389     +151     
  Branches    20492    20542      +50     
==========================================
+ Hits        74310    74376      +66     
- Misses      42688    42760      +72     
- Partials     5240     5253      +13

Flag	Coverage Δ
Linux_1	`26.55% <0.00%> (-0.01%)`	⬇️
Linux_2	`38.92% <ø> (ø)`
Linux_3	`39.51% <58.82%> (?)`
Linux_4	`33.73% <0.00%> (-0.01%)`	⬇️
Windows_1	`26.56% <0.00%> (-0.01%)`	⬇️
Windows_2	`38.90% <ø> (ø)`
Windows_3	`39.50% <58.82%> (+<0.01%)`	⬆️
Windows_4	`33.73% <0.00%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

ps48

Have some minor comments.
One Question: How permissions are enforced on saved object operations with Datasource backend roles associated to the prometheus PPL datasource?

ps48 · 2025-11-25T21:54:56Z

...urces_components/direct_query_data_connection/manage_direct_query_data_connections_table.tsx

-          record.connectionType !== DataSourceConnectionType.OpenSearchConnection
+          record.connectionType !== DataSourceConnectionType.OpenSearchConnection &&
+          // Prometheus data-connections are shown on top level
+          record.type !== 'Prometheus'


Can we use DataConnectionType.Prometheus enum?

good point, thanks

ps48 · 2025-11-25T22:26:34Z

src/plugins/data_source/common/data_connections/data_connection_saved_object_attributes.ts

 export enum DataConnectionType {
  CloudWatch = 'AWS CloudWatch',
  SecurityLake = 'AWS Security Lake',
+  Prometheus = 'Prometheus',


Do we need Amazon Prometheus here?

let's leave it till later, this PR only modifies the existing prometheus type

ps48 · 2025-11-25T22:28:46Z

src/plugins/data_source_management/server/routes/data_connections_router.ts

+        if (statusCode !== 404) {
+          console.error('Issue in deleting data connection from backend:', error);
+          const errorBody = error.body ||
+            error.response || { message: error.message || 'Unknown error occurred' };


why is only 404 handled here?

It means SQL side data source is gone, and user is deleting the orphaned data-connection saved object. We should continue

If SQL is not working (5xx) i feel we should directly show error to user and not touch saved objects

ps48 · 2025-11-25T22:36:15Z

src/plugins/data_source_management/server/routes/data_connections_router.ts

+        const dataConnectionsresponse = await client('ppl.createDataSource', {
+          body: {
+            name: request.body.name,
+            connector: request.body.connector,
+            allowedRoles: request.body.allowedRoles,
+            properties: request.body.properties,
+          },
        });
+
+        // Create data-connection saved object for Prometheus datasources
+        if (request.body.connector === 'prometheus') {
+          await context.core.savedObjects.client.create(
+            'data-connection',
+            {
+              connectionId: request.body.name,
+              type: DataConnectionType.Prometheus,
+              meta: JSON.stringify({ properties: request.body.properties }),
+            },
+            {
+              references: dataSourceMDSId
+                ? [{ id: dataSourceMDSId, type: 'data-source', name: 'dataSource' }]
+                : [],
+            }
+          );
+        }
+
+        return response.ok({ body: dataConnectionsresponse });


May be we can check if ppl datasource was created successfully and only then creation the prometheus saved object?

I believe if creation in SQL plugin failed, it goes into catch and skips the saved object creation

ps48 · 2025-11-25T22:41:12Z

src/plugins/data_source_management/public/components/utils.ts

+      if (selectedDataSource.objectType === DATA_CONNECTION_SAVED_OBJECT_TYPE) {
+        const savedObject = await savedObjectsClient.get(
+          DATA_CONNECTION_SAVED_OBJECT_TYPE,
+          selectedDataSource.id
+        );
+        const dataSourceMDSId = savedObject.references.find((r) => r.type === 'data-source')?.id;
+        return http.delete(
+          `${DATACONNECTIONS_BASE}/${selectedDataSource.title}/dataSourceMDSId=${
+            dataSourceMDSId || ''
+          }`
+        );
+      }
+      return deleteDataSourceById(selectedDataSource.id, savedObjectsClient);


Do we need to handle scenarios where one deletion fails either saved object or SQL and mention it to the user explicitly?

Some CTA for users as toast to understand what's the inconsistent state.

That should be existing logic when calling this utils. The utils only deletes and throws error up

OpenSearch-Dashboards/src/plugins/data_source_management/public/components/data_source_table/data_source_table.tsx

Lines 451 to 474 in f129a7e

deleteMultipleDataSources(savedObjects.client, selectedDataSources)

.then(() => {

setSelectedDataSources([]);

// Fetch data sources

fetchDataSources();

setConfirmDeleteVisible(false);

// Check if default data source is deleted or not.

// if yes, then set the first existing datasource as default datasource.

setDefaultDataSource();

})

.catch(() => {

handleDisplayToastMessage({

message: i18n.translate(

'dataSourcesManagement.dataSourceListing.deleteDataSourceFailMsg',

{

defaultMessage:

'Error occurred while deleting selected records for Data sources. Please try it again',

}

),

});

})

.finally(() => {

setIsDeleting(false);

});

ps48 · 2025-11-25T22:43:07Z

...nents/direct_query_data_source_configuration/prometheus/configure_prometheus_data_source.tsx

+            <EuiCompressedSelect
+              id="selectDataSource"
+              options={[
+                ...(!hideLocalCluster
+                  ? [{ value: LocalCluster.id, text: LocalCluster.label }]
+                  : []),
+                ...dataSources.map((ds) => ({ value: ds.id, text: ds.title || ds.id })),
+              ]}
+              value={currentDataSourceId}
+              onChange={(e) => setDataSourceIdForRequest(e.target.value)}
+              data-test-subj="dataSourceSelect"


Signed-off-by: Joshua Li <[email protected]>

Create saved object for prometheus data-connection

9c7fba0

Signed-off-by: Joshua Li <[email protected]>

joshuali925 requested review from TackAdam, paulstn, ruchidh and sumukhswamy as code owners November 24, 2025 19:15

github-actions bot added the distinguished-contributor label Nov 24, 2025

Changeset file for PR opensearch-project#10968 created/updated

8ca657d

ps48 reviewed Nov 25, 2025

View reviewed changes

address comments

2409bf7

Signed-off-by: Joshua Li <[email protected]>

joshuali925 marked this pull request as draft November 25, 2025 23:41

preserve original behavior if data_source is disabled

6cbecce

Signed-off-by: Joshua Li <[email protected]>

joshuali925 marked this pull request as ready for review November 26, 2025 00:44

ps48 approved these changes Nov 26, 2025

View reviewed changes

	deleteMultipleDataSources(savedObjects.client, selectedDataSources)
	.then(() => {
	setSelectedDataSources([]);
	// Fetch data sources
	fetchDataSources();
	setConfirmDeleteVisible(false);
	// Check if default data source is deleted or not.
	// if yes, then set the first existing datasource as default datasource.
	setDefaultDataSource();
	})
	.catch(() => {
	handleDisplayToastMessage({
	message: i18n.translate(
	'dataSourcesManagement.dataSourceListing.deleteDataSourceFailMsg',
	{
	defaultMessage:
	'Error occurred while deleting selected records for Data sources. Please try it again',
	}
	),
	});
	})
	.finally(() => {
	setIsDeleting(false);
	});

Create saved object for prometheus data-connection #10968

Are you sure you want to change the base?

Create saved object for prometheus data-connection #10968

Uh oh!

Conversation

joshuali925 commented Nov 24, 2025

Description

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

Uh oh!

codecov bot commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ps48 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Nov 24, 2025 •

edited

Loading