Skip to content

chore(ci): migrate dataproc to new testing #4081

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
glasnt wants to merge 2 commits into from
Closed

chore(ci): migrate dataproc to new testing #4081

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
870ba80
chore(ci): migrate dataproc to new testing
glasnt Apr 29, 2025
b555671
move to less used region (resource quota)
glasnt Apr 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/config/nodejs-dev.jsonc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@
"datacatalog/quickstart",
"datacatalog/snippets",
"datalabeling",
"dataproc",
"dialogflow",
"discoveryengine",
"document-warehouse",
Expand Down
1 change: 0 additions & 1 deletion .github/config/nodejs.jsonc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,6 @@
"automl", // (untested) FAILED_PRECONDITION: Google Cloud AutoML Natural Language was retired on March 15, 2024. Please migrate to Vertex AI instead
"cloud-sql/sqlserver/tedious", // (untested) TypeError: The "config.server" property is required and must be of type string.
"compute", // GoogleError: The resource 'projects/long-door-651/zones/us-central1-a/disks/disk-from-pool-name' was not found
"dataproc", // GoogleError: Error submitting create cluster request: Multiple validation errors
"datastore/functions", // [ERR_REQUIRE_ESM]: require() of ES Module
"dialogflow-cx", // NOT_FOUND: com.google.apps.framework.request.NotFoundException: Agent 'undefined' does not exist
"dlp", // [ERR_REQUIRE_ESM]: require() of ES Module
Expand Down
52 changes: 0 additions & 52 deletions .github/workflows/dataproc.yaml
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions dataproc/system-test/createCluster.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,15 @@ const {describe, it, after} = require('mocha');
const cp = require('child_process');
const {v4} = require('uuid');

const region = 'us-central1';
const region = 'us-west1';
const clusterName = `node-cc-test-${v4()}`;

const dataproc = require('@google-cloud/dataproc');
const clusterClient = new dataproc.v1.ClusterControllerClient({
apiEndpoint: `${region}-dataproc.googleapis.com`,
});

const projectId = process.env.GCLOUD_PROJECT;
const projectId = process.env.PROJECT_ID;

const execSync = cmd =>
cp.execSync(cmd, {
Expand Down
2 changes: 1 addition & 1 deletion dataproc/system-test/instantiateInlineWorkflowTemplate.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ const {assert} = require('chai');
const {describe, it} = require('mocha');
const cp = require('child_process');

const projectId = process.env.GCLOUD_PROJECT;
const projectId = process.env.PROJECT_ID;
const region = 'us-central1';

const execSync = cmd =>
Expand Down
2 changes: 1 addition & 1 deletion dataproc/system-test/quickstart.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ const myUuid = v4();
const region = 'us-central1';
const clusterName = `node-qs-test-${myUuid}`;
const bucketName = `node-dataproc-qs-test-${myUuid}`;
const projectId = process.env.GCLOUD_PROJECT;
const projectId = process.env.PROJECT_ID;
const jobFileName = 'sum.py';
const jobFilePath = `gs://${bucketName}/${jobFileName}`;
const sortCode =
Expand Down
2 changes: 1 addition & 1 deletion dataproc/system-test/submitJob.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ const {describe, it, before, after} = require('mocha');
const cp = require('child_process');
const {v4} = require('uuid');

const projectId = process.env.GCLOUD_PROJECT;
const projectId = process.env.PROJECT_ID;
const region = 'us-central1';
const clusterName = `node-sj-test-${v4()}`;
const cluster = {
Expand Down
Loading