diff --git a/roads_management_insights/rmi_cli_tools/README.md b/roads_management_insights/rmi_cli_tools/README.md new file mode 100644 index 0000000..6e019cb --- /dev/null +++ b/roads_management_insights/rmi_cli_tools/README.md @@ -0,0 +1,126 @@ +# RMI Command-line Tools + +This tool provides scriptable access to the Roads Management Insights (RMI) service. It includes ready-to-use workflows and standalone API clients designed for easy integration into data pipelines and development environments. + +## Architecture & Workflow + +The following diagram illustrates how these scripts interact with both Google-managed and customer-managed projects: + +```mermaid +graph TD + subgraph Google ["Google-Managed Project (maps-platform-roads-management)"] + DX[Analytics Hub Data Exchange] + TOPIC[Pub/Sub Topics] + end + + subgraph RMI_PROJ ["Customer RMI Project"] + RSA[Roads Selection API] + S1[project_rmi_setup.sh] + S3[rmi_verify_api_access.sh] + end + + subgraph DATA_PROJ ["Customer Data Project"] + BQ[BigQuery Linked Dataset] + SUB[Pub/Sub Subscription] + S2[project_cloud_setup.sh] + end + + S1 -- "Configures" --> RSA + S3 -- "Tests CRUD on" --> RSA + + DX -- "Subscribe" --> BQ + TOPIC -- "Subscribe" --> SUB + + S2 -- "Configures APIs & Subscriptions" --> DATA_PROJ + + subgraph Operations ["Maintenance & Security"] + S4[rmi_data_backup.sh] + S5[project_optional_vpcsc.sh] + end + + S4 -- "Backs up from" --> RSA + S4 -- "Copies to" --> BQ + S5 -- "Generates Rules for" --> Google + S5 -- "Generates Rules for" --> DATA_PROJ +``` + +## Getting Started + +1. **Prerequisites**: + * Ensure the [Google Cloud CLI (`gcloud`)](https://cloud.google.com/sdk/docs/install) is installed and in your PATH. + * Ensure `jq` is installed for JSON processing. + +2. **Authenticate**: + ```bash + gcloud auth application-default login + ``` + +3. **Make Executable**: + ```bash + chmod +x clients/*.sh scenarios/*.sh + ``` + +## Project Awareness + +A typical RMI implementation involves two distinct Google Cloud projects. Throughout this tool and its documentation, we intentionally distinguish between them to ensure operations are performed in the correct context: + +1. **RMI Project (`PROJECT_RMI_ID`)**: This is the project where you enable the **Roads Selection API** and register your **Routes of Interest**. It acts as the "Control Plane" for your RMI configuration. +2. **Data Project (`PROJECT_CLOUD_ID`)**: This is the project where data is accumulated and consumed. It contains your **BigQuery linked datasets** (via Analytics Hub) and **Pub/Sub subscriptions** for real-time streams. + +*Note: While these can be the same project, RMI's architecture allows them to be separate to satisfy different security, billing, or organizational requirements. Always verify you are using the intended Project ID for each command.* + +--- + +## 1. Functional Scenarios (Recommended for Setup) + +The most common tasks are bundled into "Scenarios" in the `scenarios/` directory. These scripts are modular; run them with `all` for a full run, or call individual `step_` functions for granular control. Call any script without arguments to see detailed usage. + +### `project_rmi_setup.sh` +Configures the primary Google Cloud project for RMI API access. +* **Action:** Enables APIs and grants `roles/roads.roadsSelectionAdmin`. +* **Usage:** `./scenarios/project_rmi_setup.sh all [MEMBER]` + +### `project_cloud_setup.sh` +Configures the project used for data accumulation (BigQuery) and real-time streams (Pub/Sub). +* **Action:** Enables BigQuery/Analytics Hub APIs and provides modular helpers for subscriptions. +* **Usage:** `./scenarios/project_cloud_setup.sh all [MEMBER]` + +### `rmi_verify_api_access.sh` +Performs a complete CRUD (Create, Read, List, Delete) cycle to verify API connectivity. +* **Action:** Creates a temporary route, verifies it, and deletes it. +* **Usage:** `./scenarios/rmi_verify_api_access.sh all ` + +### `rmi_data_backup.sh` +Manages the preservation of your preview data. +* **Action:** Backs up `SelectedRoutes` to JSON and copies BigQuery tables to a destination dataset or GCS. +* **Usage:** `./scenarios/rmi_data_backup.sh all ` + +### `project_optional_vpcsc.sh` +Generates YAML snippets for VPC Service Controls. +* **Action:** Creates Ingress and Egress rules to allow data flow through service perimeters. +* **Usage:** `./scenarios/project_optional_vpcsc.sh all ` + +--- + +## 2. Standalone API Clients + +For custom integrations, use the bundled building blocks in the `clients/` directory. + +### Core Clients (Wraps raw API 1-to-1) +* **`analyticshub_v1.sh`**: Data Exchanges and Listings. +* **`roadsselection_v1.sh`**: Roads Selection API. +* **`routes_v2.sh`**: Directions and Distance Matrix. + +### Utility Clients (Pagination helpers) +* **`analyticshub_v1_util.sh`** +* **`roadsselection_v1_util.sh`** + +*Example (Sourcing a Utility):* +```bash +source clients/roadsselection_v1_util.sh +# List all routes in your project +roadsselection_v1_projects_selectedRoutes_list_all "projects/$PROJECT_RMI_ID" "100" "$PROJECT_RMI_ID" +``` + +## License +Apache 2.0 diff --git a/roads_management_insights/rmi_cli_tools/clients/analyticshub_v1.sh b/roads_management_insights/rmi_cli_tools/clients/analyticshub_v1.sh new file mode 100755 index 0000000..0da8b7f --- /dev/null +++ b/roads_management_insights/rmi_cli_tools/clients/analyticshub_v1.sh @@ -0,0 +1,415 @@ +#!/bin/bash +set -euo pipefail +# +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + + +############################################################################### +# Google API Client - Internal Utilities +# +# This script provides shared internal helper functions for authentication, +# request building, and API execution. It is intended to be sourced by +# specific service client scripts. +############################################################################### + +# ============================================================================= +# INTERNAL: Authentication & API Utilities +# ============================================================================= + +# Cache the access token to avoid calling gcloud repeatedly. +_API_CLIENT_ACCESS_TOKEN="" + +# _get_access_token ensures we only call gcloud once per script execution. +_get_access_token() { + local project_id="${1:-""}" + local gcloud_args=() + + if [[ -n "${project_id}" ]]; then + gcloud_args+=("--project"="${project_id}") + fi + + if [[ -z "${_API_CLIENT_ACCESS_TOKEN}" ]]; then + if ! command -v gcloud &> /dev/null; then + echo "Error: gcloud CLI is not installed or not in PATH." >&2 + return 1 + fi + _API_CLIENT_ACCESS_TOKEN=$(gcloud auth application-default print-access-token "${gcloud_args[@]}" 2>/dev/null) + fi + echo "${_API_CLIENT_ACCESS_TOKEN}" +} +# _build_query_params safely constructs a URL query string. +# +# @param ... Key-value pairs (e.g., "pageSize=10" "pageToken=abc"). +# Only pairs with a non-empty value are included. +_build_query_params() { + local params=() + for arg in "$@"; do + local key="${arg%%=*}" + local value="${arg#*=}" + if [[ -n "$value" ]]; then + # Note: For production use, value should be URL-encoded. + params+=("${key}=${value}") + fi + done + + if ((${#params[@]} > 0)); then + ( + # Join the array with '&' + IFS='&' + echo "?${params[*]}" + ) + else + echo "" + fi +} + + +# _call_api is a helper function to make authenticated API calls. +# +# @param string method The HTTP method (e.g., GET, POST, DELETE). +# @param string url The full URL for the API endpoint. +# @param string body Optional. The request body as a JSON string. +# @param string project_id Optional. Project ID for quota and billing. +# @param string additional_headers Optional. A newline-separated string of additional headers. +_call_api() { + local method="$1" + local url="$2" + local body="${3:-""}" + local project_id="${4:-""}" + local additional_headers="${5:-""}" + local access_token + + # Log the API call to stderr (captured by logs) + echo "[API CALL] $method $url" >&2 + if [[ -n "${body}" ]]; then + echo "[API BODY] ${body}" >&2 + fi + + access_token=$(_get_access_token "${project_id}") + + if [[ -z "${access_token}" ]]; then + echo "Error: Not authenticated. Please run 'gcloud auth application-default login'." >&2 + return 1 + fi + + local curl_args=("--silent" "--show-error" "--request" "${method}") + curl_args+=("--header" "Authorization: Bearer ${access_token}") + + # Add the quota project header if a project_id was provided. + if [[ -n "${project_id}" ]]; then + curl_args+=("--header" "X-Goog-User-Project: ${project_id}") + fi + + # Parse and add additional headers + if [[ -n "${additional_headers}" ]]; then + local OLD_IFS="$IFS" + IFS=$'\n' + for header in ${additional_headers}; do + curl_args+=("--header" "${header}") + done + IFS="$OLD_IFS" + fi + + if [[ -n "${body}" ]]; then + curl_args+=("--header" "Content-Type: application/json") + curl_args+=("--data" "${body}") + fi + + curl "${curl_args[@]}" "${url}" +} + + +############################################################################### +# Analytics Hub API (v1) - JSON Helpers +# +# This script provides helper functions to construct JSON request bodies +# for the Analytics Hub API v1. +############################################################################### + +# Creates a JSON object for a DataExchange. +# @param string display_name Required. +# @param string description Optional. +# @param string primary_contact Optional. +# @param string documentation Optional. +create_data_exchange() { + local display_name="$1" + local description="${2:-null}" + local primary_contact="${3:-null}" + local documentation="${4:-null}" + + jq -c --null-input \ + --arg dn "$display_name" \ + --arg desc "$description" \ + --arg pc "$primary_contact" \ + --arg doc "$documentation" \ + '{displayName: $dn, description: $desc, primaryContact: $pc, documentation: $doc} | del(..|select(. == "null"))' +} + +# Creates a JSON object for a BigQueryDatasetSource. +# @param string dataset_resource Required. Format: projects/{project}/datasets/{dataset} +create_bigquery_dataset_source() { + jq -c --null-input --arg ds "$1" '{dataset: $ds}' +} + +# Creates a JSON object for a Listing. +# @param string display_name Required. +# @param string bigquery_dataset_source_json Required. JSON object. +# @param string description Optional. +# @param string primary_contact Optional. +# @param string documentation Optional. +create_listing() { + local display_name="$1" + local bq_source="$2" + local description="${3:-null}" + local primary_contact="${4:-null}" + local documentation="${5:-null}" + + jq -c --null-input \ + --arg dn "$display_name" \ + --argjson bqs "$bq_source" \ + --arg desc "$description" \ + --arg pc "$primary_contact" \ + --arg doc "$documentation" \ + '{displayName: $dn, bigQueryDataset: $bqs, description: $desc, primaryContact: $pc, documentation: $doc} | del(..|select(. == "null"))' +} + +# Creates a JSON object for DestinationDatasetReference. +# @param string project_id Required. +# @param string dataset_id Required. +create_destination_dataset_reference() { + jq -c --null-input --arg pid "$1" --arg did "$2" \ + '{projectId: $pid, datasetId: $did}' +} + +# Creates a JSON object for DestinationDataset. +# @param string dataset_reference_json Required. +# @param string location Required. +# @param string description Optional. +# @param string friendly_name Optional. +create_destination_dataset() { + local ref="$1" + local loc="$2" + local desc="${3:-null}" + local fname="${4:-null}" + + jq -c --null-input \ + --argjson ref "$ref" \ + --arg loc "$loc" \ + --arg desc "$desc" \ + --arg fname "$fname" \ + '{datasetReference: $ref, location: $loc, description: $desc, friendlyName: $fname} | del(..|select(. == "null"))' +} + +# Creates a JSON object for SubscribeListingRequest. +# @param string destination_dataset_json Required. +create_subscribe_listing_request() { + jq -c --null-input --argjson dd "$1" \ + '{destinationDataset: $dd}' +} + +# Creates a JSON object for SetIamPolicyRequest. +# @param string policy_json Required. +# @param string update_mask Optional. +create_set_iam_policy_request() { + local policy="$1" + local mask="${2:-null}" + jq -c --null-input --argjson p "$policy" --arg m "$mask" \ + '{policy: $p, updateMask: $m} | del(..|select(. == "null"))' +} + +# Creates a JSON object for RefreshSubscriptionRequest. +create_refresh_subscription_request() { + echo "{}" +} + +# Creates a JSON object for RevokeSubscriptionRequest. +create_revoke_subscription_request() { + echo "{}" +} + +# Creates a JSON object for SubscribeDataExchangeRequest. +# @param string destination_dataset_json Required. +create_subscribe_data_exchange_request() { + jq -c --null-input --argjson dd "$1" \ + '{destinationDataset: $dd}' +} + +# Creates a JSON object for TestIamPermissionsRequest. +# @param string permissions_array_json Required. e.g. ["bigquery.datasets.get"] +create_test_iam_permissions_request() { + jq -c --null-input --argjson p "$1" \ + '{permissions: $p}' +} + + +############################################################################### +# Analytics Hub API (v1) Client +# +# This script provides a client for the Analytics Hub API v1. +# +# Discovery Doc Revision: 20260125 +############################################################################### + +# Resolve the directory of this script to locate internal helpers. +_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +# Base URL for the Analytics Hub API +ANALYTICSHUB_V1_BASE_URL="https://analyticshub.googleapis.com/v1" + +# --- Data Exchanges --- + +# Lists all data exchanges in a given project and location. +# +# @param string parent Required. Format: projects/{project}/locations/{location} +# @param integer page_size Optional. +# @param string page_token Optional. +# @param string project_id Optional. +analyticshub_v1_projects_locations_dataExchanges_list() { + local parent="$1" + local page_size="${2:-""}" + local page_token="${3:-""}" + local project_id="${4:-""}" + + local query_args=() + if [[ -n "${page_size}" ]]; then query_args+=("pageSize=${page_size}"); fi + if [[ -n "${page_token}" ]]; then query_args+=("pageToken=${page_token}"); fi + + local query_params + query_params=$(_build_query_params "${query_args[@]}") + + local url="${ANALYTICSHUB_V1_BASE_URL}/${parent}/dataExchanges${query_params}" + _call_api "GET" "${url}" "" "${project_id}" +} + +# Creates a new data exchange. +# +# @param string parent Required. Format: projects/{project}/locations/{location} +# @param string request_body Required. DataExchange JSON. +# @param string data_exchange_id Required. The ID of the data exchange. +# @param string project_id Optional. +analyticshub_v1_projects_locations_dataExchanges_create() { + local parent="$1" + local request_body="$2" + local data_exchange_id="$3" + local project_id="${4:-""}" + + local query_params + query_params=$(_build_query_params "dataExchangeId=${data_exchange_id}") + + local url="${ANALYTICSHUB_V1_BASE_URL}/${parent}/dataExchanges${query_params}" + _call_api "POST" "${url}" "${request_body}" "${project_id}" +} + +# Gets the details of a data exchange. +# +# @param string name Required. Format: projects/{project}/locations/{location}/dataExchanges/{data_exchange} +# @param string project_id Optional. +analyticshub_v1_projects_locations_dataExchanges_get() { + local name="$1" + local project_id="${2:-""}" + local url="${ANALYTICSHUB_V1_BASE_URL}/${name}" + _call_api "GET" "${url}" "" "${project_id}" +} + +# Deletes an existing data exchange. +# +# @param string name Required. Format: projects/{project}/locations/{location}/dataExchanges/{data_exchange} +# @param string project_id Optional. +analyticshub_v1_projects_locations_dataExchanges_delete() { + local name="$1" + local project_id="${2:-""}" + local url="${ANALYTICSHUB_V1_BASE_URL}/${name}" + _call_api "DELETE" "${url}" "" "${project_id}" +} + +# --- Listings --- + +# Lists all listings in a given project and location. +# +# @param string parent Required. Format: projects/{project}/locations/{location}/dataExchanges/{data_exchange} +# @param integer page_size Optional. +# @param string page_token Optional. +# @param string project_id Optional. +analyticshub_v1_projects_locations_dataExchanges_listings_list() { + local parent="$1" + local page_size="${2:-""}" + local page_token="${3:-""}" + local project_id="${4:-""}" + + local query_args=() + if [[ -n "${page_size}" ]]; then query_args+=("pageSize=${page_size}"); fi + if [[ -n "${page_token}" ]]; then query_args+=("pageToken=${page_token}"); fi + + local query_params + query_params=$(_build_query_params "${query_args[@]}") + + local url="${ANALYTICSHUB_V1_BASE_URL}/${parent}/listings${query_params}" + _call_api "GET" "${url}" "" "${project_id}" +} + +# Creates a new listing. +# +# @param string parent Required. Format: projects/{project}/locations/{location}/dataExchanges/{data_exchange} +# @param string request_body Required. Listing JSON. +# @param string listing_id Required. The ID of the listing. +# @param string project_id Optional. +analyticshub_v1_projects_locations_dataExchanges_listings_create() { + local parent="$1" + local request_body="$2" + local listing_id="$3" + local project_id="${4:-""}" + + local query_params + query_params=$(_build_query_params "listingId=${listing_id}") + + local url="${ANALYTICSHUB_V1_BASE_URL}/${parent}/listings${query_params}" + _call_api "POST" "${url}" "${request_body}" "${project_id}" +} + +# Gets the details of a listing. +# +# @param string name Required. Format: projects/{project}/locations/{location}/dataExchanges/{data_exchange}/listings/{listing} +# @param string project_id Optional. +analyticshub_v1_projects_locations_dataExchanges_listings_get() { + local name="$1" + local project_id="${2:-""}" + local url="${ANALYTICSHUB_V1_BASE_URL}/${name}" + _call_api "GET" "${url}" "" "${project_id}" +} + +# Deletes a listing. +# +# @param string name Required. Format: projects/{project}/locations/{location}/dataExchanges/{data_exchange}/listings/{listing} +# @param string project_id Optional. +analyticshub_v1_projects_locations_dataExchanges_listings_delete() { + local name="$1" + local project_id="${2:-""}" + local url="${ANALYTICSHUB_V1_BASE_URL}/${name}" + _call_api "DELETE" "${url}" "" "${project_id}" +} + +# Subscribes to a listing. +# +# @param string name Required. Format: projects/{project}/locations/{location}/dataExchanges/{data_exchange}/listings/{listing} +# @param string request_body Required. SubscribeListingRequest JSON. +# @param string project_id Optional. +analyticshub_v1_projects_locations_dataExchanges_listings_subscribe() { + local name="$1" + local request_body="$2" + local project_id="${3:-""}" + local url="${ANALYTICSHUB_V1_BASE_URL}/${name}:subscribe" + _call_api "POST" "${url}" "${request_body}" "${project_id}" +} diff --git a/roads_management_insights/rmi_cli_tools/clients/analyticshub_v1_util.sh b/roads_management_insights/rmi_cli_tools/clients/analyticshub_v1_util.sh new file mode 100755 index 0000000..60568b6 --- /dev/null +++ b/roads_management_insights/rmi_cli_tools/clients/analyticshub_v1_util.sh @@ -0,0 +1,88 @@ +#!/bin/bash +set -euo pipefail +# +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +############################################################################### +# UTILITY SCRIPT FOR: analyticshub_v1 +############################################################################### + +# Source the main client script +_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +source "${_SCRIPT_DIR}/analyticshub_v1.sh" + + +############################################################################### +# Analytics Hub API (v1) - Utility Functions +############################################################################### + +# Lists all Data Exchanges by automatically handling pagination. +# +# Output: Stream of DataExchange resources in JSONL (Newline Delimited JSON) format. +# +# @param string parent Required. +# @param integer page_size Optional. +# @param string project_id Optional. +analyticshub_v1_projects_locations_dataExchanges_list_all() { + local parent="$1" + local page_size="${2:-""}" + local project_id="${3:-""}" + local next_page_token="" + + while true; do + local response + response=$(analyticshub_v1_projects_locations_dataExchanges_list "${parent}" "${page_size}" "${next_page_token}" "${project_id}") + + if echo "$response" | jq --exit-status '.error' > /dev/null 2>&1; then + echo "API Error: $(echo "$response" | jq --compact-output '.error')" >&2 + return 1 + fi + + echo "$response" | jq -c '.dataExchanges[]? // empty' + + next_page_token=$(echo "$response" | jq -r '.nextPageToken // empty') + if [[ -z "${next_page_token}" ]]; then break; fi + done +} + +# Lists all Listings by automatically handling pagination. +# +# Output: Stream of Listing resources in JSONL (Newline Delimited JSON) format. +# +# @param string parent Required. +# @param integer page_size Optional. +# @param string project_id Optional. +analyticshub_v1_projects_locations_dataExchanges_listings_list_all() { + local parent="$1" + local page_size="${2:-""}" + local project_id="${3:-""}" + local next_page_token="" + + while true; do + local response + response=$(analyticshub_v1_projects_locations_dataExchanges_listings_list "${parent}" "${page_size}" "${next_page_token}" "${project_id}") + + if echo "$response" | jq --exit-status '.error' > /dev/null 2>&1; then + echo "API Error: $(echo "$response" | jq --compact-output '.error')" >&2 + return 1 + fi + + echo "$response" | jq -c '.listings[]? // empty' + + next_page_token=$(echo "$response" | jq -r '.nextPageToken // empty') + if [[ -z "${next_page_token}" ]]; then break; fi + done +} diff --git a/roads_management_insights/rmi_cli_tools/clients/roadsselection_v1.sh b/roads_management_insights/rmi_cli_tools/clients/roadsselection_v1.sh new file mode 100755 index 0000000..5dd3b3c --- /dev/null +++ b/roads_management_insights/rmi_cli_tools/clients/roadsselection_v1.sh @@ -0,0 +1,310 @@ +#!/bin/bash +set -euo pipefail +# +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + + +############################################################################### +# Google API Client - Internal Utilities +# +# This script provides shared internal helper functions for authentication, +# request building, and API execution. It is intended to be sourced by +# specific service client scripts. +############################################################################### + +# ============================================================================= +# INTERNAL: Authentication & API Utilities +# ============================================================================= + +# Cache the access token to avoid calling gcloud repeatedly. +_API_CLIENT_ACCESS_TOKEN="" + +# _get_access_token ensures we only call gcloud once per script execution. +_get_access_token() { + local project_id="${1:-""}" + local gcloud_args=() + + if [[ -n "${project_id}" ]]; then + gcloud_args+=("--project"="${project_id}") + fi + + if [[ -z "${_API_CLIENT_ACCESS_TOKEN}" ]]; then + if ! command -v gcloud &> /dev/null; then + echo "Error: gcloud CLI is not installed or not in PATH." >&2 + return 1 + fi + _API_CLIENT_ACCESS_TOKEN=$(gcloud auth application-default print-access-token "${gcloud_args[@]}" 2>/dev/null) + fi + echo "${_API_CLIENT_ACCESS_TOKEN}" +} +# _build_query_params safely constructs a URL query string. +# +# @param ... Key-value pairs (e.g., "pageSize=10" "pageToken=abc"). +# Only pairs with a non-empty value are included. +_build_query_params() { + local params=() + for arg in "$@"; do + local key="${arg%%=*}" + local value="${arg#*=}" + if [[ -n "$value" ]]; then + # Note: For production use, value should be URL-encoded. + params+=("${key}=${value}") + fi + done + + if ((${#params[@]} > 0)); then + ( + # Join the array with '&' + IFS='&' + echo "?${params[*]}" + ) + else + echo "" + fi +} + + +# _call_api is a helper function to make authenticated API calls. +# +# @param string method The HTTP method (e.g., GET, POST, DELETE). +# @param string url The full URL for the API endpoint. +# @param string body Optional. The request body as a JSON string. +# @param string project_id Optional. Project ID for quota and billing. +# @param string additional_headers Optional. A newline-separated string of additional headers. +_call_api() { + local method="$1" + local url="$2" + local body="${3:-""}" + local project_id="${4:-""}" + local additional_headers="${5:-""}" + local access_token + + # Log the API call to stderr (captured by logs) + echo "[API CALL] $method $url" >&2 + if [[ -n "${body}" ]]; then + echo "[API BODY] ${body}" >&2 + fi + + access_token=$(_get_access_token "${project_id}") + + if [[ -z "${access_token}" ]]; then + echo "Error: Not authenticated. Please run 'gcloud auth application-default login'." >&2 + return 1 + fi + + local curl_args=("--silent" "--show-error" "--request" "${method}") + curl_args+=("--header" "Authorization: Bearer ${access_token}") + + # Add the quota project header if a project_id was provided. + if [[ -n "${project_id}" ]]; then + curl_args+=("--header" "X-Goog-User-Project: ${project_id}") + fi + + # Parse and add additional headers + if [[ -n "${additional_headers}" ]]; then + local OLD_IFS="$IFS" + IFS=$'\n' + for header in ${additional_headers}; do + curl_args+=("--header" "${header}") + done + IFS="$OLD_IFS" + fi + + if [[ -n "${body}" ]]; then + curl_args+=("--header" "Content-Type: application/json") + curl_args+=("--data" "${body}") + fi + + curl "${curl_args[@]}" "${url}" +} + + +############################################################################### +# Roads Selection API (v1) - JSON Helpers +# +# This script provides helper functions to construct JSON request bodies +# for the Roads Selection API. +############################################################################### + +# Creates a JSON object for a LatLng. +# @param float latitude +# @param float longitude +create_lat_lng() { + jq -c --null-input --argjson lat "$1" --argjson lon "$2" \ + '{latitude: $lat, longitude: $lon}' +} + +# Creates a JSON object for a DynamicRoute. +# @param string origin_json JSON object (LatLng) +# @param string destination_json JSON object (LatLng) +# @param string intermediates_json_array Optional. JSON array of LatLng objects. +create_dynamic_route() { + local origin="$1" + local destination="$2" + local intermediates="${3:-null}" + + jq -c --null-input \ + --argjson origin "$origin" \ + --argjson dest "$destination" \ + --argjson inter "$intermediates" \ + '{origin: $origin, destination: $dest, intermediates: $inter} | del(..|nulls)' +} + +# Creates a JSON object for a SelectedRoute. +# @param string dynamic_route_json JSON object (DynamicRoute) +# @param string display_name Optional. +# @param string route_attributes_json Optional. JSON object (map). +create_selected_route() { + local dynamic_route="$1" + local display_name="${2:-null}" + local route_attributes="${3:-null}" + + jq -c --null-input \ + --argjson dr "$dynamic_route" \ + --arg dn "$display_name" \ + --argjson ra "$route_attributes" \ + '{dynamicRoute: $dr, displayName: $dn, routeAttributes: $ra} | del(..|nulls) | if .displayName == "null" then del(.displayName) else . end' +} + +# Creates a JSON object for a CreateSelectedRouteRequest. +# @param string parent Required. Format: projects/{project} +# @param string selected_route_json Required. JSON object (SelectedRoute). +# @param string selected_route_id Optional. +create_create_selected_route_request() { + local parent="$1" + local selected_route="$2" + local selected_route_id="${3:-null}" + + jq -c --null-input \ + --arg p "$parent" \ + --argjson sr "$selected_route" \ + --arg srid "$selected_route_id" \ + '{parent: $p, selectedRoute: $sr, selectedRouteId: $srid} | del(..|nulls) | if .selectedRouteId == "null" then del(.selectedRouteId) else . end' +} + +# Creates a JSON object for a BatchCreateSelectedRoutesRequest. +# @param string requests_json_array Required. JSON array of CreateSelectedRouteRequest objects. +create_batch_create_selected_routes_request() { + local requests="$1" + + jq -c --null-input \ + --argjson reqs "$requests" \ + '{requests: $reqs}' +} + + +############################################################################### +# Roads Selection API (v1) Client +# +# This script provides a client for the Roads Selection API. +# It includes authentication helpers, request builders, and API methods. +# +# Discovery Doc Revision: 20260204 +############################################################################### + +# Resolve the directory of this script to locate internal helpers. +_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +# Base URL for the Roads Selection API +ROADS_SELECTION_V1_BASE_URL="https://roads.googleapis.com/selection/v1" + +# Creates multiple SelectedRoutes and starts a schedule. +# +# @param string parent Required. The project resource. Format: projects/{project} +# @param string request_body Required. The BatchCreateSelectedRoutesRequest JSON. +# @param string project_id Optional. Project ID for quota/billing. +roadsselection_v1_projects_selectedRoutes_batchCreate() { + local parent="$1" + local request_body="$2" + local project_id="${3:-}" + local url="${ROADS_SELECTION_V1_BASE_URL}/${parent}/selectedRoutes:batchCreate" + _call_api "POST" "${url}" "${request_body}" "${project_id}" +} + +# Creates a SelectedRoute and starts a schedule. +# +# @param string parent Required. The project resource. Format: projects/{project} +# @param string request_body Required. The SelectedRoute JSON. +# @param string selected_route_id Optional. The ID to use for the SelectedRoute. +# @param string project_id Optional. Project ID for quota/billing. +roadsselection_v1_projects_selectedRoutes_create() { + local parent="$1" + local request_body="$2" + local selected_route_id="${3:-}" + local project_id="${4:-}" + + local query_params="" + if [[ -n "${selected_route_id}" ]]; then + query_params=$(_build_query_params "selectedRouteId=${selected_route_id}") + fi + + local url="${ROADS_SELECTION_V1_BASE_URL}/${parent}/selectedRoutes${query_params}" + _call_api "POST" "${url}" "${request_body}" "${project_id}" +} + +# Deletes the specified SelectedRoute. +# +# @param string name Required. The name of the SelectedRoute to delete. +# @param string project_id Optional. Project ID for quota/billing. +roadsselection_v1_projects_selectedRoutes_delete() { + local name="$1" + local project_id="${2:-}" + local url="${ROADS_SELECTION_V1_BASE_URL}/${name}" + _call_api "DELETE" "${url}" "" "${project_id}" +} + +# Gets a SelectedRoute. +# +# @param string name Required. The name of the SelectedRoute to retrieve. +# @param string project_id Optional. Project ID for quota/billing. +roadsselection_v1_projects_selectedRoutes_get() { + local name="$1" + local project_id="${2:-}" + local url="${ROADS_SELECTION_V1_BASE_URL}/${name}" + _call_api "GET" "${url}" "" "${project_id}" +} + +# Lists all SelectedRoutes for the specified project. +# +# @param string parent Required. The parent project. Format: projects/{project}/selectedRoutes +# Note: Discovery doc says 'parent' is projects/{project}/selectedRoutes +# but usually list parent is just projects/{project}. +# Checking path: "selection/v1/{+parent}/selectedRoutes" +# This implies parent should be "projects/{project}" and the path appends "/selectedRoutes". +# However, the parameter pattern says "^projects/[^/]+$" which is projects/{project}. +# Wait, the discovery doc says: +# "path": "selection/v1/{+parent}/selectedRoutes" +# "parameter": "parent", "pattern": "^projects/[^/]+$" +# So 'parent' is indeed 'projects/my-project'. +# @param integer page_size Optional. +# @param string page_token Optional. +# @param string project_id Optional. Project ID for quota/billing. +roadsselection_v1_projects_selectedRoutes_list() { + local parent="$1" + local page_size="${2:-}" + local page_token="${3:-}" + local project_id="${4:-}" + + local query_args=() + if [[ -n "${page_size}" ]]; then query_args+=("pageSize=${page_size}"); fi + if [[ -n "${page_token}" ]]; then query_args+=("pageToken=${page_token}"); fi + + local query_params + query_params=$(_build_query_params "${query_args[@]}") + + local url="${ROADS_SELECTION_V1_BASE_URL}/${parent}/selectedRoutes${query_params}" + _call_api "GET" "${url}" "" "${project_id}" +} diff --git a/roads_management_insights/rmi_cli_tools/clients/roadsselection_v1_util.sh b/roads_management_insights/rmi_cli_tools/clients/roadsselection_v1_util.sh new file mode 100755 index 0000000..715eeb1 --- /dev/null +++ b/roads_management_insights/rmi_cli_tools/clients/roadsselection_v1_util.sh @@ -0,0 +1,68 @@ +#!/bin/bash +set -euo pipefail +# +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +############################################################################### +# UTILITY SCRIPT FOR: roadsselection_v1 +############################################################################### + +# Source the main client script +_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +source "${_SCRIPT_DIR}/roadsselection_v1.sh" + + +############################################################################### +# Roads Selection API (v1) - Utility Functions +# +# This script provides higher-level convenience functions for the Roads +# Selection API, such as automatic pagination for list methods. +############################################################################### + +# Lists all SelectedRoutes by automatically handling pagination. +# +# Output: Stream of SelectedRoute resources in JSONL (Newline Delimited JSON) format. +# +# @param string parent Required. The parent project. Format: projects/{project} +# @param integer page_size Optional. The number of results per page. +# @param string project_id Optional. Project ID for quota/billing. +roadsselection_v1_projects_selectedRoutes_list_all() { + local parent="$1" + local page_size="${2:-""}" + local project_id="${3:-""}" + local next_page_token="" + + while true; do + local response + response=$(roadsselection_v1_projects_selectedRoutes_list "${parent}" "${page_size}" "${next_page_token}" "${project_id}") + + # Check for API Error + if echo "$response" | jq --exit-status '.error' > /dev/null 2>&1; then + echo "API Error: $(echo "$response" | jq --compact-output '.error')" >&2 + return 1 + fi + + # Output the current page of routes (JSONL format) + echo "$response" | jq -c '.selectedRoutes[]? // empty' + + # Extract next page token + next_page_token=$(echo "$response" | jq -r '.nextPageToken // empty') + + if [[ -z "${next_page_token}" ]]; then + break + fi + done +} diff --git a/roads_management_insights/rmi_cli_tools/clients/routes_v2.sh b/roads_management_insights/rmi_cli_tools/clients/routes_v2.sh new file mode 100755 index 0000000..9a71474 --- /dev/null +++ b/roads_management_insights/rmi_cli_tools/clients/routes_v2.sh @@ -0,0 +1,322 @@ +#!/bin/bash +set -euo pipefail +# +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + + +############################################################################### +# Google API Client - Internal Utilities +# +# This script provides shared internal helper functions for authentication, +# request building, and API execution. It is intended to be sourced by +# specific service client scripts. +############################################################################### + +# ============================================================================= +# INTERNAL: Authentication & API Utilities +# ============================================================================= + +# Cache the access token to avoid calling gcloud repeatedly. +_API_CLIENT_ACCESS_TOKEN="" + +# _get_access_token ensures we only call gcloud once per script execution. +_get_access_token() { + local project_id="${1:-""}" + local gcloud_args=() + + if [[ -n "${project_id}" ]]; then + gcloud_args+=("--project"="${project_id}") + fi + + if [[ -z "${_API_CLIENT_ACCESS_TOKEN}" ]]; then + if ! command -v gcloud &> /dev/null; then + echo "Error: gcloud CLI is not installed or not in PATH." >&2 + return 1 + fi + _API_CLIENT_ACCESS_TOKEN=$(gcloud auth application-default print-access-token "${gcloud_args[@]}" 2>/dev/null) + fi + echo "${_API_CLIENT_ACCESS_TOKEN}" +} +# _build_query_params safely constructs a URL query string. +# +# @param ... Key-value pairs (e.g., "pageSize=10" "pageToken=abc"). +# Only pairs with a non-empty value are included. +_build_query_params() { + local params=() + for arg in "$@"; do + local key="${arg%%=*}" + local value="${arg#*=}" + if [[ -n "$value" ]]; then + # Note: For production use, value should be URL-encoded. + params+=("${key}=${value}") + fi + done + + if ((${#params[@]} > 0)); then + ( + # Join the array with '&' + IFS='&' + echo "?${params[*]}" + ) + else + echo "" + fi +} + + +# _call_api is a helper function to make authenticated API calls. +# +# @param string method The HTTP method (e.g., GET, POST, DELETE). +# @param string url The full URL for the API endpoint. +# @param string body Optional. The request body as a JSON string. +# @param string project_id Optional. Project ID for quota and billing. +# @param string additional_headers Optional. A newline-separated string of additional headers. +_call_api() { + local method="$1" + local url="$2" + local body="${3:-""}" + local project_id="${4:-""}" + local additional_headers="${5:-""}" + local access_token + + # Log the API call to stderr (captured by logs) + echo "[API CALL] $method $url" >&2 + if [[ -n "${body}" ]]; then + echo "[API BODY] ${body}" >&2 + fi + + access_token=$(_get_access_token "${project_id}") + + if [[ -z "${access_token}" ]]; then + echo "Error: Not authenticated. Please run 'gcloud auth application-default login'." >&2 + return 1 + fi + + local curl_args=("--silent" "--show-error" "--request" "${method}") + curl_args+=("--header" "Authorization: Bearer ${access_token}") + + # Add the quota project header if a project_id was provided. + if [[ -n "${project_id}" ]]; then + curl_args+=("--header" "X-Goog-User-Project: ${project_id}") + fi + + # Parse and add additional headers + if [[ -n "${additional_headers}" ]]; then + local OLD_IFS="$IFS" + IFS=$'\n' + for header in ${additional_headers}; do + curl_args+=("--header" "${header}") + done + IFS="$OLD_IFS" + fi + + if [[ -n "${body}" ]]; then + curl_args+=("--header" "Content-Type: application/json") + curl_args+=("--data" "${body}") + fi + + curl "${curl_args[@]}" "${url}" +} + + +############################################################################### +# Routes API (v2) - JSON Helpers +# +# This script provides helper functions to construct JSON request bodies +# for the Routes API v2. +############################################################################### + +# Creates a JSON object for a LatLng. +# @param float latitude +# @param float longitude +create_lat_lng() { + jq -c --null-input --argjson lat "$1" --argjson lon "$2" \ + '{latitude: $lat, longitude: $lon}' +} + +# Creates a JSON object for a Location. +# @param string lat_lng_json Required. +# @param integer heading Optional. +create_location() { + local lat_lng="$1" + local heading="${2:-null}" + + jq -c --null-input \ + --argjson ll "$lat_lng" \ + --argjson h "$heading" \ + '{latLng: $ll, heading: $h} | del(..|select(. == null))' +} + +# Creates a JSON object for a Waypoint. +# @param string location_json Optional. Location JSON. +# @param string place_id Optional. +# @param string address Optional. +# @param boolean via Optional. +create_waypoint() { + local location="${1:-null}" + local place_id="${2:-null}" + local address="${3:-null}" + local via="${4:-null}" + + jq -c --null-input \ + --argjson loc "$location" \ + --arg pid "$place_id" \ + --arg addr "$address" \ + --argjson via "$via" \ + '{location: $loc, placeId: $pid, address: $addr, via: $via} | del(..|select(. == "null" or . == null))' +} + +# Creates a JSON object for RouteModifiers. +# @param boolean avoid_tolls Optional. +# @param boolean avoid_highways Optional. +# @param boolean avoid_ferries Optional. +create_route_modifiers() { + local tolls="${1:-null}" + local highways="${2:-null}" + local ferries="${3:-null}" + + jq -c --null-input \ + --argjson tolls "$tolls" \ + --argjson highways "$highways" \ + --argjson ferries "$ferries" \ + '{avoidTolls: $tolls, avoidHighways: $highways, avoidFerries: $ferries} | del(..|select(. == null))' +} + +# Creates a JSON object for TransitPreferences. +# @param string allowed_travel_modes_array_json Optional. Array of strings (e.g. ["BUS", "SUBWAY"]). +# @param string routing_preference Optional. e.g. LESS_WALKING. +create_transit_preferences() { + local modes="${1:-null}" + local pref="${2:-null}" + + jq -c --null-input \ + --argjson modes "$modes" \ + --arg pref "$pref" \ + '{allowedTravelModes: $modes, routingPreference: $pref} | del(..|select(. == "null" or . == null))' +} + +# Creates a JSON object for a ComputeRoutesRequest. +# @param string origin_waypoint_json Required. +# @param string destination_waypoint_json Required. +# @param string travel_mode Optional. e.g. DRIVE, WALK, BICYCLE. +# @param string routing_preference Optional. e.g. TRAFFIC_AWARE, TRAFFIC_AWARE_OPTIMAL. +# @param string intermediates_array_json Optional. Array of Waypoint JSONs. +create_compute_routes_request() { + local origin="$1" + local destination="$2" + local travel_mode="${3:-null}" + local routing_pref="${4:-null}" + local intermediates="${5:-null}" + + jq -c --null-input \ + --argjson origin "$origin" \ + --argjson dest "$destination" \ + --arg tm "$travel_mode" \ + --arg rp "$routing_pref" \ + --argjson inter "$intermediates" \ + '{origin: $origin, destination: $dest, travelMode: $tm, routingPreference: $rp, intermediates: $inter} | del(..|select(. == "null" or . == null))' +} + +# Creates a JSON object for a RouteMatrixOrigin. +# @param string waypoint_json Required. +# @param string route_modifiers_json Optional. +create_route_matrix_origin() { + local waypoint="$1" + local modifiers="${2:-null}" + + jq -c --null-input \ + --argjson wp "$waypoint" \ + --argjson mod "$modifiers" \ + '{waypoint: $wp, routeModifiers: $mod} | del(..|select(. == null))' +} + +# Creates a JSON object for a RouteMatrixDestination. +# @param string waypoint_json Required. +create_route_matrix_destination() { + local waypoint="$1" + + jq -c --null-input \ + --argjson wp "$waypoint" \ + '{waypoint: $wp}' +} + +# Creates a JSON object for a ComputeRouteMatrixRequest. +# @param string origins_array_json Required. Array of RouteMatrixOrigin JSONs. +# @param string destinations_array_json Required. Array of RouteMatrixDestination JSONs. +# @param string travel_mode Optional. +# @param string routing_preference Optional. +create_compute_route_matrix_request() { + local origins="$1" + local destinations="$2" + local travel_mode="${3:-null}" + local routing_pref="${4:-null}" + + jq -c --null-input \ + --argjson origins "$origins" \ + --argjson destinations "$destinations" \ + --arg tm "$travel_mode" \ + --arg rp "$routing_pref" \ + '{origins: $origins, destinations: $destinations, travelMode: $tm, routingPreference: $rp} | del(..|select(. == "null" or . == null))' +} + + +############################################################################### +# Routes API (v2) Client +# +# This script provides a client for the Routes API v2. +# +# Discovery Doc Revision: 20260202 +############################################################################### + +# Resolve the directory of this script to locate internal helpers. +_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +# Base URL for the Routes API +ROUTES_V2_BASE_URL="https://routes.googleapis.com" + +# Returns the primary route along with optional alternate routes. +# +# @param string request_body Required. ComputeRoutesRequest JSON. +# @param string field_mask Required. Response field mask (e.g., "*"). +# @param string project_id Optional. +# @see https://developers.google.com/maps/documentation/routes/reference/rest/v2/directions/computeRoutes +routes_v2_computeRoutes() { + local request_body="$1" + local field_mask="${2:-"*"}" + local project_id="${3:-""}" + + local url="${ROUTES_V2_BASE_URL}/directions/v2:computeRoutes" + local headers="X-Goog-FieldMask: ${field_mask}" + + _call_api "POST" "${url}" "${request_body}" "${project_id}" "${headers}" +} + +# Takes in a list of origins and destinations and returns a stream of route information. +# +# @param string request_body Required. ComputeRouteMatrixRequest JSON. +# @param string field_mask Required. Response field mask. +# @param string project_id Optional. +# @see https://developers.google.com/maps/documentation/routes/reference/rest/v2/distanceMatrix/computeRouteMatrix +routes_v2_computeRouteMatrix() { + local request_body="$1" + local field_mask="${2:-"*"}" + local project_id="${3:-""}" + + local url="${ROUTES_V2_BASE_URL}/distanceMatrix/v2:computeRouteMatrix" + local headers="X-Goog-FieldMask: ${field_mask}" + + _call_api "POST" "${url}" "${request_body}" "${project_id}" "${headers}" +} diff --git a/roads_management_insights/rmi_cli_tools/scenarios/project_cloud_setup.sh b/roads_management_insights/rmi_cli_tools/scenarios/project_cloud_setup.sh new file mode 100755 index 0000000..dd10646 --- /dev/null +++ b/roads_management_insights/rmi_cli_tools/scenarios/project_cloud_setup.sh @@ -0,0 +1,289 @@ +#!/bin/bash +set -euo pipefail +# +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +############################################################################### +# Project Cloud Setup +# +# This script helps configure a Google Cloud project for accumulated data access. +# References: https://developers.google.com/maps/documentation/roads-management-insights/accumulated-data +############################################################################### + +# Locate and source the bundled utility script +_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +# This path works in the 'dist' directory structure +ANALYTICSHUB_UTIL="${_SCRIPT_DIR}/../clients/analyticshub_v1_util.sh" + +if [[ -f "$ANALYTICSHUB_UTIL" ]]; then + source "$ANALYTICSHUB_UTIL" +fi + +# Displays usage information and available commands for the script. +usage() { + echo "Usage: $0 [args]" + echo "" + echo "Commands:" + echo " all [MEMBER] Run setup and grant roles (e.g. user:email@addr)" + echo " step_enable_apis Enable BigQuery, Analytics Hub, Pub/Sub" + echo " step_verify_apis Verify APIs are enabled" + echo " step_configure_iam Grant required roles to a member" + echo "" + echo "IAM Helpers (MEMBER format: user:EMAIL, serviceAccount:EMAIL, group:EMAIL):" + echo " grant_roles_to_user " + echo " grant_roles_to_group " + echo " grant_roles_to_service_account " + echo "" + echo "Pub/Sub Helpers:" + echo " create_rmi_json_subscription [SUB_ID]" + echo " create_rmi_binary_subscription [SUB_ID]" + echo "" + echo "Analytics Hub Helpers:" + echo " subscribe_from_console_url [DATASET] [LOC]" + echo " subscribe_from_params [DATASET] [LOC]" +} + +# Enables the required Google Cloud APIs (BigQuery, Analytics Hub, Pub/Sub) for the data project. +step_enable_apis() { + local project_id="${1:-${PROJECT_CLOUD_ID:-}}" + if [[ -z "$project_id" ]]; then echo "Error: PROJECT_CLOUD_ID required"; exit 1; fi + echo "Enabling APIs for $project_id..." + gcloud services enable bigquery.googleapis.com --project "$project_id" + gcloud services enable analyticshub.googleapis.com --project "$project_id" + gcloud services enable pubsub.googleapis.com --project "$project_id" + echo "APIs enabled." +} + +# Verifies that the required APIs are successfully enabled in the project. +step_verify_apis() { + local project_id="${1:-${PROJECT_CLOUD_ID:-}}" + if [[ -z "$project_id" ]]; then echo "Error: PROJECT_CLOUD_ID required"; exit 1; fi + echo "Verifying enabled APIs for $project_id..." + gcloud services list --project "$project_id" --filter="config.name:bigquery.googleapis.com" + gcloud services list --project "$project_id" --filter="config.name:analyticshub.googleapis.com" + gcloud services list --project "$project_id" --filter="config.name:pubsub.googleapis.com" +} + +# Internal helper to grant a specific IAM role to a member. +_grant_role() { + local project_id="$1" + local member="$2" + local role="$3" + echo "Granting $role to $member on project $project_id..." + + if ! gcloud projects add-iam-policy-binding "$project_id" --member="$member" --role="$role" --quiet > /dev/null 2> /tmp/gcloud_error.log; then + local error_msg + error_msg=$(cat /tmp/gcloud_error.log) + if [[ "$error_msg" == *"PERMISSION_DENIED"* ]]; then + echo "Error: Permission Denied while granting $role." + echo "Your account may lack 'resourcemanager.projects.setIamPolicy' on project $project_id." + echo "Please ask a Project Owner or IAM Admin to run this command for you:" + echo " gcloud projects add-iam-policy-binding $project_id --member='$member' --role='$role'" + return 1 + else + echo "Error: $error_msg" + return 1 + fi + fi +} + +# Configures the necessary IAM roles for a member in the data project. +step_configure_iam() { + local project_id="$1" + local member="$2" + if [[ -z "$project_id" || -z "$member" ]]; then echo "Usage: step_configure_iam "; exit 1; fi + + echo "Configuring IAM roles for $member on $project_id..." + local failed=0 + _grant_role "$project_id" "$member" "roles/analyticshub.subscriber" || failed=1 + _grant_role "$project_id" "$member" "roles/bigquery.user" || failed=1 + _grant_role "$project_id" "$member" "roles/pubsub.editor" || failed=1 + + if [[ $failed -eq 1 ]]; then + echo "IAM configuration partially or fully failed. See errors above." + return 1 + fi + echo "IAM configuration complete." +} + +# Helper to grant roles specifically to a user account. +grant_roles_to_user() { + step_configure_iam "$1" "user:$2" +} + +# Helper to grant roles specifically to a Google Group. +grant_roles_to_group() { + step_configure_iam "$1" "group:$2" +} + +# Helper to grant roles specifically to a Service Account. +grant_roles_to_service_account() { + step_configure_iam "$1" "serviceAccount:$2" +} + +# Runs the complete setup pipeline sequentially. +run_all() { + local project_cloud_id="${1:-}" + local member="${2:-}" # Optional: user:email@example.com + + if [[ -z "$project_cloud_id" ]]; then + usage + exit 1 + fi + + echo "Configuring Data Project (Cloud): $project_cloud_id" + step_enable_apis "$project_cloud_id" + step_verify_apis "$project_cloud_id" + + if [[ -n "$member" ]]; then + step_configure_iam "$project_cloud_id" "$member" + else + echo "----------------------------------------------------------------" + echo "INFO: No member provided. To grant roles, run:" + echo " $0 step_configure_iam $project_cloud_id user:YOUR_EMAIL" + echo "----------------------------------------------------------------" + fi + + echo "Data Project Configuration Helper Complete." +} + +# Internal function to handle subscription logic to an Analytics Hub data exchange. +_subscribe_to_exchange() { + local exchange_path="$1" + local project_cloud_id="$2" + local dest_dataset="$3" + local dest_location="$4" + + echo "Target Exchange: $exchange_path" + + if ! command -v analyticshub_v1_projects_locations_dataExchanges_listings_list &> /dev/null; then + echo "Error: Analytics Hub client not found. Ensure you are running from the 'dist' folder." + return 1 + fi + + echo "Fetching listings for exchange..." + local listings_json + listings_json=$(analyticshub_v1_projects_locations_dataExchanges_listings_list "$exchange_path" "" "" "$project_cloud_id") + + local listing_name + listing_name=$(echo "$listings_json" | jq -r '.listings[0].name // empty') + + if [[ -z "$listing_name" ]]; then + echo "Error: No listings found in this exchange." + return 1 + fi + + echo "Found Listing: $listing_name" + + # Prepare Subscription Request + local dest_ref + dest_ref=$(create_destination_dataset_reference "$project_cloud_id" "$dest_dataset") + local dest_ds + dest_ds=$(create_destination_dataset "$dest_ref" "$dest_location") + local sub_req + sub_req=$(create_subscribe_listing_request "$dest_ds") + + echo "Subscribing to listing..." + analyticshub_v1_projects_locations_dataExchanges_listings_subscribe "$listing_name" "$sub_req" "$project_cloud_id" +} + +# Subscribes to a data exchange using a URL from the Google Cloud Console. +subscribe_from_console_url() { + local url="$1" + local project_cloud_id="$2" + local dest_dataset="${3:-rmi_data_dataset}" + local dest_location="${4:-us}" + + # Extract components from Console URL + if [[ "$url" =~ projects/([^/]+)/locations/([^/]+)/dataExchanges/([^/]+) ]]; then + local src_project="${BASH_REMATCH[1]}" + local src_location="${BASH_REMATCH[2]}" + local src_exchange="${BASH_REMATCH[3]}" + local exchange_path="projects/$src_project/locations/$src_location/dataExchanges/$src_exchange" + + _subscribe_to_exchange "$exchange_path" "$project_cloud_id" "$dest_dataset" "$dest_location" + else + echo "Error: Could not parse Analytics Hub URL." + echo "Expected format: https://console.cloud.google.com/bigquery/analytics-hub/exchanges/projects/.../locations/.../dataExchanges/..." + fi +} + +# Subscribes to a data exchange using explicit project, location, and exchange IDs. +subscribe_from_params() { + local src_project="$1" + local src_location="$2" + local src_exchange="$3" + local project_cloud_id="$4" + local dest_dataset="${5:-rmi_data_dataset}" + local dest_location="${6:-us}" + + local exchange_path="projects/$src_project/locations/$src_location/dataExchanges/$src_exchange" + _subscribe_to_exchange "$exchange_path" "$project_cloud_id" "$dest_dataset" "$dest_location" +} + +# Creates a Pub/Sub subscription to the RMI real-time topic in JSON format. +create_rmi_json_subscription() { + local project_rmi_id="$1" + local project_cloud_id="$2" + local sub_id="${3:-rmi-real-time-json-sub}" + + # Get RMI project number + local project_number + project_number=$(gcloud projects describe "$project_rmi_id" --format="value(projectNumber)") + + local topic="projects/maps-platform-roads-management/topics/rmi-roadsinformation-$project_number-json" + + echo "Creating JSON subscription '$sub_id' in $project_cloud_id for topic $topic..." + gcloud pubsub subscriptions create "$sub_id" --topic="$topic" --project="$project_cloud_id" +} + +# Creates a Pub/Sub subscription to the RMI real-time topic in Binary (Proto) format. +create_rmi_binary_subscription() { + local project_rmi_id="$1" + local project_cloud_id="$2" + local sub_id="${3:-rmi-real-time-sub}" + + # Get RMI project number + local project_number + project_number=$(gcloud projects describe "$project_rmi_id" --format="value(projectNumber)") + + local topic="projects/maps-platform-roads-management/topics/rmi-roadsinformation-$project_number" + + echo "Creating Binary subscription '$sub_id' in $project_cloud_id for topic $topic..." + gcloud pubsub subscriptions create "$sub_id" --topic="$topic" --project="$project_cloud_id" +} + +# Command dispatcher +if [[ $# -eq 0 ]]; then + usage + exit 0 +fi + +case "$1" in + all) + shift + run_all "$@" + ;; + step_*|subscribe_*|create_*|grant_*) + cmd="$1" + shift + "$cmd" "$@" + ;; + *) + usage + exit 1 + ;; +esac diff --git a/roads_management_insights/rmi_cli_tools/scenarios/project_optional_vpcsc.sh b/roads_management_insights/rmi_cli_tools/scenarios/project_optional_vpcsc.sh new file mode 100755 index 0000000..b02373b --- /dev/null +++ b/roads_management_insights/rmi_cli_tools/scenarios/project_optional_vpcsc.sh @@ -0,0 +1,146 @@ +#!/bin/bash +set -euo pipefail +# +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +############################################################################### +# Project Optional VPC-SC Setup +# +# This script provides guidance and YAML generation for VPC Service Controls (VPC-SC) +# configuration required to consume RMI data. +# References: https://developers.google.com/maps/documentation/roads-management-insights/vpc-sc +############################################################################### + +# Displays usage information and available commands for the VPC-SC setup script. +usage() { + echo "Usage: $0 [args]" + echo "" + echo "Commands:" + echo " all Generate all VPC-SC rule snippets" + echo " step_show_restricted_services Show required restricted services" + echo " step_generate_ingress Generate Ingress Rule YAML" + echo " step_generate_egress Generate Egress Rule YAML" + echo "" + echo "MEMBER_LIST: Comma-separated identities (e.g. user:email@addr,serviceAccount:sa@proj.iam.gserviceaccount.com)" +} + +# Displays the list of services that must be restricted within the VPC-SC perimeter. +step_show_restricted_services() { + echo "Ensure the following services are added to the 'Restricted Services' list in your perimeter:" + echo " - analyticshub.googleapis.com" + echo " - bigquery.googleapis.com" + echo " - pubsub.googleapis.com" +} + +# Generates the YAML snippet for a VPC-SC Ingress Rule. +step_generate_ingress() { + local project_cloud_id="$1" + local members_csv="$2" + if [[ -z "$project_cloud_id" || -z "$members_csv" ]]; then echo "Usage: step_generate_ingress "; exit 1; fi + + # Get project number + echo "Fetching project number for $project_cloud_id..." >&2 + local project_number + project_number=$(gcloud projects describe "$project_cloud_id" --format="value(projectNumber)") + + echo "--- Ingress Rule YAML ---" + echo "- ingressFrom:" + echo " identities:" + IFS=',' read -ra ADDR <<< "$members_csv" + for member in "${ADDR[@]}"; do + echo " - $member" + done + echo " sources:" + echo " - accessLevel: '*'" + echo " ingressTo:" + echo " operations:" + echo " - serviceName: analyticshub.googleapis.com" + echo " methodSelectors:" + echo " - method: '*'" + echo " - serviceName: bigquery.googleapis.com" + echo " methodSelectors:" + echo " - method: '*'" + echo " - serviceName: pubsub.googleapis.com" + echo " methodSelectors:" + echo " - method: '*'" + echo " resources:" + echo " - projects/$project_number" +} + +# Generates the YAML snippet for a VPC-SC Egress Rule. +step_generate_egress() { + local members_csv="$1" + if [[ -z "$members_csv" ]]; then echo "Usage: step_generate_egress "; exit 1; fi + + echo "--- Egress Rule YAML ---" + echo "- egressTo:" + echo " operations:" + echo " - serviceName: analyticshub.googleapis.com" + echo " methodSelectors:" + echo " - method: '*'" + echo " - serviceName: bigquery.googleapis.com" + echo " methodSelectors:" + echo " - method: '*'" + echo " - serviceName: pubsub.googleapis.com" + echo " methodSelectors:" + echo " - method: '*'" + echo " resources:" + echo " - projects/maps-platform-roads-management" + echo " egressFrom:" + echo " identities:" + IFS=',' read -ra ADDR <<< "$members_csv" + for member in "${ADDR[@]}"; do + echo " - $member" + done +} + +# Runs the complete VPC-SC configuration guidance pipeline. +run_all() { + local project_cloud_id="$1" + local members_csv="$2" + + echo "=== RMI VPC-SC Configuration Guide ===" + echo "" + step_show_restricted_services + echo "" + step_generate_ingress "$project_cloud_id" "$members_csv" + echo "" + step_generate_egress "$members_csv" + echo "" + echo "VPC-SC configuration rules generated. Please provide these to your Security Administrator." +} + +# Command dispatcher +if [[ $# -eq 0 ]]; then + usage + exit 0 +fi + +case "$1" in + all) + shift + run_all "$@" + ;; + step_*) + cmd="$1" + shift + "$cmd" "$@" + ;; + *) + usage + exit 1 + ;; +esac diff --git a/roads_management_insights/rmi_cli_tools/scenarios/project_rmi_setup.sh b/roads_management_insights/rmi_cli_tools/scenarios/project_rmi_setup.sh new file mode 100755 index 0000000..7c9f586 --- /dev/null +++ b/roads_management_insights/rmi_cli_tools/scenarios/project_rmi_setup.sh @@ -0,0 +1,172 @@ +#!/bin/bash +set -euo pipefail +# +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +############################################################################### +# Project RMI Setup +# +# This script helps configure a Google Cloud project for RMI. +# References: https://developers.google.com/maps/documentation/roads-management-insights/cloud-setup +############################################################################### + +# Displays usage information and available commands for the script. +usage() { + echo "Usage: $0 [args]" + echo "" + echo "Commands:" + echo " all [MEMBER] Run all steps sequentially (MEMBER: user:EMAIL, etc.)" + echo " step_check_billing Check if billing is enabled" + echo " step_enable_apis Enable required GCP APIs" + echo " step_verify_apis Verify APIs are enabled" + echo " step_configure_iam Configure IAM roles for a member" + echo "" + echo "IAM Helpers (MEMBER format: user:EMAIL, serviceAccount:EMAIL, group:EMAIL):" + echo " grant_roles_to_user " + echo " grant_roles_to_group " + echo " grant_roles_to_service_account " +} + +# Checks if billing is enabled for the specified Google Cloud project. +step_check_billing() { + local project_id="${1:-${PROJECT_RMI_ID:-}}" + if [[ -z "$project_id" ]]; then echo "Error: PROJECT_RMI_ID required"; exit 1; fi + echo "Checking billing status for $project_id..." + local billing_enabled + billing_enabled=$(gcloud beta billing projects describe "$project_id" --format="value(billingEnabled)" 2>/dev/null || echo "false") + if [[ "$billing_enabled" != "true" ]]; then + echo "Error: Billing is not enabled for project $project_id." + echo "Please enable billing in the Google Cloud Console." + exit 1 + fi + echo "Billing is enabled." +} + +# Enables the required Google Cloud APIs for the RMI project. +step_enable_apis() { + local project_id="${1:-${PROJECT_RMI_ID:-}}" + if [[ -z "$project_id" ]]; then echo "Error: PROJECT_RMI_ID required"; exit 1; fi + echo "Enabling APIs for $project_id..." + gcloud services enable roadsselection.googleapis.com --project "$project_id" + gcloud services enable analyticshub.googleapis.com --project "$project_id" + echo "APIs enabled." +} + +# Verifies that the required APIs are successfully enabled in the project. +step_verify_apis() { + local project_id="${1:-${PROJECT_RMI_ID:-}}" + if [[ -z "$project_id" ]]; then echo "Error: PROJECT_RMI_ID required"; exit 1; fi + echo "Verifying enabled APIs for $project_id..." + gcloud services list --project "$project_id" --filter="config.name:roadsselection.googleapis.com" + gcloud services list --project "$project_id" --filter="config.name:analyticshub.googleapis.com" +} + +# Internal helper to grant a specific IAM role to a member. +_grant_role() { + local project_id="$1" + local member="$2" + local role="$3" + echo "Granting $role to $member on project $project_id..." + + if ! gcloud projects add-iam-policy-binding "$project_id" --member="$member" --role="$role" --quiet > /dev/null 2> /tmp/gcloud_error.log; then + local error_msg + error_msg=$(cat /tmp/gcloud_error.log) + if [[ "$error_msg" == *"PERMISSION_DENIED"* ]]; then + echo "Error: Permission Denied while granting $role." + echo "Your account may lack 'resourcemanager.projects.setIamPolicy' on project $project_id." + echo "Please ask a Project Owner or IAM Admin to run this command for you:" + echo " gcloud projects add-iam-policy-binding $project_id --member='$member' --role='$role'" + return 1 + else + echo "Error: $error_msg" + return 1 + fi + fi +} + +# Configures the necessary IAM roles for a member in the RMI project. +step_configure_iam() { + local project_id="$1" + local member="$2" + if [[ -z "$project_id" || -z "$member" ]]; then echo "Usage: step_configure_iam "; exit 1; fi + + echo "Configuring IAM roles for $member on $project_id..." + _grant_role "$project_id" "$member" "roles/roads.roadsSelectionAdmin" + echo "IAM configuration complete." +} + +# Helper to grant roles specifically to a user account. +grant_roles_to_user() { + step_configure_iam "$1" "user:$2" +} + +# Helper to grant roles specifically to a Google Group. +grant_roles_to_group() { + step_configure_iam "$1" "group:$2" +} + +# Helper to grant roles specifically to a Service Account. +grant_roles_to_service_account() { + step_configure_iam "$1" "serviceAccount:$2" +} + +# Runs the complete setup pipeline sequentially. +run_all() { + local project_rmi_id="${1:-}" + local member="${2:-}" + + if [[ -z "$project_rmi_id" ]]; then + usage + exit 1 + fi + + echo "Configuring RMI project: $project_rmi_id" + + step_check_billing "$project_rmi_id" + step_enable_apis "$project_rmi_id" + step_verify_apis "$project_rmi_id" + + if [[ -n "$member" ]]; then + step_configure_iam "$project_rmi_id" "$member" + else + echo "No member provided. To grant admin access to a member, run:" + echo " $0 step_configure_iam $project_rmi_id user:USER_EMAIL" + fi + + echo "RMI Project Configuration Complete." +} + +# Command dispatcher +if [[ $# -eq 0 ]]; then + usage + exit 0 +fi + +case "$1" in + all) + shift + run_all "$@" + ;; + step_*|grant_*) + cmd="$1" + shift + "$cmd" "$@" + ;; + *) + usage + exit 1 + ;; +esac diff --git a/roads_management_insights/rmi_cli_tools/scenarios/rmi_data_backup.sh b/roads_management_insights/rmi_cli_tools/scenarios/rmi_data_backup.sh new file mode 100755 index 0000000..065fbe9 --- /dev/null +++ b/roads_management_insights/rmi_cli_tools/scenarios/rmi_data_backup.sh @@ -0,0 +1,171 @@ +#!/bin/bash +set -euo pipefail +# +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +############################################################################### +# RMI Data Backup +# +# This script helps back up RMI-related data. +############################################################################### + +# Locate and source the bundled utility script +_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROADS_SELECTION_UTIL="${_SCRIPT_DIR}/../clients/roadsselection_v1_util.sh" + +if [[ -f "$ROADS_SELECTION_UTIL" ]]; then + source "$ROADS_SELECTION_UTIL" +fi + +# Displays usage information and available commands for the backup script. +usage() { + echo "Usage: $0 [args]" + echo "" + echo "Commands:" + echo " all [OUT_FILE]" + echo " Run full backup (Routes + BigQuery Copy)" + echo " step_initialize_backup " + echo " Show backup configuration" + echo " step_backup_routes [OUT_FILE]" + echo " Backup SelectedRoutes to a JSON file" + echo " step_backup_bq [DEST_DATASET_ID]" + echo " Copy travel time data to a backup dataset" + echo " step_export_to_gcs " + echo " Export data to Cloud Storage as JSONL" +} + +# Initializes and displays the backup configuration. +step_initialize_backup() { + local project_rmi_id="${1:-${PROJECT_RMI_ID:-}}" + local project_cloud_id="${2:-${PROJECT_CLOUD_ID:-}}" + echo "RMI Data Backup Script" + echo "Source (RMI Project): $project_rmi_id" + echo "Destination (Cloud Project): $project_cloud_id" +} + +# Backs up Routes of Interest (SelectedRoutes) to a local JSON file. +step_backup_routes() { + local project_id="${1:-${PROJECT_RMI_ID:-}}" + local output_file="${2:-selected_routes_backup.json}" + if [[ -z "$project_id" ]]; then echo "Error: PROJECT_RMI_ID required"; exit 1; fi + + echo "Backing up SelectedRoutes for project $project_id to $output_file..." + + if ! command -v roadsselection_v1_projects_selectedRoutes_list_all &> /dev/null; then + echo "Error: Roads Selection utility not found. Run bundle.sh first." + return 1 + fi + + local parent="projects/$project_id" + roadsselection_v1_projects_selectedRoutes_list_all "$parent" "1000" "$project_id" > "$output_file" + + local count + count=$(wc -l < "$output_file") + echo "Backup complete. $count routes saved to $output_file" +} + +# Copies travel time data from a subscribed dataset to a new backup dataset in the cloud project. +step_backup_bq() { + local project_cloud_id="${1:-${PROJECT_CLOUD_ID:-}}" + local src_project_id="$2" + local src_dataset_id="$3" + local dest_dataset_id="${4:-rmi_backup}" + + if [[ -z "$project_cloud_id" || -z "$src_project_id" || -z "$src_dataset_id" ]]; then + echo "Usage: step_backup_bq [DEST_DATASET_ID]" + exit 1 + fi + + echo "Backing up BigQuery data from $src_project_id.$src_dataset_id to $project_cloud_id.$dest_dataset_id..." + + # 1. Create the backup dataset if it doesn't exist + echo "Ensuring dataset $dest_dataset_id exists in $project_cloud_id..." + bq --project_id="$project_cloud_id" mk --dataset --if_exists=true "$dest_dataset_id" + + # 2. Copy tables + local tables=("routes_status" "historical_travel_time" "recent_roads_data") + for table in "${tables[@]}"; do + echo "Copying $table..." + bq --project_id="$project_cloud_id" query --use_legacy_sql=false \ + "CREATE OR REPLACE TABLE \`$project_cloud_id.$dest_dataset_id.$table\` AS SELECT * FROM \`$src_project_id.$src_dataset_id.$table\`" + done + + echo "BigQuery backup complete." +} + +# Exports travel time data from the subscribed dataset to Google Cloud Storage. +step_export_to_gcs() { + local project_cloud_id="$1" + local src_project_id="$2" + local src_dataset_id="$3" + local gcs_path="$4" + + if [[ -z "$project_cloud_id" || -z "$src_project_id" || -z "$src_dataset_id" || -z "$gcs_path" ]]; then + echo "Usage: step_export_to_gcs " + exit 1 + fi + + echo "Exporting BigQuery data from $src_project_id.$src_dataset_id to $gcs_path..." + + local tables=("routes_status" "historical_travel_time" "recent_roads_data") + for table in "${tables[@]}"; do + echo "Exporting $table..." + bq --project_id="$project_cloud_id" extract --destination_format=NEWLINE_DELIMITED_JSON \ + "$src_project_id:$src_dataset_id.$table" "$gcs_path/${table}-*.json" + done + + echo "GCS export complete." +} + +# Executes the complete backup pipeline (Routes + BigQuery Copy). +run_all() { + local project_rmi_id="${1:-}" + local project_cloud_id="${2:-}" + local src_dataset_proj="${3:-}" + local src_dataset_name="${4:-}" + local output_file="${5:-selected_routes_backup.json}" + + if [[ -z "$project_rmi_id" || -z "$project_cloud_id" || -z "$src_dataset_proj" || -z "$src_dataset_name" ]]; then + usage + exit 1 + fi + + step_initialize_backup "$project_rmi_id" "$project_cloud_id" + step_backup_routes "$project_rmi_id" "$output_file" + step_backup_bq "$project_cloud_id" "$src_dataset_proj" "$src_dataset_name" +} + +# Command dispatcher +if [[ $# -eq 0 ]]; then + usage + exit 0 +fi + +case "$1" in + all) + shift + run_all "$@" + ;; + step_*) + cmd="$1" + shift + "$cmd" "$@" + ;; + *) + usage + exit 1 + ;; +esac diff --git a/roads_management_insights/rmi_cli_tools/scenarios/rmi_verify_api_access.sh b/roads_management_insights/rmi_cli_tools/scenarios/rmi_verify_api_access.sh new file mode 100755 index 0000000..37a505b --- /dev/null +++ b/roads_management_insights/rmi_cli_tools/scenarios/rmi_verify_api_access.sh @@ -0,0 +1,233 @@ +#!/bin/bash +set -euo pipefail +# +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + + +############################################################################### +# RMI API Access Verification +# +# This script demonstrates and tests the Create, Read, List, and Delete +# operations for the Roads Selection API. +# +# Usage: ./rmi_verify_api_access.sh +############################################################################### + +PROJECT_RMI_ID="${1:-}" + +if [[ -z "$PROJECT_RMI_ID" ]]; then + echo "Usage: $0 " + exit 1 +fi + +# Locate and source the bundled utility script +_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +DIST_SCRIPT="${_SCRIPT_DIR}/../clients/roadsselection_v1_util.sh" + +if [[ ! -f "$DIST_SCRIPT" ]]; then + echo "Error: Bundled script not found at $DIST_SCRIPT" + echo "Please run 'scripts/bash/bundle.sh' first." + exit 1 +fi + +source "$DIST_SCRIPT" + +# Displays usage information and available commands for the API access test script. +usage() { + echo "Usage: $0 [args]" + echo "" + echo "Commands:" + echo " all Run all CRUD tests sequentially" + echo " step_prepare_data Prepare sample route data" + echo " step_create_route Create a selected route" + echo " step_get_route Retrieve the created route" + echo " step_list_routes List routes and verify presence" + echo " step_delete_route Delete the created route" + echo " step_verify_deletion Verify the route is gone (404)" +} + +# Global variables to store test state (for sequential 'all' run) +ORIGIN="" +DESTINATION="" +DYNAMIC_ROUTE="" +DISPLAY_NAME="" +SELECTED_ROUTE="" +ROUTE_NAME="" + +# Prepares the necessary data structures (LatLng, Route) for the test. +step_prepare_data() { + echo "--- Step 1: Preparing Data ---" + # Tokyo Tower + ORIGIN=$(create_lat_lng 35.658581 139.745433) + # Roppongi Hills + DESTINATION=$(create_lat_lng 35.660456 139.729067) + # Simple Route + DYNAMIC_ROUTE=$(create_dynamic_route "$ORIGIN" "$DESTINATION") + DISPLAY_NAME="CRUD Test Route $(date +%s)" + SELECTED_ROUTE=$(create_selected_route "$DYNAMIC_ROUTE" "$DISPLAY_NAME") + + echo "Origin: $ORIGIN" + echo "Destination: $DESTINATION" + echo "Display Name: $DISPLAY_NAME" +} + +# Creates a new selected route in the specified project. +step_create_route() { + local project_id="${1:-$PROJECT_RMI_ID}" + if [[ -z "$project_id" ]]; then echo "Error: PROJECT_RMI_ID required"; exit 1; fi + if [[ -z "$SELECTED_ROUTE" ]]; then step_prepare_data; fi + + echo "--- Step 2: Create Selected Route in $project_id ---" + local parent="projects/$project_id" + local response_create + response_create=$(roadsselection_v1_projects_selectedRoutes_create "$parent" "$SELECTED_ROUTE" "" "$project_id") + + # Check for errors + if echo "$response_create" | jq -e '.error' >/dev/null; then + echo "Error Creating Route:" + echo "$response_create" | jq . + exit 1 + fi + + ROUTE_NAME=$(echo "$response_create" | jq -r '.name') + echo "Created Route Name: $ROUTE_NAME" +} + +# Retrieves the details of a specific selected route. +step_get_route() { + local project_id="${1:-$PROJECT_RMI_ID}" + local route_name="${2:-$ROUTE_NAME}" + if [[ -z "$project_id" || -z "$route_name" ]]; then echo "Usage: step_get_route "; exit 1; fi + + echo "--- Step 3: Get Selected Route $route_name ---" + local response_get + response_get=$(roadsselection_v1_projects_selectedRoutes_get "$route_name" "$project_id") + + if echo "$response_get" | jq -e '.error' >/dev/null; then + echo "Error Getting Route:" + echo "$response_get" | jq . + exit 1 + fi + + local get_display_name + get_display_name=$(echo "$response_get" | jq -r '.displayName') + echo "Retrieved Route Display Name: $get_display_name" +} + +# Lists all selected routes in the project and checks if a specific route exists. +step_list_routes() { + local project_id="${1:-$PROJECT_RMI_ID}" + local route_name="${2:-$ROUTE_NAME}" + if [[ -z "$project_id" ]]; then echo "Error: PROJECT_RMI_ID required"; exit 1; fi + + echo "--- Step 4: List Selected Routes in $project_id ---" + local parent="projects/$project_id" + local response_list + response_list=$(roadsselection_v1_projects_selectedRoutes_list "$parent" "100" "" "$project_id") + + if echo "$response_list" | jq -e '.error' >/dev/null; then + echo "Error Listing Routes:" + echo "$response_list" | jq . + exit 1 + fi + + if [[ -n "$route_name" ]]; then + local found_in_list + found_in_list=$(echo "$response_list" | jq --arg name "$route_name" '[.selectedRoutes[]? | select(.name == $name)] | length') + if [[ "$found_in_list" -gt 0 ]]; then + echo "PASS: Route $route_name found in list." + else + echo "FAIL: Route $route_name not found in list." + fi + fi +} + +# Deletes a specific selected route from the project. +step_delete_route() { + local project_id="${1:-$PROJECT_RMI_ID}" + local route_name="${2:-$ROUTE_NAME}" + if [[ -z "$project_id" || -z "$route_name" ]]; then echo "Usage: step_delete_route "; exit 1; fi + + echo "--- Step 5: Delete Selected Route $route_name ---" + local response_delete + response_delete=$(roadsselection_v1_projects_selectedRoutes_delete "$route_name" "$project_id") + + if echo "$response_delete" | jq -e '.error' >/dev/null; then + echo "Error Deleting Route:" + echo "$response_delete" | jq . + exit 1 + fi + + echo "Delete command executed successfully." +} + +# Verifies that a route has been successfully deleted by attempting to retrieve it. +step_verify_deletion() { + local project_id="${1:-$PROJECT_RMI_ID}" + local route_name="${2:-$ROUTE_NAME}" + if [[ -z "$project_id" || -z "$route_name" ]]; then echo "Usage: step_verify_deletion "; exit 1; fi + + echo "--- Step 6: Verify Deletion of $route_name ---" + local response_get_again + response_get_again=$(roadsselection_v1_projects_selectedRoutes_get "$route_name" "$project_id") + + if echo "$response_get_again" | jq -e '.error.code == 404' >/dev/null; then + echo "PASS: Route successfully deleted (Received 404 as expected)." + else + echo "FAIL: Route might still exist or unexpected error." + echo "Response: $response_get_again" + exit 1 + fi +} + +# Executes the complete CRUD test scenario sequentially. +run_all() { + PROJECT_RMI_ID="${1:-}" + if [[ -z "$PROJECT_RMI_ID" ]]; then usage; exit 1; fi + + echo "=== Starting CRUD Test for RMI Project: $PROJECT_RMI_ID ===" + + step_prepare_data + step_create_route "$PROJECT_RMI_ID" + step_get_route "$PROJECT_RMI_ID" "$ROUTE_NAME" + step_list_routes "$PROJECT_RMI_ID" "$ROUTE_NAME" + step_delete_route "$PROJECT_RMI_ID" "$ROUTE_NAME" + step_verify_deletion "$PROJECT_RMI_ID" "$ROUTE_NAME" + + echo "=== CRUD Test Scenario Complete ===" +} + +# Command dispatcher +if [[ $# -eq 0 ]]; then + usage + exit 0 +fi + +case "$1" in + all) + shift + run_all "$@" + ;; + step_*) + cmd="$1" + shift + "$cmd" "$@" + ;; + *) + usage + exit 1 + ;; +esac