feat(gpu): Add robust proxy support for driver installation (#1361)

This PR introduces comprehensive HTTP/S proxy support for the GPU
driver installation script, enabling its use in environments with
restricted internet egress, such as those using Secure Web Proxy.

The `set_proxy` function, controlled by the `http-proxy` and new
`http-proxy-pem-uri` metadata attributes, now configures APT, GPG,
Java, pip, and Conda to route traffic through the specified proxy. If a
PEM certificate URI is provided, the certificate is installed into the
OS, Conda, and Java trust stores. The script now correctly handles
the proxy scheme (HTTP vs HTTPS) based on the presence of the
`http-proxy-pem-uri` metadata.

This change was validated in a development environment where all
internet access was routed through an explicit proxy.

Additional changes:

- `README.md` updated to document the new `http-proxy-pem-uri`
  metadata option and clarify `http-proxy` usage.
- GCS caching for the NVIDIA driver is checked earlier to avoid
  unnecessary HEAD requests to the NVIDIA CDN.
- `configure_dkms_certs` is now more idempotent.
- Spark RAPIDS versions and repository URL aligned with
  `spark-rapids/spark-rapids.sh` as part of a move towards a unified
  GPU/RAPIDS installation script.
- Switched to using `/sys/bus/pci/devices/*/uevent` for GPU detection
  to remove dependency on pciutils
- Moved `set_proxy` call earlier in `prepare_to_install`.
- Refactored `no_proxy` and `nvcc_gencode` list generation.

fix(ci): Add retry logic to kubectl logs in presubmit

- Wrapped `kubectl logs` command in `run-presubmit-on-k8s.sh` with a
  retry loop to handle transient "No agent available" errors from GKE.

Author: cjac

cloudbuild/run-presubmit-on-k8s.sh

@@ -42,19 +42,46 @@ EOF
 kubectl apply -f $POD_CONFIG
 
 # Delete POD on exit and describe it before deletion if exit was unsuccessful
-trap '[[ $? != 0 ]] && kubectl describe "pod/${POD_NAME}"; kubectl delete pods "${POD_NAME}"' EXIT
+trap 'exit_code=$?
+if [[ ${exit_code} != 0 ]]; then
+  echo "Presubmit failed for ${POD_NAME}. Describing pod..."
+  kubectl describe "pod/${POD_NAME}" || echo "Failed to describe pod."
+
+  PROJECT_ID=$(gcloud config get-value project 2>/dev/null || echo "unknown-project")
+  BUCKET="dataproc-init-actions-test-${PROJECT_ID}"
+  LOG_GCS_PATH="gs://${BUCKET}/${BUILD_ID}/logs/${POD_NAME}.log"
+
+  echo "Attempting to upload logs to ${LOG_GCS_PATH}"
+  if kubectl logs "${POD_NAME}" | gsutil cp - "${LOG_GCS_PATH}"; then
+    echo "Logs for failed pod ${POD_NAME} uploaded to: ${LOG_GCS_PATH}"
+  else
+    echo "Log upload to ${LOG_GCS_PATH} failed."
+  fi
+fi
+echo "Deleting pod ${POD_NAME}..."
+kubectl delete pods "${POD_NAME}" --ignore-not-found=true
+exit ${exit_code}' EXIT
 
 kubectl wait --for=condition=Ready "pod/${POD_NAME}" --timeout=15m
 
+# To mitigate problems with early test failure, retry kubectl logs
+sleep 10s
 while ! kubectl describe "pod/${POD_NAME}" | grep -q Terminated; do
-  kubectl logs -f "${POD_NAME}" --since-time="${LOGS_SINCE_TIME}" --timestamps=true
+  # Try to stream logs, but primary log capture is now in the trap
+  kubectl logs -f "${POD_NAME}" --since-time="${LOGS_SINCE_TIME}" --timestamps=true || true
   LOGS_SINCE_TIME=$(date --iso-8601=seconds)
+  sleep 2 # Short sleep to avoid busy waiting if logs -f exits
 done
 
-EXIT_CODE=$(kubectl get pod "${POD_NAME}" \
-  -o go-template="{{range .status.containerStatuses}}{{.state.terminated.exitCode}}{{end}}")
+# Final check on the pod exit code
+EXIT_CODE=$(kubectl get pod "${POD_NAME}" -o go-template="{{range .status.containerStatuses}}{{.state.terminated.exitCode}}{{end}}" || echo "1")
 
 if [[ ${EXIT_CODE} != 0 ]]; then
-  echo "Presubmit failed!"
+  echo "Presubmit final state for ${POD_NAME} indicates failure (Exit Code: ${EXIT_CODE})."
+  # The trap will handle the log upload and cleanup
   exit 1
 fi
+
+echo "Presubmit for ${POD_NAME} successful."
+# Explicitly exit 0 to clear the trap's exit code
+exit 0

gpu/README.md

@@ -225,6 +225,18 @@ sometimes found in the "building from source" sections.
     modulus md5sum of the files referenced by both the private and
     public secret names.
 
+-   `http-proxy: <HOST>:<PORT>` - Optional. The address of an HTTP
+    proxy to use for internet egress. The script will configure `apt`,
+    `curl`, `gsutil`, `pip`, `java`, and `gpg` to use this proxy.
+
+-   `http-proxy-pem-uri: <GS_PATH>` - Optional. A `gs://` path to the
+    PEM-encoded certificate file used by the proxy specified in
+    `http-proxy`. This is needed if the proxy uses TLS and its
+    certificate is not already trusted by the cluster's default trust
+    store (e.g., if it's a self-signed certificate or signed by an
+    internal CA). The script will install this certificate into the
+    system and Java trust stores.
+
 #### Loading built kernel module
 
 For platforms which do not have pre-built binary kernel drivers, the