From 00532452fd8861da4c79ba7c980e3a25c309b388 Mon Sep 17 00:00:00 2001 From: Brent Shaffer Date: Fri, 22 Sep 2023 14:31:02 -0700 Subject: [PATCH 1/4] feat: add AIP 4120 --- aip/auth/4120.md | 96 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 aip/auth/4120.md diff --git a/aip/auth/4120.md b/aip/auth/4120.md new file mode 100644 index 0000000000..a123d53d70 --- /dev/null +++ b/aip/auth/4120.md @@ -0,0 +1,96 @@ +--- +id: 4120 +state: draft +created: 2023-12-07 +--- + +# Universe Domain requirements in Authentication Libraries + +This AIP outlines the requirements for Cloud SDK authentication libraries to support Universe Domains, in order to authenticate with APIs outside the GDU (Google Default Universe). + +## Guidance + +Universe Domain support in the auth library has the following requirements: + + - Credentials implement a `universe_domain` getter + - Credentials accept a user-supplied `universe_domain`. + - Service Account and External Account credentials read the universe domain from the credentials JSON file. + - GCE credentials retrieve the universe domain from the metadata server. + +### Naming + +We aim to have a consistent user experience across a broad range of clients. One of the invariants is the `universe_domain` name that should be used to represent the universe domain in configuration, user-readable properties and methods. Languages that use different casing conventions should adapt the name to be idiomatic but keep the two specific words in the same order, so `universe-domain`, `universeDomain` and `SetUniverseDomain` are all acceptable, but e.g. `DomainOfUniverse` or `Endpoint` is not. + +### Credentials implement a `universe_domain` getter (method, property) + +> This requirement applies to **all** flows. + +Every credentials class/object **must** expose the `universe_domain` getter (property, method, etc). The auth library end-users, including the client libraries will use this property. + +The credentials classes for flows that are not supported outside the GDU (e.g. user credentials) **must** still expose the `universe_domain` to the end-user. These credentials **must** return the default value `"googleapis.com"` for the `universe_domain`. + +### Credentials accept a user-supplied `universe_domain` + +> This requirement applies **only** to the flows supported outside the GDU. + +The end-user **must** be able to create universe-specific credentials in code. This can take many forms: constructor arguments, set properties/methods, copy-on-write methods, etc. This requirement is to treat `universe_domain` as a first-class property in regards to modification and user visibility. + +E.g. if the authentication library provides a `with_scope` copy-on-write method it can satisfy this requirement by providing a similar `with_universe_domain` method. +This requirement applies only to the credentials classes that are related to the authentication flows supported outside the GDU. This is in contrast to the requirement to "expose the `universe_domain` to the end-user" which applies to all credentials classes. + +### Service Account credentials read the universe domain from the credentials JSON file + +> This requirement applies **only** to the Service Account flow. + +The JSON credential files will have a new property: `universe_domain`. The authentication libraries **must** read it when parsing JSON credential files and expose it to the libraries' end-users. If the `universe_domain` is not present, the credentials file is assumed to be (a legacy one) from the Google default universe, and the `universe_domain` value **must** default to `"googleapis.com"`. + +This value can also be used to make decisions about whether the library is in the GDU or not e.g. the purpose of throwing specific errors if the end-user is forcing an unsupported authentication flow with non-GDU credentials. + +#### Authentication libraries **must** use the self-signed JWT flow when authenticating outside the GDU using Service Account credentials + +Currently the self-signed JWT (SSJ) sub-flow is the default option for Service Account auth flow, but user parameters provided, such as `scope`, `audience`, or `useJwtWithScopes` can change that. E.g. setting `useJwtWithScopes` to `false` when specifying `scope` would result in auth library fallback to token exchange flow for service accounts. +The token exchange for Service Accounts is not a supported flow outside the GDU, and services must support SSJ with scopes. Therefore authentication libraries **must** use the self-signed JWT flow when authenticating outside the GDU using Service Account credentials. If the `useJwtWithScopes` parameter exists (whatever form it takes in specific languages), it **must** be ignored outside the GDU, and its documentation **must** be updated to reflect that. + +### GCE credentials retrieve the universe domain from the metadata server + +> This requirement applies **only** to the VM metadata flow. + +GCE credentials (also known as VM Metadata credentials) directly retrieves an access token from the Metadata Server (MDS). The access token has no universe information. The authentication libraries **must** modify the VM metadata flow to make a second request to retrieve the universe_domain from the following URI: `http://metadata.google.internal/computeMetadata/v1/universe/universe_domain`. + +Once the `universe_domain` is retrieved, the authentication libraries **must** cache it to avoid making unnecessary MDS requests. + +Failure modes when retrieving the universe_domain from MDS: + +| Error when retrieving the universe_domain from MDS | Action | +| -------------------------------------------------- | ------ | +| 404 | The authentication libraries **must** set the `universe_domain` to the default value `"googleapis.com"`. | +| Timeout | The authentication libraries **must not** fall back to the `"googleapis.com"` default value. A retry mechanism is recommended. The authentication libraries **should** do the same action as when retrieving the access token times out. | +| Any other error | The authentication libraries **must not** fall back to the `"googleapis.com"` default value. The authentication libraries **should** do the same action as when the same error happens when retrieving the access token. | + +Additional recommendations: + + - GCE credentials **should** retrieve the universe domain in a 'lazy' way, when the end-user requests the `universe_domain` via the property/get method. They **should not** do it during the credentials initialization. + +### Other requirements + +#### Authentication libraries **must not** use hardcoded non-constant information in the authentication flows. + +> This requirement applies **only** to the flows which are supported outside the GDU. + +The code for the authentication flows **must not** contain hardcoded endpoints, endpoint-alikes or other authentication information unless this information is constant across all flows for all universes. Any information that is possible to read from the credentials file should be read from the credentials file. For example, `scopes` contain the "googleapis.com" domain but are still valid outside the GDU because their values are consistent across all universes, whereas the STS endpoint "sts.googleapis.com" is not allowed because it is different for every universe. + +#### Authentication libraries **must** add the `universe_domain` property to the token authentication credentials constructor (if supported) and document the default behavior + +> This requirement applies **only** to the access token flow (if supported by the auth library). + +If the authentication library provides a "helper" credentials class that the end-user can initialize with the access token they already have, this class should additionally expose the `universe_domain` property in the constructor. +The access tokens are not guaranteed to be end-user decodable or have universe domain information. Therefore the end-user should be able to conveniently specify which universe's access token they are providing. Preferably it should be in the same fashion that they provide the access token itself, so if e.g. the access token is a constructor parameter, the `universe_domain` should also be a parameter. +This constructor property should have a default value of `"googleapis.com"` for backwards compatibility. This behavior should be documented in a way to draw the developer's attention since we want to prevent them from making a mistake and sending an access token from a different universe to the GDU "by default". + +### Authentication flows supported outside the GDU + +The official authentication flows which are supported outside the GDU are the following: + + - Service Account Credentials (through use of Service Account JWT Access Credentials only) + - Metadata Credentials + - External Account Credentials From e2ea58b1c52b4e301030a2f2513ff3f49f877b7f Mon Sep 17 00:00:00 2001 From: Brent Shaffer Date: Tue, 13 Feb 2024 10:03:17 -0800 Subject: [PATCH 2/4] address review comments --- aip/auth/4120.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/aip/auth/4120.md b/aip/auth/4120.md index a123d53d70..9514dbc08c 100644 --- a/aip/auth/4120.md +++ b/aip/auth/4120.md @@ -25,7 +25,9 @@ We aim to have a consistent user experience across a broad range of clients. One > This requirement applies to **all** flows. -Every credentials class/object **must** expose the `universe_domain` getter (property, method, etc). The auth library end-users, including the client libraries will use this property. +Every credentials class/object **must** expose the `universe_domain` getter (property, method, etc). + +The client libraries and auth library end-users will use this property to construct URIs for the appropriate universe domain. This value can also be used to make decisions about whether the library is in the GDU or not e.g. the purpose of throwing specific errors if the end-user is forcing an unsupported authentication flow with non-GDU credentials. The credentials classes for flows that are not supported outside the GDU (e.g. user credentials) **must** still expose the `universe_domain` to the end-user. These credentials **must** return the default value `"googleapis.com"` for the `universe_domain`. @@ -33,19 +35,17 @@ The credentials classes for flows that are not supported outside the GDU (e.g. u > This requirement applies **only** to the flows supported outside the GDU. -The end-user **must** be able to create universe-specific credentials in code. This can take many forms: constructor arguments, set properties/methods, copy-on-write methods, etc. This requirement is to treat `universe_domain` as a first-class property in regards to modification and user visibility. +The end-user **must** be able to create universe-specific credentials in code. This can take many forms: constructor arguments, set properties/methods, copy-on-write methods, etc. This requirement is to treat `universe_domain` as a first-class property in regards to modification and user visibility. This will allow users to explicitly override the credential universe domain for testing and development. E.g. if the authentication library provides a `with_scope` copy-on-write method it can satisfy this requirement by providing a similar `with_universe_domain` method. This requirement applies only to the credentials classes that are related to the authentication flows supported outside the GDU. This is in contrast to the requirement to "expose the `universe_domain` to the end-user" which applies to all credentials classes. -### Service Account credentials read the universe domain from the credentials JSON file +### Service Account credentials read the universe domain from the Service account key JSON file > This requirement applies **only** to the Service Account flow. The JSON credential files will have a new property: `universe_domain`. The authentication libraries **must** read it when parsing JSON credential files and expose it to the libraries' end-users. If the `universe_domain` is not present, the credentials file is assumed to be (a legacy one) from the Google default universe, and the `universe_domain` value **must** default to `"googleapis.com"`. -This value can also be used to make decisions about whether the library is in the GDU or not e.g. the purpose of throwing specific errors if the end-user is forcing an unsupported authentication flow with non-GDU credentials. - #### Authentication libraries **must** use the self-signed JWT flow when authenticating outside the GDU using Service Account credentials Currently the self-signed JWT (SSJ) sub-flow is the default option for Service Account auth flow, but user parameters provided, such as `scope`, `audience`, or `useJwtWithScopes` can change that. E.g. setting `useJwtWithScopes` to `false` when specifying `scope` would result in auth library fallback to token exchange flow for service accounts. From 8202ff4d3b023f960ade65ebe5a9d9b3de380ab2 Mon Sep 17 00:00:00 2001 From: Brent Shaffer Date: Tue, 13 Feb 2024 11:28:37 -0800 Subject: [PATCH 3/4] more review comments --- aip/auth/4120.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/aip/auth/4120.md b/aip/auth/4120.md index 9514dbc08c..7dfcba5449 100644 --- a/aip/auth/4120.md +++ b/aip/auth/4120.md @@ -40,16 +40,15 @@ The end-user **must** be able to create universe-specific credentials in code. T E.g. if the authentication library provides a `with_scope` copy-on-write method it can satisfy this requirement by providing a similar `with_universe_domain` method. This requirement applies only to the credentials classes that are related to the authentication flows supported outside the GDU. This is in contrast to the requirement to "expose the `universe_domain` to the end-user" which applies to all credentials classes. -### Service Account credentials read the universe domain from the Service account key JSON file +### Service Account Key credentials read the universe domain from the Service account key JSON file > This requirement applies **only** to the Service Account flow. The JSON credential files will have a new property: `universe_domain`. The authentication libraries **must** read it when parsing JSON credential files and expose it to the libraries' end-users. If the `universe_domain` is not present, the credentials file is assumed to be (a legacy one) from the Google default universe, and the `universe_domain` value **must** default to `"googleapis.com"`. -#### Authentication libraries **must** use the self-signed JWT flow when authenticating outside the GDU using Service Account credentials +#### Authentication libraries **must** use the self-signed JWT flow when authenticating outside the GDU using Service Account Key credentials -Currently the self-signed JWT (SSJ) sub-flow is the default option for Service Account auth flow, but user parameters provided, such as `scope`, `audience`, or `useJwtWithScopes` can change that. E.g. setting `useJwtWithScopes` to `false` when specifying `scope` would result in auth library fallback to token exchange flow for service accounts. -The token exchange for Service Accounts is not a supported flow outside the GDU, and services must support SSJ with scopes. Therefore authentication libraries **must** use the self-signed JWT flow when authenticating outside the GDU using Service Account credentials. If the `useJwtWithScopes` parameter exists (whatever form it takes in specific languages), it **must** be ignored outside the GDU, and its documentation **must** be updated to reflect that. +The token exchange for Service Accounts is not a supported flow outside the GDU, and services must support SSJ with scopes. Therefore authentication libraries **must** use the self-signed JWT flow when authenticating outside the GDU using Service Accountj key credentials. If the `UseJWTAccessWithScope` parameter exists (whatever form it takes in specific languages), it **must** be ignored outside the GDU, and its documentation **must** be updated to reflect that. ### GCE credentials retrieve the universe domain from the metadata server @@ -91,6 +90,6 @@ This constructor property should have a default value of `"googleapis.com"` for The official authentication flows which are supported outside the GDU are the following: - - Service Account Credentials (through use of Service Account JWT Access Credentials only) - - Metadata Credentials + - Service Account Key Credentials (through use of Service Account JWT Access Credentials only) + - Credentials provided by metadata service - External Account Credentials From 24c3fd86d5354eb57cd222697bf1e84623500ffc Mon Sep 17 00:00:00 2001 From: Brent Shaffer Date: Tue, 13 Feb 2024 11:38:31 -0800 Subject: [PATCH 4/4] remove saying the client libraries use the value to construct the URI --- aip/auth/4120.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aip/auth/4120.md b/aip/auth/4120.md index 7dfcba5449..04ce86144f 100644 --- a/aip/auth/4120.md +++ b/aip/auth/4120.md @@ -27,7 +27,7 @@ We aim to have a consistent user experience across a broad range of clients. One Every credentials class/object **must** expose the `universe_domain` getter (property, method, etc). -The client libraries and auth library end-users will use this property to construct URIs for the appropriate universe domain. This value can also be used to make decisions about whether the library is in the GDU or not e.g. the purpose of throwing specific errors if the end-user is forcing an unsupported authentication flow with non-GDU credentials. +This property will be used by the client libraries and auth library end-users. This value can be used to make decisions about whether the library is in the GDU or not e.g. the purpose of throwing specific errors if the end-user is forcing an unsupported authentication flow with non-GDU credentials. The credentials classes for flows that are not supported outside the GDU (e.g. user credentials) **must** still expose the `universe_domain` to the end-user. These credentials **must** return the default value `"googleapis.com"` for the `universe_domain`.