Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement propagation of x-goog-spanner-request-id gRPC header on every call #1261

Open
6 tasks
odeke-em opened this issue Dec 9, 2024 · 0 comments · May be fixed by #1264
Open
6 tasks

Implement propagation of x-goog-spanner-request-id gRPC header on every call #1261

odeke-em opened this issue Dec 9, 2024 · 0 comments · May be fixed by #1264
Assignees
@rahul2393
Labels
api: spanner Issues related to the googleapis/python-spanner API.

Comments

@odeke-em
Copy link
Contributor

odeke-em commented Dec 9, 2024

This is a feature request to implement propagation of x-goog-spanner-request-id gRPC header on every call.

Reasoning

x-goog-spanner-request-id allows propagation of an ID that'll be correlated between client and server calls. We can't rely on trace headers because they are sampled, convoluted through a bunch of infrastructure so Google Engineering asked that we craft up a better mechanism. The value will allow tracking the operations of an RPC even through retries

Requirements

Each x-goog-spanner-request-id requires the following fields

  • Version of the specification being implemented: ordinal number: 1 for version 1
  • random Uint64 for the process' lifetime
  • ID of the SpannerClient being used
  • channelID for the gRPC channel being used
  • Nth Request Count: nthReq
  • RPC attempt number for nthReq

Specification

Please reference https://orijtech.notion.site/x-goog-spanner-request-id-always-on-gRPC-header-to-aid-in-quick-debugging-of-errors-14aba6bc91348091a58fca7a505c9827?pvs=4

Reference implementation

I built a Go reference implementation at googleapis/google-cloud-go#11048

/cc @tharoldD
@product-auto-label product-auto-label bot added the api: spanner Issues related to the googleapis/python-spanner API. label Dec 9, 2024
@odeke-em odeke-em linked a pull request Dec 11, 2024 that will close this issue
Open
odeke-em added a commit to odeke-em/python-spanner that referenced this issue Dec 12, 2024
@odeke-em
feat(x-goog-spanner-request-id): implement request_id generation and …
4f1da67 
…propagation

Generates a request_id that is then injected inside metadata
that's sent over to the Cloud Spanner backend.

Fixes googleapis#1261
odeke-em added a commit to odeke-em/python-spanner that referenced this issue Dec 14, 2024
@odeke-em
feat(x-goog-spanner-request-id): implement request_id generation and …
8024fca 
…propagation

Generates a request_id that is then injected inside metadata
that's sent over to the Cloud Spanner backend.

Fixes googleapis#1261
odeke-em added a commit to odeke-em/python-spanner that referenced this issue Dec 16, 2024
@odeke-em
feat(x-goog-spanner-request-id): implement request_id generation and …
e456865 
…propagation

Generates a request_id that is then injected inside metadata
that's sent over to the Cloud Spanner backend.

Fixes googleapis#1261
odeke-em added a commit to odeke-em/python-spanner that referenced this issue Dec 18, 2024
@odeke-em
feat(x-goog-spanner-request-id): introduce AtomicCounter
1f52008 
This change introduces AtomicCounter, a concurrency/thread-safe
counter do deal with the multi-threaded nature of variables.
It permits operations:
* atomic_counter += 1
* value = atomic_counter + 1
* atomic_counter.value

that'll be paramount to bringing in the logic for
x-goog-spanner-request-id in much reduced changelists.

Updates googleapis#1261
Carved out from PR googleapis#1264
@odeke-em odeke-em mentioned this issue Dec 18, 2024
Merged
odeke-em added a commit to odeke-em/python-spanner that referenced this issue Dec 18, 2024
@odeke-em
feat(x-goog-spanner-request-id): implement request_id generation and …
ca31466 
…propagation

Generates a request_id that is then injected inside metadata
that's sent over to the Cloud Spanner backend.

Fixes googleapis#1261
olavloite added a commit that referenced this issue Dec 19, 2024
@odeke-em @olavloite
feat(x-goog-spanner-request-id): introduce AtomicCounter (#1275)
f2483e1 
* feat(x-goog-spanner-request-id): introduce AtomicCounter

This change introduces AtomicCounter, a concurrency/thread-safe
counter do deal with the multi-threaded nature of variables.
It permits operations:
* atomic_counter += 1
* value = atomic_counter + 1
* atomic_counter.value

that'll be paramount to bringing in the logic for
x-goog-spanner-request-id in much reduced changelists.

Updates #1261
Carved out from PR #1264

* Tests for with_request_id

* chore: remove sleep

* chore: remove unused import

---------

Co-authored-by: Knut Olav Løite <[email protected]>
odeke-em added a commit to odeke-em/python-spanner that referenced this issue Dec 20, 2024
@odeke-em
feat(x-goog-spanner-request-id): implement request_id generation and …
65757b5 
…propagation

Generates a request_id that is then injected inside metadata
that's sent over to the Cloud Spanner backend.

Officially inject the first set of x-goog-spanner-request-id values into header metadata
Add request-id interceptor to use in asserting tests
Wrap Snapshot methods with x-goog-request-id metadata injector
Setup scaffolding for XGoogRequestIdHeader checks
Wire up XGoogSpannerRequestIdInterceptor for TestDatabase checks
Inject header in more Session using spots plus more tests
Base for tests with retries on abort
More plumbing for Transaction and Database
Update unit tests for Transaction
Wrap more in Transaction + update tests
Update tests
Plumb in more tests
Update TestDatabase

Fixes googleapis#1261
odeke-em added a commit to odeke-em/python-spanner that referenced this issue Dec 24, 2024
@odeke-em
feat(x-goog-spanner-request-id): implement request_id generation and …
9b79dcb 
…propagation

Generates a request_id that is then injected inside metadata
that's sent over to the Cloud Spanner backend.

Officially inject the first set of x-goog-spanner-request-id values into header metadata
Add request-id interceptor to use in asserting tests
Wrap Snapshot methods with x-goog-request-id metadata injector
Setup scaffolding for XGoogRequestIdHeader checks
Wire up XGoogSpannerRequestIdInterceptor for TestDatabase checks
Inject header in more Session using spots plus more tests
Base for tests with retries on abort
More plumbing for Transaction and Database
Update unit tests for Transaction
Wrap more in Transaction + update tests
Update tests
Plumb in more tests
Update TestDatabase

Fixes googleapis#1261
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rahul2393 rahul2393

Labels
api: spanner Issues related to the googleapis/python-spanner API.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(x-goog-spanner-request-id): implement Request-ID odeke-em/python-spanner
2 participants
@odeke-em @rahul2393