Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add updated span events + trace more methods #1259

Conversation

odeke-em
Copy link
Contributor

@odeke-em odeke-em commented Dec 6, 2024

This change carves out parts of PR #1241 in smaller pieces to ease with smaller reviews.
This change adds more span events, updates important spans to make them more distinct like changing:

"CloudSpanner.ReadWriteTransaction" to more direct and more pointed spans like:

  • CloudSpanner.Transaction.execute_streaming_sql

Also added important spans:

  • CloudSpanner.Database.run_in_transaction
  • CloudSpanner.Session.run_in_transaction

@odeke-em odeke-em requested review from a team as code owners December 6, 2024 15:03
@product-auto-label product-auto-label bot added size: l Pull request size is large. api: spanner Issues related to the googleapis/python-spanner API. labels Dec 6, 2024
@harshachinta harshachinta added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 6, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 6, 2024
@odeke-em odeke-em force-pushed the trace-update-trace_call-with-db_name-retrieval branch 2 times, most recently from b0d9a6c to f437955 Compare December 7, 2024 03:09
@odeke-em
Copy link
Contributor Author

odeke-em commented Dec 7, 2024

@sakthivelmanii @harshachinta kindly help me run the bots. Thank you

@harshachinta harshachinta added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 7, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 7, 2024
# Empty context manager. Users will have to check if the generated value is None or a span
yield None
return

if session:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@odeke-em
We should move this latest change of updating session last use time to beginning of this function, otherwise when opentlemetry is not installed then that update will never happen.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@odeke-em
this is not yet addresses.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed that @harshachinta as it is from the bigger PR and I don't think we shall need it anymore once some tests are sent in the other PR. I deleted it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not from your PR, this was added by us to handle one case. Don't remove that
I am talking about this code. We need this to know when was a session last used to avoid making a backend call to verify session existence.

    if session:
        session._last_use_time = datetime.now()

google/cloud/spanner_v1/_opentelemetry_tracing.py Outdated Show resolved Hide resolved
google/cloud/spanner_v1/_opentelemetry_tracing.py Outdated Show resolved Hide resolved
google/cloud/spanner_v1/_opentelemetry_tracing.py Outdated Show resolved Hide resolved
@@ -70,6 +73,10 @@ def insert(self, table, columns, values):
:param values: Values to be modified.
"""
self._mutations.append(Mutation(insert=_make_write_pb(table, columns, values)))
add_event_on_current_span(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand this requirement was bought out when we met last time.
Thinking out loud.

Cloud spanner has a mutation limit of 40000 in one request.
So lets say a customer adds mutations in a loop, then this will print 40,000 events on the span.
Is this a good user experience? Also this could add up to the storage cost on customer end?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great question but it won't be a problem because:

What we can do is perhaps document how to change span event limits and also document that we add events for span mutations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sakthivelmanii
I’d like to get your thoughts on this.

In my opinion, adding a separate event for every mutation could clutter the trace spans with too many events. Instead, I’d suggest avoiding one event per mutation. We already track the num_mutations attribute in the Commit request (here), which effectively provides customers with insights into the number of mutations in a commit.

Let me know what you think.

google/cloud/spanner_v1/session.py Show resolved Hide resolved
google/cloud/spanner_v1/session.py Outdated Show resolved Hide resolved
google/cloud/spanner_v1/session.py Outdated Show resolved Hide resolved
google/cloud/spanner_v1/session.py Outdated Show resolved Hide resolved
google/cloud/spanner_v1/session.py Outdated Show resolved Hide resolved
tests/_helpers.py Outdated Show resolved Hide resolved
tests/system/test_session_api.py Outdated Show resolved Hide resolved
google/cloud/spanner_v1/snapshot.py Show resolved Hide resolved
tests/unit/test_pool.py Outdated Show resolved Hide resolved
tests/unit/test_pool.py Outdated Show resolved Hide resolved
tests/unit/test_pool.py Show resolved Hide resolved
tests/unit/test_pool.py Outdated Show resolved Hide resolved
Comment on lines 797 to 802
"CloudSpanner.CreateSession",
"CloudSpanner.Transaction.execute_streaming_sql",
"CloudSpanner.Transaction.execute_streaming_sql",
"CloudSpanner.Transaction.commit",
"CloudSpanner.Session.run_in_transaction",
"CloudSpanner.Database.run_in_transaction",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is some mistake in ordering here. Can you check?
Ideally the order should be,

        "CloudSpanner.Database.run_in_transaction",
        "CloudSpanner.CreateSession",
        "CloudSpanner.Session.run_in_transaction",
        "CloudSpanner.Transaction.execute_streaming_sql",
        "CloudSpanner.Transaction.execute_streaming_sql",
        "CloudSpanner.Transaction.commit",
        ```

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It isn't a mistake though but rather the order in which spans are ended @harshachinta.

tests/system/test_session_api.py Outdated Show resolved Hide resolved
tests/system/test_session_api.py Outdated Show resolved Hide resolved
@odeke-em odeke-em force-pushed the trace-update-trace_call-with-db_name-retrieval branch 3 times, most recently from 992d6c9 to 6d6b30a Compare December 10, 2024 10:25
@harshachinta harshachinta added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 10, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 10, 2024
@odeke-em odeke-em changed the title observability: add updated span events + traace more methods observability: add updated span events + trace more methods Dec 16, 2024
@@ -70,6 +73,10 @@ def insert(self, table, columns, values):
:param values: Values to be modified.
"""
self._mutations.append(Mutation(insert=_make_write_pb(table, columns, values)))
add_event_on_current_span(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sakthivelmanii
I’d like to get your thoughts on this.

In my opinion, adding a separate event for every mutation could clutter the trace spans with too many events. Instead, I’d suggest avoiding one event per mutation. We already track the num_mutations attribute in the Commit request (here), which effectively provides customers with insights into the number of mutations in a commit.

Let me know what you think.

Comment on lines +364 to +380
"exception",
{
"exception.type": "IndexError",
"exception.message": "pop from empty list",
"exception.stacktrace": "EPHEMERAL",
"exception.escaped": "False",
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is there a repeation of exception event?

Copy link
Contributor Author

@odeke-em odeke-em Dec 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So given that we have a direct invocation of with trace_call("testBind") per

        try: 
            with trace_call("testBind", fauxSession):
                pool.bind(database)
        except Exception:
            pass 

that then invokes code internally where an exception happens. With the exception re-raised and unhandled until the highest parent, OpenTelemetry records the exception even on the parent. If we swallowed the exception internally we wouldn't

tests/system/test_observability_options.py Outdated Show resolved Hide resolved
@odeke-em odeke-em force-pushed the trace-update-trace_call-with-db_name-retrieval branch 2 times, most recently from 20fe353 to a69a4dc Compare December 16, 2024 08:54
@harshachinta harshachinta added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 16, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 16, 2024
@harshachinta harshachinta changed the title observability: add updated span events + trace more methods feat: add updated span events + trace more methods Dec 16, 2024
@harshachinta harshachinta added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 16, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 16, 2024
This change carves out parts of PR googleapis#1241 in smaller pieces to
ease with smaller reviews.
This change adds more span events, updates important spans
to make them more distinct like changing:

"CloudSpanner.ReadWriteTransaction" to more direct and more
pointed spans like:
* CloudSpanner.Transaction.execute_streaming_sql

Also added important spans:
* CloudSpanner.Database.run_in_transaction
* CloudSpanner.Session.run_in_transaction
Referencing issue googleapis#1269, this update removes adding
a span event per mutation, in favour of a future TODO.
@odeke-em odeke-em force-pushed the trace-update-trace_call-with-db_name-retrieval branch from 5208f8e to 37594f7 Compare December 16, 2024 12:21
@harshachinta harshachinta added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 16, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 16, 2024
@harshachinta harshachinta merged commit ad69c48 into googleapis:main Dec 17, 2024
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the googleapis/python-spanner API. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants