Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CNCF OpenTelemetry with optional gzip compression #1980

Closed
wants to merge 2 commits into from

Conversation

darach
Copy link
Member

@darach darach commented Sep 22, 2022

Pull request

This is a manual patch / cherrypick of daksh's contribution for CNCF OpenTelemetry with optional gzip
compression support originated in the stalled #1818 branch. Recent changes in external dependencies
and to tremor mean this work is now mergeable.

@dak-x Let me know if you'd prefer to resolve #1818 or we can go ahead with this branch which basically
avoids most of the merge pain!

Description

Adds compression support to the otel connectors:

  define connector otel_client from otel_client
  with
    config = {
      "url": "127.0.0.1:4317",
      "compression": "gzip", # This is new from Daksh's contribution
    },
    reconnect = {
      "retry": {
        "interval_ms": 100,
        "growth_rate": 2,
        "max_retries": 3,
      }
    }
  end;

Related

Supercedes #1818

Checklist

  • The RFC, if required, has been submitted and approved
  • Any user-facing impact of the changes is reflected in docs.tremor.rs
  • The code is tested
  • Use of unsafe code is reasoned about in a comment
  • Update CHANGELOG.md appropriately, recording any changes, bug fixes, or other observable changes in behavior
  • The performance impact of the change is measured (see below)

Performance

Compression is a tradeoff of network bandwidth utilisation and cpu utilization. Unless explicitly configured
compression is disabled by default.

@darach darach mentioned this pull request Sep 22, 2022
6 tasks
Signed-off-by: Darach Ennis <[email protected]>
@dak-x
Copy link
Contributor

dak-x commented Nov 6, 2022

I believe we would need to spend effort in upgrading our tonic in termor-otelapis. Tonic 0.6 and 0.7 both are being consumed due to a dependency on googapis. I remember @mfelsche mentioning a solution in this regard (couldn't find it).

@darach
Copy link
Member Author

darach commented Nov 7, 2022

Yes. Its a fairly deep set of changes given that we need a common tonic/prost across all gRPC or protocol buffer
based connectors. Additionally the GCP and Otel specifications have revv'd resulting in some other changes to
underlying deps. I have most of the changes in progress with a preview/draft branch hopefully soon!

@Licenser
Copy link
Member

Licenser commented Dec 8, 2022

closing this as its part of the huge otel update coming down the line

@Licenser Licenser closed this Dec 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants