Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pubsub-open-telemetry): add quickstart example #265

Merged
merged 10 commits into from
Nov 28, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions pubsub-open-telemetry/.bazelrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# To workaround a bug in Bazel [1], gRPC requires this flag on macOS builds,
# and there is (AFAIK) no way to "inherit" their definitions.
# [1]: https://github.com/bazelbuild/bazel/issues/4341
build --copt=-DGRPC_BAZEL_BUILD
alevenberg marked this conversation as resolved.
Show resolved Hide resolved

# Use host-OS-specific config lines from bazelrc files.
build --enable_platform_specific_config=true

# The project requires C++ >= 14. By default Bazel adds `-std=c++0x` which
# disables C++14 features, even if the compilers defaults to C++ >= 14
build:linux --cxxopt=-std=c++14
build:macos --cxxopt=-std=c++14

# Do not create the convenience links. They are inconvenient when the build
# runs inside a docker image.
build --experimental_convenience_symlinks=ignore

# Enable OpenTelemetry tracing instrumentation for google-cloud-cpp.
build --@io_opentelemetry_cpp//api:with_abseil
build --@google_cloud_cpp//:enable_opentelemetry
26 changes: 26 additions & 0 deletions pubsub-open-telemetry/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

package(default_visibility = ["//visibility:private"])

licenses(["notice"]) # Apache 2.0

cc_binary(
name = "quickstart",
srcs = ["quickstart.cc"],
deps = [
"@google_cloud_cpp//:opentelemetry",
"@google_cloud_cpp//:pubsub",
],
)
35 changes: 35 additions & 0 deletions pubsub-open-telemetry/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# ~~~
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ~~~

cmake_minimum_required(VERSION 3.20)

# Define the project name and where to report bugs.
set(PACKAGE_BUGREPORT
"https://github.com/GoogleCloudPlatform/cpp-samples/issues")
project(pubsub-open-telemetry CXX)

find_package(google_cloud_cpp_pubsub CONFIG REQUIRED)
find_package(google_cloud_cpp_opentelemetry CONFIG REQUIRED)
find_package(google_cloud_cpp_trace CONFIG REQUIRED)
alevenberg marked this conversation as resolved.
Show resolved Hide resolved

add_executable(quickstart quickstart.cc)
target_compile_features(quickstart PRIVATE cxx_std_20)
alevenberg marked this conversation as resolved.
Show resolved Hide resolved
target_link_libraries(
quickstart PRIVATE google-cloud-cpp::pubsub google-cloud-cpp::opentelemetry
google-cloud-cpp::trace)
alevenberg marked this conversation as resolved.
Show resolved Hide resolved

include(GNUInstallDirs)
install(TARGETS quickstart RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
alevenberg marked this conversation as resolved.
Show resolved Hide resolved
166 changes: 166 additions & 0 deletions pubsub-open-telemetry/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
# Enabling Open Telemetry for the Pub/Sub library with Cloud Trace

## Overview
alevenberg marked this conversation as resolved.
Show resolved Hide resolved

In v2.16, we GA'd [OpenTelemetry tracing](https://github.com/googleapis/google-cloud-cpp/blob/v2.16.0/CHANGELOG.md#v2160---2023-10). This provides basic instrumentation for all the google-cloud-cpp libraries.
alevenberg marked this conversation as resolved.
Show resolved Hide resolved

In v2.19 release[^1], we added instrumentation for the Google Cloud Pub/Sub C++ library on the Publish side. This example provides a basic tracing application that exports spans to Cloud Trace.

[^1]: The [telemetry data](https://github.com/googleapis/google-cloud-cpp/blob/main/doc/public-api.md#telemetry-data) emitted by the google-cloud-cpp library does not follow any versioning guarentees and is subject to change without notice in later versions.
alevenberg marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites

### 1. Create a project in the Google Cloud Platform Console

If you haven't already created a project, create one now.

Projects enable you to manage all Google Cloud Platform resources for your app, including deployment, access control, billing, and services.

1. Open the [Cloud Platform Console](https://console.cloud.google.com/).
2. In the drop-down menu at the top, select Create a project.
3. Give your project a name.
4. Make a note of the project ID, which might be different from the project name. The project ID is used in commands and in configurations.

### 2. Enable billing for your project
If you haven't already enabled billing for your
project, [enable billing now](https://console.cloud.google.com/project/_/settings). Enabling billing allows the
application to consume billable resources such as Pub/Sub API calls.

See [Cloud Platform Console Help](https://support.google.com/cloud/answer/6288653) for more information about billing
settings.

### 3. Enable APIs for your project
[Click here](https://console.cloud.google.com/flows/enableapi?apiid=speech&showconfirmation=true) to visit Cloud
Platform Console and enable the Pub/Sub and Trace API via the UI.

Or use the CLI:

```
gcloud services enable trace.googleapis.com
gcloud services enable pubsub.googleapis.com
```

### 4. If needed, override the Billing Project

If you are using a [user account] for authentication, you need to set the `GOOGLE_CLOUD_CPP_USER_PROJECT`
environment variable to the project you created in the previous step. Be aware that you must have
`serviceusage.services.use` permission on the project. Alternatively, use a service account as described next.

[user account]: https://cloud.google.com/docs/authentication#principals
alevenberg marked this conversation as resolved.
Show resolved Hide resolved

#### Download service account credentials

These samples can use service accounts for authentication.

1. Visit the [Cloud Console](http://cloud.google.com/console), and navigate to:
`API Manager > Credentials > Create credentials > Service account key`
2. Under **Service account**, select `New service account`.
3. Under **Service account name**, enter a service account name of your choosing. For example, `transcriber`.
4. Under **Role**, select `Project > Owner`.
5. Under **Key type**, leave `JSON` selected.
6. Click **Create** to create a new service account, and download the json credentials file.
7. Set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to point to your downloaded service account
credentials:
```
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/your/credentials-key.json
```
See the [Cloud Platform Auth Guide](https://cloud.google.com/docs/authentication#developer_workflow) for more
information.

alevenberg marked this conversation as resolved.
Show resolved Hide resolved
### 5. Create the Cloud Pub/Sub topic
alevenberg marked this conversation as resolved.
Show resolved Hide resolved

```sh
export=GOOGLE_CLOUD_PROJECT=<project-id>
export=GOOGLE_CLOUD_TOPIC=<topic-id>
gcloud pubsub topics create "--project=${GOOGLE_CLOUD_PROJECT}" ${GOOGLE_CLOUD_TOPIC}
```

## Build and run using CMake and Vcpkg
### 1. Install vcpkg
This project uses [`vcpkg`](https://github.com/microsoft/vcpkg) for dependency management. Clone the vcpkg repository
to your preferred location. In these instructions we use`$HOME`:
```shell
git clone -C $HOME https://github.com/microsoft/vcpkg.git
cd $HOME/vcpkg
./vcpkg install google-cloud-cpp
```

### 2. Download or clone this repo

```shell
git clone https://github.com/GoogleCloudPlatform/cpp-samples
```

### 3. Compile these examples

Use the `vcpkg` toolchain file to download and compile dependencies. This file would be in the directory you
cloned `vcpkg` into, `$HOME/vcpkg` if you are following the instructions to the letter. Note that building all the
dependencies can take up to an hour, depending on the performance of your workstation. These dependencies are cached,
so a second build should be substantially faster.
```sh
cd cpp-samples/pubsub-open-telemetry
cmake -S . -B .build -DCMAKE_TOOLCHAIN_FILE=$HOME/vcpkg/scripts/buildsystems/vcpkg.cmake -G Ninja
cmake --build .build
```

### 4. Run the examples

```shell
.build/quickstart [project-name] [topic-id]
```

## Build and run using Bazel

### 1. Download or clone this repo

```shell
git clone https://github.com/GoogleCloudPlatform/cpp-samples
```

### 2. Compile these examples

```shell
cd cpp-samples/pubsub-open-telemetry
bazel build //:quickstart
```

### 3. Run these examples

```shell
bazel run //:quickstart [project-name] [topic-id]
```

#### Run with a local version of google-cloud-cpp

```shell
bazel run //:quickstart --override_repository=google_cloud_cpp=$HOME/your-path-to-the-repo/google-cloud-cpp -- [project-name] [topic-id]
```

alevenberg marked this conversation as resolved.
Show resolved Hide resolved
## Platform Specific Notes

### macOS

gRPC [requires][grpc-roots-pem-bug] an environment variable to configure the
trust store for SSL certificates, you can download and configure this using:

```bash
curl -Lo roots.pem https://pki.google.com/roots.pem
export GRPC_DEFAULT_SSL_ROOTS_FILE_PATH="$PWD/roots.pem"
```

### Windows

gRPC [requires][grpc-roots-pem-bug] an environment variable to configure the
trust store for SSL certificates, you can download and configure this using:

```console
@powershell -NoProfile -ExecutionPolicy unrestricted -Command ^
(new-object System.Net.WebClient).Downloadfile( ^
'https://pki.google.com/roots.pem', 'roots.pem')
set GRPC_DEFAULT_SSL_ROOTS_FILE_PATH=%cd%\roots.pem
```

[grpc-roots-pem-bug]: https://github.com/grpc/grpc/issues/16571
[choco-cmake-link]: https://chocolatey.org/packages/cmake
[homebrew-cmake-link]: https://formulae.brew.sh/formula/cmake
[cmake-download-link]: https://cmake.org/download/
37 changes: 37 additions & 0 deletions pubsub-open-telemetry/WORKSPACE.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

workspace(name = "pubsub-open-telemetery")

# Google Cloud Cpp
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "google_cloud_cpp",
sha256 = "7ca7f583b60d2aa1274411fed3b9fb3887119b2e84244bb3fc69ea1db819e4e5",
strip_prefix = "google-cloud-cpp-2.16.0",
url = "https://github.com/googleapis/google-cloud-cpp/archive/v2.16.0.tar.gz",
)

load("@google_cloud_cpp//bazel:google_cloud_cpp_deps.bzl", "google_cloud_cpp_deps")
google_cloud_cpp_deps()
load("@com_google_googleapis//:repository_rules.bzl", "switched_rules_by_language")
switched_rules_by_language(
name = "com_google_googleapis_imports",
cc = True,
grpc = True,
)
load("@com_github_grpc_grpc//bazel:grpc_deps.bzl", "grpc_deps")
grpc_deps()
load("@com_github_grpc_grpc//bazel:grpc_extra_deps.bzl", "grpc_extra_deps")
grpc_extra_deps()
63 changes: 63 additions & 0 deletions pubsub-open-telemetry/quickstart.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#include "google/cloud/opentelemetry/configure_basic_tracing.h"
#include "google/cloud/opentelemetry_options.h"
#include "google/cloud/pubsub/publisher.h"
#include <iostream>
#include <string_view>
alevenberg marked this conversation as resolved.
Show resolved Hide resolved

// Create a few namespace aliases to make the code easier to read.
namespace gc = ::google::cloud;
namespace pubsub = gc::pubsub;
namespace otel = gc::otel;

int main(int argc, char* argv[]) try {
if (argc != 3) {
std::cerr << "Usage: " << argv[0] << " <project-id> <topic-id>\n";
return 1;
}
std::string const project_id = argv[1];
std::string const topic_id = argv[2];
auto project = gc::Project(project_id);

//! [START pubsub_publish_otel_tracing]
auto configuration = otel::ConfigureBasicTracing(project);
alevenberg marked this conversation as resolved.
Show resolved Hide resolved
auto publisher = pubsub::Publisher(pubsub::MakePublisherConnection(
pubsub::Topic(project_id, topic_id),
gc::Options{}.set<gc::OpenTelemetryTracingOption>(true)));

std::vector<gc::future<void>> ids;
coryan marked this conversation as resolved.
Show resolved Hide resolved
alevenberg marked this conversation as resolved.
Show resolved Hide resolved
for (int i = 0; i < 5; i++) {
auto id = publisher.Publish(pubsub::MessageBuilder().SetData("Hi!").Build())
.then([](gc::future<gc::StatusOr<std::string>> f) {
auto status = f.get();
alevenberg marked this conversation as resolved.
Show resolved Hide resolved
if (!status) {
std::cout << "Error in publish: " << status.status()
<< "\n";
return;
}
std::cout << "Sent message with id: (" << *status << ")\n";
});
ids.push_back(std::move(id));
alevenberg marked this conversation as resolved.
Show resolved Hide resolved
}
// Block until they are actually sent.
alevenberg marked this conversation as resolved.
Show resolved Hide resolved
for (auto& id : ids) id.get();
//! [END pubsub_publish_otel_tracing]

return 0;
} catch (google::cloud::Status const& status) {
alevenberg marked this conversation as resolved.
Show resolved Hide resolved
std::cerr << "google::cloud::Status thrown: " << status << "\n";
return 1;
}
16 changes: 16 additions & 0 deletions pubsub-open-telemetry/vcpkg.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"name": "gcp-cpp-samples-pubsub-open-telemetry-publisher",
"version-string": "unversioned",
"homepage": "https://github.com/GoogleCloudPlatform/cpp-samples/",
"description": "An example using Open Telemetry and the Pub/Sub library",
"dependencies": [
{
"name": "google-cloud-cpp",
"features": [
"pubsub",
"trace",
alevenberg marked this conversation as resolved.
Show resolved Hide resolved
"opentelemetry"
]
}
]
}