Add transformation test make target and docs (#1722)

Signed-off-by: braydonk <[email protected]>

Author: braydonk

.gitignore

@@ -1,6 +1,14 @@
+# Ignore a file actually called Makefile. The presence of this file interferes
+# with our Linux package building processes. To make a local symlink called
+# "Makefile" for development convenience, use this command:
+# make -f tasks.mak makefile_symlink
 Makefile
 
 # Ignore generated credentials from google-github-actions/auth
 gha-creds-*.json
 
-.vscode/launch.json
+# This is a folder used to hold local builds of fluent-bit and otelopscol
+# when running transformation tests locally.
+dist/
+
+.vscode/launch.json

builds/agent_wrapper.sh

@@ -1,4 +1,18 @@
 #!/bin/bash
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 set -x -e
 DESTDIR=$1
 mkdir -p "$DESTDIR/opt/google-cloud-ops-agent/libexec"

builds/fluent_bit.sh

@@ -1,4 +1,18 @@
 #!/bin/bash
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 set -x -e
 DESTDIR=$1
 mkdir -p $DESTDIR

builds/ops_agent_diagnostics.sh

@@ -1,4 +1,18 @@
 #!/bin/bash
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 set -x -e
 DESTDIR=$1
 mkdir -p "$DESTDIR/opt/google-cloud-ops-agent/libexec"

builds/otel.sh

@@ -1,26 +1,51 @@
 #!/bin/bash
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 set -x -e
-DESTDIR=$1
+
 otel_dir=/opt/google-cloud-ops-agent/subagents/opentelemetry-collector
-DESTDIR="${DESTDIR}${otel_dir}"
-
-cd submodules/opentelemetry-java-contrib
-mkdir -p "$DESTDIR"
-./gradlew --no-daemon -Djdk.lang.Process.launchMechanism=vfork :jmx-metrics:build
-cp jmx-metrics/build/libs/opentelemetry-jmx-metrics-*-alpha-SNAPSHOT.jar "$DESTDIR/opentelemetry-java-contrib-jmx-metrics.jar"
-
-# Rename LICENSE file because it causes issues with file hash consistency due to an unknown
-# issue with the debuild/rpmbuild processes. Something is unzipping the jar in a case-insensitive
-# environment and having a conflict between the LICENSE file and license/ directory, leading to a changed jar file
-mkdir ./META-INF
-unzip -j "$DESTDIR/opentelemetry-java-contrib-jmx-metrics.jar" "META-INF/LICENSE" -d ./META-INF
-zip -d "$DESTDIR/opentelemetry-java-contrib-jmx-metrics.jar" "META-INF/LICENSE"
-mv ./META-INF/LICENSE ./META-INF/LICENSE.renamed
-zip -u "$DESTDIR/opentelemetry-java-contrib-jmx-metrics.jar" "META-INF/LICENSE.renamed"
-
-cd ../opentelemetry-operations-collector
-# Using array assignment to drop the filename from the sha256sum output
-JAR_SHA_256=($(sha256sum "$DESTDIR/opentelemetry-java-contrib-jmx-metrics.jar"))
-go build -tags=gpu -buildvcs=false -o "$DESTDIR/otelopscol" \
--ldflags "-X github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jmxreceiver.MetricsGathererHash=$JAR_SHA_256" \
-./cmd/otelopscol
+DESTDIR="${1}${otel_dir}"
+
+mkdir -p $DESTDIR
+
+LDFLAGS=""
+
+# If this is not a local build, then we build otel java and add it to
+# the ldflags.
+if [ "$SKIP_OTEL_JAVA" != "true" ]; then
+    cd submodules/opentelemetry-java-contrib
+
+    mkdir -p "$DESTDIR"
+    ./gradlew --no-daemon -Djdk.lang.Process.launchMechanism=vfork :jmx-metrics:build
+    cp jmx-metrics/build/libs/opentelemetry-jmx-metrics-*-alpha-SNAPSHOT.jar "$DESTDIR/opentelemetry-java-contrib-jmx-metrics.jar"
+
+    # Rename LICENSE file because it causes issues with file hash consistency due to an unknown
+    # issue with the debuild/rpmbuild processes. Something is unzipping the jar in a case-insensitive
+    # environment and having a conflict between the LICENSE file and license/ directory, leading to a changed jar file
+    mkdir ./META-INF
+    unzip -j "$DESTDIR/opentelemetry-java-contrib-jmx-metrics.jar" "META-INF/LICENSE" -d ./META-INF
+    zip -d "$DESTDIR/opentelemetry-java-contrib-jmx-metrics.jar" "META-INF/LICENSE"
+    mv ./META-INF/LICENSE ./META-INF/LICENSE.renamed
+    zip -u "$DESTDIR/opentelemetry-java-contrib-jmx-metrics.jar" "META-INF/LICENSE.renamed"
+
+    # Using array assignment to drop the filename from the sha256sum output
+    JAR_SHA_256=($(sha256sum "$DESTDIR/opentelemetry-java-contrib-jmx-metrics.jar"))
+    LDFLAGS="-X github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jmxreceiver.MetricsGathererHash=$JAR_SHA_256"
+
+    cd ../..
+fi
+
+cd submodules/opentelemetry-operations-collector
+go build -tags=gpu -buildvcs=false -o "$DESTDIR/otelopscol" -ldflags "$LDFLAGS" ./cmd/otelopscol

builds/systemd.sh

@@ -1,4 +1,18 @@
 #!/bin/bash
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 set -x -e
 DESTDIR=$1
 

cloudbuild/new-distro-detector/run.sh

@@ -1,4 +1,18 @@
 #!/bin/bash
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 
 # See go/sdi-new-distro-detector for details on how this script is being used.
 # A short summary is that it is run periodically by GCB to file bugs when

cmd/agent_wrapper/main_test.go

@@ -1,3 +1,17 @@
+// Copyright 2024 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package main
 
 import (

confgenerator/feature_tracking_internal_test.go

@@ -1,3 +1,17 @@
+// Copyright 2024 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package confgenerator
 
 import (

dev-docs/makefile.md

@@ -6,14 +6,18 @@ The Makefile `tasks.mak` is a Makefile for the project that is used to quickly p
 
 We are unable to use the actual `Makefile` name in this repo; because we build the Ops Agent into a deb package, `debhelper` tries to autodetect `make` as the build system in most of its `dh_auto` steps. We do not want `debhelper` to pick up the Makfile and try picking up different targets to build and test it.
 
+## Disclaimer
+
+`tasks.mak` is provided without any guarantees or warranty on its targets. It is meant purely for developer convenience, and it is advised not to make any dependency on the targets since they are subject to change at any time.
+
 ## Usage
 
 For convenience it helps to make a symlink from `Makefile` to `tasks.mak` . There is a target to do this, to run it:
 ```
 make -f tasks.mak makefile_symlink
 ```
 
-Now you can call the targets simply with the `make` command without needing to specify the `tasks.mak` file. The file `Makefile` is gitignored, since we don't want to push it to the repo for the reasons stated in the previous section. 
+Now you can call the targets simply with the `make` command without needing to specify the `tasks.mak` file. The file `Makefile` is gitignored, since we don't want to push it to the repo for the reasons stated in the previous section.
 
 To run something from the Makefile, you will need to give `make` the `-f` (file) flag with a path to `tasks.mak`. For example, to run the `test` target:
 ```
@@ -35,8 +39,4 @@ A new target should be added to `tasks.mak` if:
 * There's a new package with Unit Tests
 * There is something that developers may need to run often, especially if it's a complicated command
 * There are new tasks to run in CI
-* There is a new development tool to be installed and used 
-
-## Disclaimer
-
-`tasks.mak` is provided without any guarantees or warranty on its targets. It is meant purely for developer convenience, and it is advised not to make any dependency on the targets since they are subject to change at any time.
+* There is a new development tool to be installed and used