v0.1 in rust #6

escanda · 2022-12-06T23:28:23Z

I open this PR for review. In a Fedora 37 Workstation shell you can do from the project's root: sudo dnf -y install podman-compose && podman-compose up. We have some work to do yet but I open the issue in case you find something shaky we might need to change.

sbates130272

@escanda looks like we have the beginnings of a v0.1. I would like two things.

Address all the comments in my review.
Move to an Ubuntu based approach. Change any Fedora based comments to Ubuntu based ones.
Let's reduce this branch to 3 commits. One for the main code, one for the docker files and docker-compose. And one for documentation (update the README).

.gitignore

Cargo.lock

Cargo.toml

build.rs

docker-compose.yml

fedora-with-blktrace/Dockerfile

prometheus.yml

…pages.

…prometheus, config.

sbates130272

@escanda a few more updates. Getting closer. I should have some time tomorrow to meet.

.github/workflows/rust.yml

sbates130272 · 2022-12-14T02:15:07Z

Dockerfile

@@ -0,0 +1,13 @@
+FROM rust:1.65.0
+RUN apt-get update && \
+    apt-get install -y g++ cmake llvm-dev clang libclang-dev


NIT: @escanda I prefer one package per line. Much cleaner to update that way.

Not sure if best, we may create too many intermediate docker images if we do it by package, each upper case command creates an intermediate image, shall we?

@escanda i mean one run statement but put each package on its own line.

sbates130272 · 2022-12-14T02:16:54Z

.gitignore

@@ -1 +1,5 @@
 *~
+/target


The commit message for this is way to short. Use something like this:

iowatcher-ng: Initial rust-based exporter.

Commit the initial files associated with a rust-based Prometheus exporter for iowatcher-ng. This includes the cargo manifesto and the source code for an initial version.

sbates130272 · 2022-12-14T02:19:59Z

Dockerfile

+COPY . .
+RUN cargo install --path .
+
+FROM ubuntu:22.04


Do we really need both FROM? Can we not do this from Ubuntu 22.04?

Rust image to build is based in debian so no dynamic libs issues transferring then the image to ubuntu. It even helps the build be smaller. Beside rust execs are static by default. What shall we do? :)

sbates130272 · 2022-12-14T02:20:21Z

docker-compose.yml

+
+volumes:
+  prometheus_data: {}
+  grafana-data: {}


Why is one a hyphen and one an underscore. Be consistent please.

sbates130272 · 2022-12-14T02:21:07Z

prometheus.yml

+    static_configs:
+    - targets: ["node-exporter:9100"]
+
+  - job_name: "blkio-exporter"


Why use blkio? Use iowatcher-ng-exporter please.

sbates130272 · 2022-12-14T02:22:42Z

Dockerfile

@@ -0,0 +1,13 @@
+FROM rust:1.65.0


Again the commit message is way to short. Use something like:

docker: Add Dockerfile and docker-compose for project.

The iowatcher-ng exporter runs best inside a docker container so we add a Dockerfile and docker-compose to assist in deployment.

sbates130272 · 2022-12-14T02:23:57Z

docs/index.adoc

@@ -0,0 +1,16 @@
+= iowatcher-ng documentation


For commit message this is not a workflow so do something like

docs: Add documentation

etc.

But to be honest I prefer markdown as it does not need a compile step so let's switch to that.

sbates130272 · 2022-12-14T12:53:00Z

@escanda node that the build step seems to be failing in the workflow?

sbates130272 · 2022-12-14T13:50:52Z

@escanda I think you also want to add a .dockerignore file to ignore the large target folder which may exist. It is not needed in the docker context.

sbates130272 · 2022-12-14T13:55:50Z

batesste@bunbeg:~/Projects/iowatcher-ng$ docker ps
CONTAINER ID   IMAGE                         COMMAND                  CREATED              STATUS         PORTS                                                           NAMES
1fcab7ea4d0e   prom/prometheus:v2.40.5       "/bin/prometheus --c…"   About a minute ago   Up 6 seconds   0.0.0.0:9090->9090/tcp, :::9090->9090/tcp                       prometheus
011a105a9a78   prom/node-exporter:v1.5.0     "/bin/node_exporter …"   About a minute ago   Up 6 seconds   0.0.0.0:9100->9100/tcp, :::9100->9100/tcp                       node-exporter
5cc4d8fe0352   grafana/grafana:7.5.7         "/run.sh"                2 minutes ago        Up 6 seconds   0.0.0.0:3000->3000/tcp, :::3000->3000/tcp                       iowatcher-ng-grafana-1
18f986a584e3   iowatcherng-exporter:v0.1.0   "bash -c 'nc -l -p 9…"   2 minutes ago        Up 6 seconds   0.0.0.0:9000-9001->9000-9001/tcp, :::9000-9001->9000-9001/tcp   iowatcher-ng-blkio-exporter-1

Can we make the container names consistent?

sbates130272 · 2022-12-14T13:58:04Z

Please use port 9975 for iowatcher-ng-exporter as I have reserved this port for this exporter. See here for details.

sbates130272 · 2022-12-14T14:05:50Z

So things seem to be sort of working in the docker-compose. I can see the Prometheus server and the targets all seem to be up. I can see node-exporter on 9100 but I cannot see the iowatcher metrics on 9000 or 9001?

sbates130272 · 2022-12-14T14:06:50Z

Oh and I see the iowatcher contain died:

iowatcher-ng-grafana-1         | t=2022-12-14T14:01:10+0000 lvl=eror msg="Data proxy error" logger=data-proxy-log userId=1 orgId=1 uname=admin path=/api/datasources/proxy/4/api/v1/query remote_addr=172.18.0.1 referer="http://localhost:3000/datasources/edit/4/?utm_source=grafana_gettingstarted" error="http: proxy error: dial tcp 127.0.0.1:9090: connect: connection refused"
iowatcher-ng-grafana-1         | t=2022-12-14T14:01:10+0000 lvl=eror msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=POST path=/api/datasources/proxy/4/api/v1/query status=502 remote_addr=172.18.0.1 time_ms=4 size=0 referer="http://localhost:3000/datasources/edit/4/?utm_source=grafana_gettingstarted"
iowatcher-ng-grafana-1         | t=2022-12-14T14:01:19+0000 lvl=eror msg="Data proxy error" logger=data-proxy-log userId=1 orgId=1 uname=admin path=/api/datasources/proxy/4/api/v1/query remote_addr=172.18.0.1 referer="http://localhost:3000/datasources/edit/4/?utm_source=grafana_gettingstarted" error="http: proxy error: dial tcp 127.0.0.1:9090: connect: connection refused"
iowatcher-ng-grafana-1         | t=2022-12-14T14:01:19+0000 lvl=eror msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=POST path=/api/datasources/proxy/4/api/v1/query status=502 remote_addr=172.18.0.1 time_ms=3 size=0 referer="http://localhost:3000/datasources/edit/4/?utm_source=grafana_gettingstarted"
iowatcher-ng-grafana-1         | t=2022-12-14T14:01:33+0000 lvl=eror msg="Data proxy error" logger=data-proxy-log userId=1 orgId=1 uname=admin path=/api/datasources/proxy/4/api/v1/query remote_addr=172.18.0.1 referer="http://localhost:3000/datasources/edit/4/?utm_source=grafana_gettingstarted" error="http: proxy error: dial tcp 127.0.0.1:9090: connect: connection refused"
iowatcher-ng-grafana-1         | t=2022-12-14T14:01:33+0000 lvl=eror msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=POST path=/api/datasources/proxy/4/api/v1/query status=502 remote_addr=172.18.0.1 time_ms=1 size=0 referer="http://localhost:3000/datasources/edit/4/?utm_source=grafana_gettingstarted"
iowatcher-ng-grafana-1         | t=2022-12-14T14:01:35+0000 lvl=eror msg="Data proxy error" logger=data-proxy-log userId=1 orgId=1 uname=admin path=/api/datasources/proxy/4/api/v1/query remote_addr=172.18.0.1 referer="http://localhost:3000/datasources/edit/4/?utm_source=grafana_gettingstarted" error="http: proxy error: dial tcp 127.0.0.1:9090: connect: connection refused"
iowatcher-ng-grafana-1         | t=2022-12-14T14:01:35+0000 lvl=eror msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=POST path=/api/datasources/proxy/4/api/v1/query status=502 remote_addr=172.18.0.1 time_ms=0 size=0 referer="http://localhost:3000/datasources/edit/4/?utm_source=grafana_gettingstarted"
iowatcher-ng-grafana-1         | t=2022-12-14T14:01:47+0000 lvl=eror msg="Data proxy error" logger=data-proxy-log userId=1 orgId=1 uname=admin path=/api/datasources/proxy/4/api/v1/query remote_addr=172.18.0.1 referer="http://localhost:3000/datasources/edit/4/?utm_source=grafana_gettingstarted" error="http: proxy error: dial tcp 127.0.0.1:9090: connect: connection refused"
iowatcher-ng-grafana-1         | t=2022-12-14T14:01:47+0000 lvl=eror msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=POST path=/api/datasources/proxy/4/api/v1/query status=502 remote_addr=172.18.0.1 time_ms=1 size=0 referer="http://localhost:3000/datasources/edit/4/?utm_source=grafana_gettingstarted"
iowatcher-ng-blkio-exporter-1  | Bad pkt magic
iowatcher-ng-blkio-exporter-1 exited with code 0

sbates130272 · 2022-12-18T20:22:47Z

When running docker-compose up

iowatcherng_exporter  | bash: line 1: iowatcherng-exporter: command not found

gitguardian · 2022-12-29T01:13:54Z

⚠️ GitGuardian has uncovered 3 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	Secret	Commit	Filename
5353261	RSA Private Key	`2309eb5`	certs/ca.key	View secret
5353262	RSA Private Key	`2309eb5`	certs/client1.key	View secret
5353263	RSA Private Key	`2309eb5`	certs/localhost.key	View secret

🛠 Guidelines to remediate hardcoded secrets

Understand the implications of revoking this secret by investigating where it is used in your code.
Replace and store your secrets safely. Learn here the best practices.
Revoke and rotate these secrets.
If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

following these best practices for managing and storing secrets including API keys and other credentials
install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Our GitHub checks need improvements? Share your feedbacks!}

escanda requested a review from sbates130272 December 6, 2022 23:28

escanda self-assigned this Dec 6, 2022

escanda mentioned this pull request Dec 6, 2022

Version 0.1 #5

Open

sbates130272 requested changes Dec 8, 2022

View reviewed changes

escanda mentioned this pull request Dec 8, 2022

added rust action #7

Closed

escanda force-pushed the gcifuentes/dev/rust branch from 8d7cc97 to 17e6e84 Compare December 12, 2022 23:01

escanda and others added 4 commits December 13, 2022 00:58

workflow: github action on build & clippy lint, also github asciidoc …

39147ab

…pages.

rust: initial metric

6e46ae5

docker: descriptors of docker build, docker-compose and grafana, and …

4521bae

…prometheus, config.

workflow: added asciidoctor docs.

5b2fd44

escanda force-pushed the gcifuentes/dev/rust branch from 17e6e84 to 5b2fd44 Compare December 13, 2022 00:09

Merge branch 'escanda-patch-1' into gcifuentes/dev/rust

e058d2d

escanda requested a review from sbates130272 December 13, 2022 00:19

workflow: fixed gh-pages generation.

e92f637

escanda force-pushed the gcifuentes/dev/rust branch from 6319aa9 to e92f637 Compare December 13, 2022 00:35

sbates130272 requested changes Dec 14, 2022

View reviewed changes

Guillermo Cifuentes added 8 commits December 16, 2022 14:03

gh-pages workbook

1167a9d

more changes

ffa3187

fix ci

2e05738

fix ci

37dc8aa

more ci

36c6a44

more ci

19ff050

more ci

edecff8

more ci

f369bb5

Guillermo Cifuentes added 15 commits December 16, 2022 15:16

wip ci

e82889e

wip

87cf37e

more ci

a10f1f9

Merge branch 'gh-pages' into gcifuentes/dev/rust

c797ded

more ci

541b973

cleanup docs

4cfba2b

more ci

147f97a

fixed makefile

da7e03e

fix cwd in ci.yml

227bf12

more ci

163f8b0

more ci

eeed954

more ci

5ef6d2e

various fixes

a3f470e

wip

dd39524

prometheus port fix

0ef81bb

Guillermo Cifuentes and others added 3 commits December 19, 2022 09:16

blktrace over quicc wip

cc0a89f

stash

64a6cf2

quicc connects

2309eb5

Guillermo Cifuentes added 3 commits December 30, 2022 14:18

refactor wip

9ab4e59

stash

5e77801

renamed fn

56445d6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v0.1 in rust #6

v0.1 in rust #6

escanda commented Dec 6, 2022

sbates130272 left a comment

sbates130272 left a comment

sbates130272 Dec 14, 2022

escanda Dec 14, 2022

sbates130272 Dec 15, 2022

sbates130272 Dec 14, 2022

sbates130272 Dec 14, 2022

escanda Dec 14, 2022

sbates130272 Dec 14, 2022

sbates130272 Dec 14, 2022

sbates130272 Dec 14, 2022

sbates130272 Dec 14, 2022

sbates130272 commented Dec 14, 2022

sbates130272 commented Dec 14, 2022

sbates130272 commented Dec 14, 2022

sbates130272 commented Dec 14, 2022

sbates130272 commented Dec 14, 2022

sbates130272 commented Dec 14, 2022

sbates130272 commented Dec 18, 2022

gitguardian bot commented Dec 29, 2022 •

edited

Loading

		@@ -1 +1,5 @@
		*~
		/target

v0.1 in rust #6

Are you sure you want to change the base?

v0.1 in rust #6

Conversation

escanda commented Dec 6, 2022

sbates130272 left a comment

Choose a reason for hiding this comment

sbates130272 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sbates130272 commented Dec 14, 2022

sbates130272 commented Dec 14, 2022

sbates130272 commented Dec 14, 2022

sbates130272 commented Dec 14, 2022

sbates130272 commented Dec 14, 2022

sbates130272 commented Dec 14, 2022

sbates130272 commented Dec 18, 2022

gitguardian bot commented Dec 29, 2022 • edited Loading

⚠️ GitGuardian has uncovered 3 secrets following the scan of your pull request.

gitguardian bot commented Dec 29, 2022 •

edited

Loading