fix(slos): BoolGauge promql expr should not be multiplied by 100

[alexberry]

While building out an uptime SLO using a BoolGauge we performed some simple testing by forcing a failure and observed the following issues:

• The promql expression linked from the errors graph starts with 100 *, which returns a true percentage
• The errors graph expects a percentage represented as a decimal fraction (i.e. a float between 0 & 1), and are shared amongst all slos
• The burn rates also depend upon this interpretation

This resulted in the error graph making sense for our Ratio SLOs, however for a BoolGauge SLOs would return 10000% failure in the error graph (as opposed to the expected 100%), and non-sensical burn rates.

This fixes #1257

[deanb-everc]

would also appreciate for this PR to be merged, seems like we are facing a similar issue

[deanb-everc]

any news here? we want to start using pyrra mainly with bool gauges, and currently it's unusable due to the error described here

[alexberry]

any news here? we want to start using pyrra mainly with bool gauges, and currently it's unusable due to the error described here

Hey @deanb-everc, I can give you some guidance to build your own image if you'd like, this is how we're working around the issue in our production envs. There is likely a much better way to build an image locally, but this is what worked for me.

Clone my fork & checkout the branch with the fix

 git clone https://github.com/alexberry/pyrra.git
 git switch fix-boolean-gauge-metric

Set up Build Environment

Setting up your build env is documented in CONTRIBUTING.md, so please refer to that document, however the highlights are as follows:

Required build tools

Install these in whichever way your local OS supports it, I am on MacOS so I would use brew install make npm go and then would install docker desktop:

• make
• npm
• go
• docker

Required build dependencies

npm install react-scripts
make install

Build

Building the binary

export GOOS=linux GOARCH=amd64
make all

After performing this step, you should have a new file in the root of the repo named pyrra - this is the pyrra executable

Building the container image

mkdir -p dist/pyrra_linux_amd64_v1
cp pyrra dist/pyrra_linux_amd64_v1/.
docker build . -t pyrra:dev --build-arg TARGETOS=linux --build-arg TARGETARCH=amd64

This will have now built a local image pyrra:dev. You will now need to push this to a self-hosted registry that your kubernetes cluster has access to pull from. Let's say your registry's address is my-lovely-registry.somecompany.sometld, you can push this custom image up to your registry with:

docker tag pyrra:dev my-lovely-registry.somecompany.sometld/pyrra:dev
docker push my-lovely-registry.somecompany.sometld/pyrra:dev

Configuring Helm to use your custom image

Finally, if you are using the helm chart by rlex to deploy pyrra to your cluster, you can add the following to your values.yaml to switch to your custom image:

image:
  repository: my-lovely-registry.somecompany.sometld/pyrra
  tag: dev

Enjoy :)

[deanb-everc]

got it @alexberry , thanks!

[alexberry]

@deanb-everc I wanted a better build mechanism than locally-built images, so I created my own (throwaway) PR to generate the container image ghcr.io/alexberry/pyrra:pr-3. You can configure helm to use it with:

image:
  repository: ghcr.io/alexberry/pyrra
  tag: pr-3

[alexberry]

hello @metalmatze, is there any chance this can be reviewed? We've been hoping to revert to the master branch of this repo for over a month now, but are still having to run a custom-built image.

[fbegyn]

Looked this over and have confirmed the behavior on some of our instances. LGTM and hoping this can get merged in.

[metalmatze]

Hello everyone!
Thank you for the great discussion and first of all for sending the PR to begin with.
I was super busy organising PromCon the last 6-8 weeks.

This seems logical to me.
We should just make sure that there's nothing else depending on the 100 *, other than that this looks good to me.

[metalmatze]

Again, thank you so much!
Given this is merged into main. Do we want to cherry pick it into release-0.7 so we can get you a patch release?

[alexberry]

Again, thank you so much! Given this is merged into main. Do we want to cherry pick it into release-0.7 so we can get you a patch release?

Yes please, that would be very helpful :)