feat: accept new SLO specs using the filesystem HTTP api

[harmw]

Implement an HTTP endpoint at which new SLO specs may be published when using the filesystem operating mode.

Accept files over HTTP at /specs/create and writes them to disk, creates new rules and reloads prometheus.

Available (bare) endpoints:

/specs/create
/specs/remove
/specs/list

Sadly, the fsnotify watcher never managed to see any of the newly uploaded files so I had to force the rules creation a bit 🤔

It seems to work, likely a little rough on the edges includes tests now

[harmw]

@metalmatze would appreciate your thoughts on this, thank you.

[metalmatze]

Thank you for the contribution, and I am sorry for the long silence.

I'm still wondering if we should have this as part of Pyrra directly or if this will be a separate component people can install. I would only add this to Pyrra by disabling this feature by default. I fear that opening this up by default isn't a sane default without any authn/authz.

[harmw]

Thank you for the contribution, and I am sorry for the long silence.

I'm still wondering if we should have this as part of Pyrra directly or if this will be a separate component people can install. I would only add this to Pyrra by disabling this feature by default. I fear that opening this up by default isn't a sane default without any authn/authz.

Thanks! Sounds pretty reasonable though, suppose putting it behind a runtime switch shouldn't be too hard 👍

[aaranmcguire]

@harmw wondering if you had any time to action this.

[harmw]

@harmw wondering if you had any time to action this.

runtime switch is added 😅

[metalmatze]

Excellent changes in the last commits!
I have some last concerns about the API. Especially the uploading files API.

There are some golangci-lint errors. They will disappear if you rebase to the latest main.

Thank you for your contribution thus far!

[metalmatze]

This is pretty pragmatic and should be ok for a proof-of-concept.
Quickly, this will become a foot gun where the API will accept any file and write the contents to disk.

We want to ensure the sent data is correct and has a JSON/YAML objective configuration.

A helper function to read from a file and then unmarshal and validate the content already exists with:

pyrra/filesystem.go

Lines 387 to 404 in 174b026

	func objectiveFromFile(file string) (v1alpha1.ServiceLevelObjective, slo.Objective, error) {
	bytes, err := os.ReadFile(file)
	if err != nil {
	return v1alpha1.ServiceLevelObjective{}, slo.Objective{}, fmt.Errorf("failed to read file %q: %w", file, err)
	}
	var config v1alpha1.ServiceLevelObjective
	if err := yaml.UnmarshalStrict(bytes, &config); err != nil {
	return v1alpha1.ServiceLevelObjective{}, slo.Objective{}, fmt.Errorf("failed to unmarshal objective %q: %w", file, err)
	}
	objective, err := config.Internal()
	if err != nil {
	return v1alpha1.ServiceLevelObjective{}, slo.Objective{}, fmt.Errorf("failed to get objective %q: %w", file, err)
	}
	return config, objective, nil
	}

If we change that function to take a io.Reader we can pass both a os.File (for the existing use case) or a bytes.Buffer which we would write the HTTP payload content into to that helper function.

-func objectiveFromFile(file string) (v1alpha1.ServiceLevelObjective, slo.Objective, error) {
+func objectiveFromReader(content io.Reader) (v1alpha1.ServiceLevelObjective, slo.Objective, error) { 

It's probably easiest to then marshal the parsed config struct back into YAML and write that to disk.

Let me know if you need any more pointers or anything else. I'm happy to help!

[harmw]

@metalmatze quick update: just need (time) to resolve the file handling part 😅