Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Notes from working through the tutorial #441

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/tutorials/issue-tracker/04-boilerplate.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

Since this is a fresh project, we can take advantage of the CLI's tooling to set up boilerplate code for us which we can then extend on. Note that this is not strictly necessary for writing an application (whereas running the CUE codegen is something you'll likely want for every project), but it makes initial project bootstrapping simpler, and will help us move along here faster. If you decide in future projects you want to handle your routing, storage, or front-end framework differently, you can eschew some or all of the things laid out in this section.

## The `project add` Command
## The `project component add` Command

Earlier, we used the CLI's `project` command with `project init`, initializing our project with some very basic stuff. Now, we can again use the `project` command, this time to add boilerplate components to our app. These are added using the `project add` command, with the name of one or more components you wish to add to the project. To see the list of possible components, you can run it sans arguments, like so:
Earlier, we used the CLI's `project` command with `project init`, initializing our project with some very basic stuff. Now, we can again use the `project` command, this time to add boilerplate components to our app. These are added using the `project component add` command, with the name of one or more components you wish to add to the project. To see the list of possible components, you can run it sans arguments, like so:
```shell
$ grafana-app-sdk project add
$ grafana-app-sdk project component add
Usage: grafana-app-sdk project component add [options] <components>
where <components> are one or more of:
backend
Expand Down Expand Up @@ -432,4 +432,4 @@ Here we only have one file, created for our Issue kind. If we had more kinds, we
Next, now that we have minimal functioning code, we can try, [deploying our project locally](05-local-deployment.md)

### Prev: [Generating Kind Code](03-generate-kind-code.md)
### Next: [Local Deployment](05-local-deployment.md)
### Next: [Local Deployment](05-local-deployment.md)
3 changes: 1 addition & 2 deletions docs/tutorials/issue-tracker/05-local-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ Failed to start grafana. error: app provisioning error: plugin not installed: "i
app provisioning error: plugin not installed: "issue-tracker-project-app"
```

So this is just something we didn't do before we started our environment. While we built the plugin, we want to also deploy it to our local cluster. The local cluster can only read from `local/mounted-files`, so we need it there. Simple enough fix, we run
So this is just something we didn't do before we started our environment. While we built the plugin, we want to also deploy it to our local cluster. The local cluster can only read from `local/mounted-files`, so we need it there. Simple enough fix: in another terminal session, we run
```shell
$ make local/deploy_plugin
tilt disable grafana
Expand Down Expand Up @@ -224,7 +224,6 @@ Since our plugin is automatically installed, we can go to [grafana.k3d.localhost
Right now, if I do a curl to our list endpoint, we'll get back a response with an empty list:
```shell
$ curl http://grafana.k3d.localhost:9999/api/plugins/issue-tracker-project-app/resources/v1/issues | jq .
```
{
"kind": "IssueList",
"apiVersion": "issue-tracker-project.ext.grafana.com/v1",
Expand Down
5 changes: 4 additions & 1 deletion docs/tutorials/issue-tracker/06-frontend.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,10 @@ $ make local/deploy_plugin
And just like that, we can refresh or go to [http://grafana.k3d.localhost:9999/a/issue-tracker-project-app], and see our brand-new plugin UI.
If we create a new issue, we can see that it shows up in the list, or via a `kubectl get issues`.

> [!NOTE]
> If you do not see the changes in the plugin UI, try clearing your browser cache.

Comment on lines +104 to +106
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent 20 minutes banging my head against this 😅

Now all that's left is to think a bit about our operator.

### Prev: [Local Deployment](05-local-deployment.md)
### Next: [Adding Operator Code](07-operator-watcher.md)
### Next: [Adding Operator Code](07-operator-watcher.md)
75 changes: 45 additions & 30 deletions docs/tutorials/issue-tracker/07-operator-watcher.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,36 @@ By default, the operator is run as a separate container alongside your grafana d
The operator is a logical pattern which runs one or more controllers. The typical use-case for a controller is the `operator.InformerController`, which holds:
* One or more informers, which subscribe to events for a particular resource kind and namespace
* One or more watchers, which consume events for particular kinds
In our case, the boilerplate code uses the `simple.Operator` operator type, which handles dealing with tying together informers and watchers (or reconcilers) in an `InformerController` for us in `cmd/operator/main.go`:

In our case, the boilerplate code uses the opinionated `simple.App` App type, which handles dealing with tying together informers and watchers (or reconcilers) as Managed Kinds for us in `pkg/app/app.go`:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bit has not been updated since the refactor away from directly using simple.App. It could do with a redraft by someone more familiar IMO

```golang
issueWatcher, err := watchers.NewIssueWatcher()
if err != nil {
logging.DefaultLogger.With("error", err).Error("Unable to create IssueWatcher")
panic(err)
}
err = runner.WatchKind(issue.Kind(), issueWatcher, simple.ListWatchOptions{
Namespace: resource.NamespaceAll,
})
if err != nil {
return nil, fmt.Errorf("unable to create IssueWatcher: %w", err)
}

config := simple.AppConfig{
Name: "issue-tracker-project",
KubeConfig: cfg.KubeConfig,
InformerConfig: simple.AppInformerConfig{
ErrorHandler: func(ctx context.Context, err error) {
logging.FromContext(ctx).With("error", err).Error("Informer processing error")
},
},
ManagedKinds: []simple.AppManagedKind{
{
Kind: issuev1.Kind(),
Watcher: issueWatcher,
},
},
}

a, err := simple.NewApp(config)
if err != nil {
return nil, err
}
```
This code creates a watcher for receiving events, and then has the `simple.Operator` attach it to its own internal controller, and create an informer for it that uses the filtering provided in `simple.ListWatchOptions`. As a nice addition to this, `simpple.Operator.WatchKind` also wraps our `IssueWatcher` in an `operator.OpinionatedWatcher`, which handles making sure the operator is always in-the-know on events via finalizers (so, for example, a `delete` event is blocked from completing until it gets processed by our operator).
This code creates a watcher for receiving events, and then has the `simple.App` watch for changes on the Kind by configuring it as a Managed Kind. As a nice addition to this, the `simple.App` instance also wraps our `IssueWatcher` in an `operator.OpinionatedWatcher`, which handles making sure the operator is always in-the-know on events via finalizers (so, for example, a `delete` event is blocked from completing until it gets processed by our operator).

The other thing we're doing here is calling `watchers.NewIssueWatcher()`. The `watchers` package was added by our `project add operator` command, so let's take a look at what's there:
```go
Expand All @@ -41,22 +59,24 @@ func (s *IssueWatcher) Add(ctx context.Context, rObj resource.Object) error {
}

// TODO
fmt.Println("Added ", object.StaticMetadata().Identifier())
logging.FromContext(ctx).Debug("Added resource", "name", object.GetStaticMetadata().Identifier().Name)
return nil
}
```
Each method does a check to see if the provided `resource.Object` is of type `*issue.Issue` (it always should be, provided we gave the informer a client with the correct `resource.Schema`). We then just print a line declaring what resource was added, which we saw when [testing our local deployment](05-local-deployment.md).

So what else can we do in our watcher?

Well, right now, we could integrate with some third-party service, maybe you want to sync the issues created in you plugin with GitHub, or some internal issue-tracking tool. You may have some other task which should be performed when an issue is added, or updated, or deleted, which you should do in the operator. As more of grafana begins to use a kubernetes-like storage system, you could even create a resource of another kind in response to an add event, which some other operator would pick up and do something with. Why not do these things in the plugin backend?
Well, right now, we could integrate with some third-party service, maybe you want to sync the issues created in your plugin with GitHub, or some internal issue-tracking tool. You may have some other task which should be performed when an issue is added, or updated, or deleted, which you should do in the operator. As more of grafana begins to use a kubernetes-like storage system, you could even create a resource of another kind in response to an add event, which some other operator would pick up and do something with. Why not do these things in the plugin backend?

Well, as we saw before, your plugin API isn't the only way to interact with Issues. You can create, update, or delete them via `kubectl`. But even if you restrict `kubectl` access, but perhaps another plugin operator may want to create an Issue in response to one of _their_ events. If they did that via directly interfacing with the storage layer, you wouldn't notice that it happened. The operator ensures that no matter _how_ the mutation in the storage layer occurred (API, kubectl, other access), you are informed and can take action.

## Adding Behavior to Our Watcher

Let's add some simple behavior to our issue watcher to export metrics on issue counts by status. To do this, we'll want to export an additional metric from our operator, which we'll have to track in our watcher. Let's update our watcher code to add a `prometheus.GaugeVec` that we can use to track issue counts by status as a gauge (a gauge represents numbers which can increase or decrease, as opposed to a counter, which will only increase):
```go
// pkg/watchers/watcher_issue.go

import (
// ...Existing imports omitted for brevity
"github.com/prometheus/client_golang/prometheus"
Expand Down Expand Up @@ -172,23 +192,18 @@ func (s *IssueWatcher) PrometheusCollectors() []prometheus.Collector {
return []prometheus.Collector{s.statsGauge}
}
```
Easy! Now all we need to do is register it with our operator so it can expose it. The `simple.Operator` provides a method we can call in our `main` function, which we can add right after the part that creates and adds the watcher with `runner.WatchKind` (`runner` is the variable name for our `simple.Operator`):
Easy! Now we need to do is register the Collector.
```go
// Wrap our resource watchers in OpinionatedWatchers, then add them to the controller
issueWatcher, err := watchers.NewIssueWatcher()
if err != nil {
logging.DefaultLogger.With("error", err).Error("Unable to create IssueWatcher")
panic(err)
}
err = runner.WatchKind(issue.Kind(), issueWatcher, simple.ListWatchOptions{
Namespace: resource.NamespaceAll,
})
if err != nil {
logging.DefaultLogger.With("error", err).Error("Error adding Issue watcher to controller")
panic(err)
}
// Register prometheus collectors from the watcher
runner.RegisterMetricsCollectors(issueWatcher.PrometheusCollectors()...)
// pkg/app/app.go

func New(cfg app.Config) (app.App, error) {
issueWatcher, err := watchers.NewIssueWatcher()
if err != nil {
return nil, fmt.Errorf("unable to create IssueWatcher: %w", err)
}

prometheus.MustRegister(issueWatcher.PrometheusCollectors()...)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bodge that worked for me, i suspect there's a better way to register this collector, but I couldn't see anything useful in simple.App

...
```
There we go! Now we'll be exposing the metric through the operator's `/metrics` prometheus scrape endpoint. Our local deployment already scrapes our operator, so if we re-build and re-deploy our operator, we'll start picking up that new metric in our local grafana.
```shell
Expand Down Expand Up @@ -275,13 +290,13 @@ if _, err := s.issueStore.UpdateSubresource(ctx, object.GetStaticMetadata().Iden
```
Now that our watcher methods are performing an action that can result in an error, we want to return said error if it occurs, so that the action can be retried. We want to do this _before_ our gauge code, because if we return an error and retry, we don't want to increase the gauge again. We also want this timestamp to reflect the last time we updated the gauge.

We can't re-build our operator yet, because now our `NewIssueWatcher` function requires an argument: a `resource.ClientGenerator`. A `ClientGenerator` is an object which can return `resource.Client` implementations for a provided kind. We can generate one from scratch for `k8s.Client` with `k8s.NewClientRegistry`, but `simple.NewOperator` actually already does this for us, and we can access the `ClientGenerator` it created with the `ClientGenerator()` method. So our line in `main.go` just needs to go from this:
We can't re-build our operator yet, because now our `NewIssueWatcher` function requires an argument: a `resource.ClientGenerator`. A `ClientGenerator` is an object which can return `resource.Client` implementations for a provided kind. We can generate one from scratch for `k8s.Client` with `k8s.NewClientRegistry`, and we can get the required `rest.Config` from the provided `app.Config`. So our line in `main.go` needs to go from this:
```go
issueWatcher, err := watchers.NewIssueWatcher()
```
to this:
```go
issueWatcher, err := watchers.NewIssueWatcher(runner.ClientGenerator())
issueWatcher, err := watchers.NewIssueWatcher(k8s.NewClientRegistry(cfg.KubeConfig, k8s.ClientConfig{}))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another bodge from me to highlight a change required. This worked, but I imagine its not the way it "should" work after the refactor.

```
Now, because we altered the schema of our kind, we'll need to re-deploy our local environment after re-building and pushing our operator:
```shell
Expand Down Expand Up @@ -323,4 +338,4 @@ Tracking data in the `status` subresource is an operator best practice. For othe
Now that we have a watcher that does something, let's look at adding some other capabilities to our operator: [API admission control](08-adding-admission-control.md).

### Prev: [Writing Our Front-End](06-frontend.md)
### Next: [Adding Admission Control](08-adding-admission-control.md)
### Next: [Adding Admission Control](08-adding-admission-control.md)
4 changes: 2 additions & 2 deletions docs/tutorials/issue-tracker/08-adding-admission-control.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ type KindValidator interface {
```
Basically, it consumes an admission request and produces a validation error if validation fails, or returns nil on success.
The `simple` package has a ready-to-go implementation for this: `simple.Validator`, which calls `ValidateFunc` when `Validate` is called.
That's what we're using here, but you can always define your own type to implement `KindValidator`, too.
We use the `simple.Validator` struct here to validate, but you can use any type that implements `simple.KindValidator`.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't get these changes to apply for me, even after a teardown & clean. But its near the end of the day, I can try again tomorrow!


Just like with `runner.WatchKind`, `runner.ValidateKind` takes the kind, and then the object to apply to it. In this case, to validate a kind, we need a `ValidatingAdmissionController`. This is a simple one-method interface, which we could define a type for ourselves, but we can also use `resource.SimpleValidatingAdmissionController` as a default implementation. In `SimpleValidatingAdmissionController`, `ValidateFunc` is called by the `Validate` function, so we just need to define our validation function in `ValidateFunc`.

Expand Down Expand Up @@ -148,4 +148,4 @@ The neat part about this validation and mutation is that it occurs irrespective
For more details on webhooks and admission control, see [Admission Control](../../admission-control.md).

### Prev: [Writing Operator Code](07-operator-watcher.md)
### Next: [Wrap-Up and Further Reading](09-wrap-up.md)
### Next: [Wrap-Up and Further Reading](09-wrap-up.md)
Loading