[App Manifest + Docs] Tutorial Doc Update for New Manifest, Associate…

…d Fixes (#525) Update the tutorial doc based on the changes from #483 and #491. Running through the tutorial again ran into a few bugs that this PR also resolves: * The template kind used `permissions` instead of `extraPermissions` * The default plugin name uses the full group, which contains `.` and is invalid. Fixed to use only the appName portion * The updated kind in `cue.def` still had a reference to `apiResource` in the version map for inheriting validation/mutation, fixed to not use apiResource * Prometheus metrics for watchers cannot be exposed in the `simple.App` design as they can for `simple.Operator`, updated `operator.OpinionatedWatcher` to expose metrics from wrapped watchers, and a method to `simple.App` to allow exposing arbitrary prometheus collectors to the runner. The tutorial front-end has also been updated to use the grafana API server, rather than the plugin back-end. The plugin back-end will be phased out of the tutorial entirely in the near future.
grafana · Dec 5, 2024 · 69d129d · 69d129d
1 parent a9d2b24
commit 69d129d
Show file tree

Hide file tree

Showing 20 changed files with 453 additions and 437 deletions.
diff --git a/cmd/grafana-app-sdk/templates/manifest.cue.tmpl b/cmd/grafana-app-sdk/templates/manifest.cue.tmpl
@@ -7,9 +7,9 @@ manifest: {
 	// kinds is the list of kinds that your app defines and manages. If your app deals with kinds defined/managed
 	// by another app, use permissions.accessKinds to allow your app access
 	kinds: []
-	// permissions contains any additional permissions your app may require to function.
+	// extraPermissions contains any additional permissions your app may require to function.
 	// Your app will always have all permissions for each kind it manages (the items defined in 'kinds').
-	permissions: {
+	extraPermissions: {
 		// If your app needs access to additional kinds supplied by other apps, you can list them here
 		accessKinds: [
 			// Here is an example for your app accessing the playlist kind for reads and watch

diff --git a/codegen/cuekind/def.cue b/codegen/cuekind/def.cue
@@ -139,8 +139,8 @@ Kind: S={
 			// Fields must be from the root of the schema, i.e. 'spec.foo', and have a string type.
 			// Fields cannot include custom metadata (TODO: check if we can use annotations for field selectors)
 			selectableFields: [...string]
-			validation: #AdmissionCapability | *S.apiResource.validation
-			mutation: #AdmissionCapability | *S.apiResource.mutation
+			validation: #AdmissionCapability | *S.validation
+			mutation: #AdmissionCapability | *S.mutation
 			// additionalPrinterColumns is a list of additional columns to be printed in kubectl output
 			additionalPrinterColumns?: [...#AdditionalPrinterColumns]
 		}

diff --git a/codegen/jennies/backendplugin.go b/codegen/jennies/backendplugin.go
@@ -2,6 +2,7 @@ package jennies
 
 import (
 	"bytes"
+	"strings"
 
 	"github.com/grafana/codejen"
 
@@ -36,7 +37,7 @@ func (m *backendPluginMainGenerator) Generate(decls ...codegen.Kind) (*codejen.F
 	for _, decl := range decls {
 		tmd.Resources = append(tmd.Resources, decl.Properties())
 		if decl.Properties().Group != "" {
-			tmd.PluginID = decl.Properties().Group
+			tmd.PluginID = strings.Split(decl.Properties().Group, ".")[0]
 		}
 	}
 

diff --git a/codegen/testing/golden_generated/manifest/go/testkinds/testapp_manifest.go.txt b/codegen/testing/golden_generated/manifest/go/testkinds/testapp_manifest.go.txt
@@ -33,14 +33,27 @@ var appManifestData = app.ManifestData{
 			Conversion: true,
 			Versions: []app.ManifestKindVersion{
 				{
-					Name:   "v1",
+					Name: "v1",
+					Admission: &app.AdmissionCapabilities{
+						Validation: &app.ValidationCapability{
+							Operations: []app.AdmissionOperation{
+								app.AdmissionOperationCreate,
+								app.AdmissionOperationUpdate,
+							},
+						},
+					},
 					Schema: &versionSchemaTestKindv1,
 				},
 
 				{
 					Name: "v2",
 					Admission: &app.AdmissionCapabilities{
-
+						Validation: &app.ValidationCapability{
+							Operations: []app.AdmissionOperation{
+								app.AdmissionOperationCreate,
+								app.AdmissionOperationUpdate,
+							},
+						},
 						Mutation: &app.MutationCapability{
 							Operations: []app.AdmissionOperation{
 								app.AdmissionOperationCreate,

diff --git a/codegen/testing/golden_generated/manifest/test-app-manifest.yaml.txt b/codegen/testing/golden_generated/manifest/test-app-manifest.yaml.txt
@@ -10,6 +10,11 @@ spec:
           scope: Namespaced
           versions:
             - name: v1
+              admission:
+                validation:
+                    operations:
+                        - CREATE
+                        - UPDATE
               schema:
                 spec:
                     properties:
@@ -58,6 +63,10 @@ spec:
                     x-kubernetes-preserve-unknown-fields: true
             - name: v2
               admission:
+                validation:
+                    operations:
+                        - CREATE
+                        - UPDATE
                 mutation:
                     operations:
                         - CREATE

diff --git a/docs/tutorials/issue-tracker/01-project-init.md b/docs/tutorials/issue-tracker/01-project-init.md
@@ -19,21 +19,13 @@ go install github.com/grafana/grafana-app-sdk/cmd/grafana-app-sdk@latest
 If you're unfamiliar with `go install`, it's similar to `go get`, but will compile a binary for the `main` package in what it pulls, and put that in `$GOPATH/bin`. If you don't have `$GOPATH/bin` in your path, you will want to add it, otherwise the CLI commands won't work for you. You can check if the CLI was installed successfully with:
 You can then check if the install was successful by running.
 
-> [!NOTE]  
-> There is currently a [known issue with running go install](https://github.com/grafana/grafana-app-sdk/issues/189) for many versions. 
-> You can install locally with
-> ```shell
-> git clone [email protected]:grafana/grafana-app-sdk.git && cd grafana-app-sdk/cmd/grafana-app-sdk && go install 
-> ```
-> But be advised this will install the latest `main` commit. To install a specific version, use `git checkout <version>` before running `go install`.
->
-> Alternatively, you can install by running `make build` in the repository root, and copying `target/grafana-app-sdk` into your `PATH`.
-
-If you're not comfortable using `go install`, the [github releases page](https://github.com/grafana/grafana-app-sdk/releases) for the project includes a binary for each architecture per release. You can download the binary and add it to your `PATH` to use the SDK CLI the same way as if you used `go install`.
 ```shell
 grafana-app-sdk --help
 ```
 
+> [!NOTE]
+> If you're not comfortable using `go install`, the [github releases page](https://github.com/grafana/grafana-app-sdk/releases) for the project includes a binary for each architecture per release. You can download the binary and add it to your `PATH` to use the SDK CLI the same way as if you used `go install`.
+
 Now that we have the CLI installed, let's initialize our project. In this tutorial, we're going to use `github.com/grafana/issue-tracker-project` as our go module name, but you can use whatever name you like--it won't affect anything except some imports on code that we work on later.
 ```shell
 grafana-app-sdk project init "github.com/grafana/issue-tracker-project"
@@ -44,6 +36,7 @@ $ grafana-app-sdk project init "github.com/grafana/issue-tracker-project"
  * Writing file go.mod
  * Writing file go.sum
  * Writing file kinds/cue.mod/module.cue
+ * Writing file kinds/manifest.cue
  * Writing file Makefile
  * Writing file local/config.yaml
  * Writing file local/scripts/cluster.sh
@@ -56,6 +49,10 @@ $ tree .
 │   └── operator
 ├── go.mod
 ├── go.sum
+├── kinds
+│   ├── cue.mod
+│   │   └── module.cue
+│   └── manifest.cue
 ├── local
 │   ├── Tiltfile
 │   ├── additional
@@ -66,12 +63,9 @@ $ tree .
 │       ├── cluster.sh
 │       └── push_image.sh
 ├── pkg
-├── plugin
-└── kinds
-    └── cue.mod
-        └── module.cue
+└── plugin
 
-12 directories, 8 files
+12 directories, 9 files
 ```
 
 As we can see from the command output, and the `tree` command, the `project init` command created a go module in the current directory, a Makefile, and several other directories. 

diff --git a/docs/tutorials/issue-tracker/02-defining-our-kinds.md b/docs/tutorials/issue-tracker/02-defining-our-kinds.md
@@ -22,8 +22,7 @@ package kinds
 
 issue: {
 	kind: "Issue"
-	group: "issue-tracker-project"
-	apiResource: {}
+	scope: "Namespaced"
 	codegen: {
 		frontend: true
 		backend: true
@@ -52,7 +51,7 @@ Now, we get to the actual definition of our `issue` model:
 ```cue
 issue: {
 	kind: "Issue"
-	apiResource: {}
+	scope: "Namespaced"
 	codegen: {
 		frontend: true
 		backend: true
@@ -63,13 +62,13 @@ issue: {
 ```
 Here we have a collection of metadata relating to our model, and then a `versions` definition, which we'll get to in a moment. Let's break down the other fields first:
 
-| Snippit                               | Meaning                                                                                                                                                                                                                                                                                                                               |
-|---------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| <nobr>`kind: "Issue"`</nobr>           | This is just the human-readable name, which will be used for naming some things in-code.                                                                                                                                                                                                                                              |
-| <nobr>`apiResource: {}`</nobr>         | This field is an indicator that the kind is expressible as an API Server resource (typically, a Custom Resource Definition). From a codegen perspective, that means that generated go code will be compatible with `resource`-package interfaces. There are fields that can be set within `apiResource`, but we're ok with its default values |
- | <nobr>`codegen.frontend: true`</nobr> | This instructs the CLI to generate front-end (TypeScript interfaces) code for our schema. This defaults to `true`, but we're specifying it here for clarity.                                                                                                                                                                          |
-| <nobr>`codegen.backend: true`</nobr>  | This instructs the CLI to generate back-end (go) code for our schema. This defaults to `true`, but we're specifying it here for clarity.                                                                                                                                                                                              |
-| <nobr>`currentVersion: [0,0]`</nobr>  | This is the current version of the Schema to use for codegen (will default to the latest if not present)                                                                                                                                                                                                                              |
+| Snippit                               | Meaning                                                                                                                                                                                                           |
+|---------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| <nobr>`kind: "Issue"`</nobr>          | This is just the human-readable name, which will be used for naming some things in-code.                                                                                                                          |
+| <nobr>`scope: "Namespaced"`</nobr>    | This is an optional field, which designates whether instances of the kind (resources) are created on a per-tenant basis (Namespaced) or globally (Cluster). It defaults to Namespaced if you leave the field out. |
+ | <nobr>`codegen.frontend: true`</nobr> | This instructs the CLI to generate front-end (TypeScript interfaces) code for our schema. This defaults to `true`, but we're specifying it here for clarity.                                                      |
+| <nobr>`codegen.backend: true`</nobr>  | This instructs the CLI to generate back-end (go) code for our schema. This defaults to `true`, but we're specifying it here for clarity.                                                                          |
+| <nobr>`current: "v1"`</nobr>          | This is the current version of the Schema to use for codegen.                                                                                                                                                     |
 
 ok, now back to `versions`. In the CUE kind, `versions` is a map of a version name to an object containing meta-information about the version, and its `schema`. 
 The map is unordered, so there is no implicit or explicit sequential relationship between versions in your Kind definition, but consider naming conventions that portray the evolution of the kind (for example, `v1`, `v1beta`, `v2`, etc. like kubernetes' kind versions). The `current` top-level Kind attribute declares which version in this map is the current, 

diff --git a/docs/tutorials/issue-tracker/03-generate-kind-code.md b/docs/tutorials/issue-tracker/03-generate-kind-code.md
@@ -1,10 +1,50 @@
 # Generating Kind Code
 
-Now that we have our kind and schema defined, we want to generate code from them that we can use. In the future, we'll want to re-generate this code whenever we change anything in our `kinds` directory. The SDK provides a command for this: `grafana-app-sdk generate`, but our project init also gave us a make target which will do the same thing, so you can run either. Here, I'm running the make target:
+Now that we have our kind and schema defined, we want to generate code from them that we can use. 
+To do this, we need to add the kind to our **manifest**. The manifest is information about our app, 
+including the kinds it provides. If we don't add our kind to the manifest, it isn't considered part of our app. 
+
+Our project init created a default manifest for us, found in `./kinds/manifest.cue`:
+```cue
+package kinds
+
+manifest: {
+	// appName is the unique name of your app. It is used to reference the app from other config objects,
+	// and to generate the group used by your app in the app platform API.
+	appName: "issue-tracker-project"
+	// kinds is the list of kinds that your app defines and manages. If your app deals with kinds defined/managed
+	// by another app, use permissions.accessKinds to allow your app access
+	kinds: []
+	// extraPermissions contains any additional permissions your app may require to function.
+	// Your app will always have all permissions for each kind it manages (the items defined in 'kinds').
+	extraPermissions: {
+		// If your app needs access to additional kinds supplied by other apps, you can list them here
+		accessKinds: [
+			// Here is an example for your app accessing the playlist kind for reads and watch
+			// {
+			//	group: "playlist.grafana.app"
+			//	resource: "playlists"
+			//	actions: ["get","list","watch"]
+			// }
+		]
+	}
+}
+```
+
+By default, the `appName` is the final part of our module path. We don't need to worry about `extraPermissions`, as our app doesn't need to work with any other apps' kinds. 
+However, right now, our `kinds` list is empty. To add our kind to the manifest, we just need to add the field name of our kind:
+```cue
+kinds: [issue]
+```
+
+> [!TIP]
+> If you add a new kind to your project with the command `grafana-app-sdk project kind add <KindName>`, it will automatically add it to the manifest as well
+
+In the future, we'll want to re-generate this code whenever we change anything in our `kinds` directory. The SDK provides a command for this: `grafana-app-sdk generate`, but our project init also gave us a make target which will do the same thing, so you can run either. Here, I'm running the make target:
 ```shell
 make generate
 ```
-This command should ouput a list of all the files it writes:
+This command should output a list of all the files it writes:
 ```shell
 $ make generate
  * Writing file pkg/generated/resource/issue/v1/issue_codec_gen.go
@@ -17,7 +57,9 @@ $ make generate
  * Writing file plugin/src/generated/issue/v1/types.metadata.gen.ts
  * Writing file plugin/src/generated/issue/v1/types.spec.gen.ts
  * Writing file plugin/src/generated/issue/v1/types.status.gen.ts
- * Writing file definitions/issue.issue-tracker-project.ext.grafana.com.json
+ * Writing file definitions/issue.issuetrackerproject.ext.grafana.com.json
+ * Writing file definitions/issue-tracker-project-manifest.json
+ * Writing file pkg/generated/issuetrackerproject_manifest.go
 ```
 That's a bunch of files written! Let's tree the directory to understand the structure a bit better.
 ```shell
@@ -27,13 +69,15 @@ $ tree .
 ├── cmd
 │   └── operator
 ├── definitions
-│   └── issue.issue-tracker-project.ext.grafana.com.json
+│   ├── issue-tracker-project-manifest.json
+│   └── issue.issuetrackerproject.ext.grafana.com.json
 ├── go.mod
 ├── go.sum
 ├── kinds
 │   ├── cue.mod
 │   │   └── module.cue
-│   └── issue.cue
+│   ├── issue.cue
+│   └── manifest.cue
 ├── local
 │   ├── Tiltfile
 │   ├── additional
@@ -45,6 +89,7 @@ $ tree .
 │       └── push_image.sh
 ├── pkg
 │   └── generated
+│       ├── issuetrackerproject_manifest.go
 │       └── resource
 │           └── issue
 │               └── v1
@@ -64,15 +109,11 @@ $ tree .
                     ├── types.spec.gen.ts
                     └── types.status.gen.ts
 
-21 directories, 20 files
+21 directories, 23 files
 ```
 
-So we can now see that all our generated go code lives in the `pkg/generated` package. Since our `target` was `"resource"`, the generated code for `issue` is in the `pkg/generated/resource` package. 
-Each `resource`-target kind then lives in a package defined by the name of the kind: in our case, that is `issue`. If we created another kind in our `kinds` directory called "foo", we'd see a `pkg/generated/resource/foo` directory.
-
-If we had a separate kind which didn't have the `apiResource` field, we'd see a `pkg/generated/models` package directory. We'll see that later, in our follow-up, where we extend on the project.
-
-Note that the go types are in versioned packages, as by default the SDK will generate types for each version of the kind.
+All of our generated go code lives in `pkg/generated`, and all the generated TypeScript lives in `plugin/src/generated`. 
+We also have some JSON files in `definitions`
 
 Note that we also have generated TypeScript in our previously-empty `plugin` directory. By convention, the Grafana plugin for your project will live in the `plugin` directory, so here we've got some TypeScript generated in `plugin/src/generated` to use when we start working on the front-end of our plugin.
 
@@ -85,6 +126,7 @@ Let's take a closer look at the list of files:
 ```shell
 $ tree pkg/generated
 pkg/generated
+├── issuetrackerproject_manifest.go
 └── resource
     └── issue
         └── v1
@@ -95,7 +137,7 @@ pkg/generated
             ├── issue_spec_gen.go
             └── issue_status_gen.go
 
-4 directories, 6 files
+4 directories, 7 files
 ```
 
 The exported go types from our kind's `v1` schema definition are `issue_spec_gen.go` and `issue_status_gen.go`. 
@@ -108,6 +150,10 @@ Likewise, `issue_schema_gen.go` defines a `resource.Schema` for this specific ve
 in addition to a `resource.Kind` for the kind. Finally, `issue_codec_gen.go` contains code for a kubernetes-JSON-bytes<->Issue `Object` codec, 
 which is used by the `Kind` for marshaling and unmarshaling our Object when interacting with the API server.
 
+Finally, we have `issuetrackerproject_manifest.go` in the `pkg/generated` directory. This contains an in-code version of our manifest. 
+As we'll see a bit later, the manifest is also used in code to communicate app capabilities, so we have this in-code representation, 
+as well as an API server one.
+
 ## Generated TypeScript Code
 
 ```shell
@@ -135,12 +181,15 @@ Finally, we have the custom resource definition file that describes our `issue`
 Note that this is a CRD of the kind, not just the schema, so the CRD will contain all schema versions in the kind. 
 This can be used to set up kubernetes as our storage layer for our project.
 
+We also have a manifest here, which will be used by the grafana API server in the future to register the app.
+
 ```shell
 $ tree definitions
 definitions
-└── issue.issue-tracker-project.ext.grafana.com.json
+├── issue-tracker-project-manifest.json
+└── issue.issuetrackerproject.ext.grafana.com.json
 
-1 directory, 1 file
+1 directory, 2 files
 ```
 
 So now we have a bunch of generated code, but we still need a project to actually use it in.