diff --git a/README.md b/README.md index d22a01d0..1792239f 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,3 @@ -Table of Contents -================= - -* [IDP Builder](#idp-builder) - * [About](#about) - * [Quickstart](#quickstart) - * [Running the idpbuilder](#running-the-idpbuilder) - * [Use](#use) - * [Architecture](#architecture) -* [Extending the IDP builder](#extending-the-idpbuilder) -* [Developer notes](#developer-notes) - # IDP Builder Internal development platform binary launcher. @@ -27,6 +15,8 @@ This can be useful in several ways: ## Quickstart +If you are interested in running idpbuilder in Codespaces through your browser, check out the [Codespaces](#running-in-codespaces) section. + ### Download and install the idpbuilder Download the latest release with the commands: @@ -44,14 +34,16 @@ tar xzf idpbuilder.tar.gz Alternatively, you can download the latest binary from [the latest release page](https://github.com/cnoe-io/idpbuilder/releases/latest). -### Using the idpbuilder +## Using the idpbuilder + +### Basic usage + +The most basic command which creates a Kubernetes Cluster (Kind cluster) with the core packages installed. ```bash ./idpbuilder create ``` -This is the most basic command which creates a Kubernetes Cluster (Kind cluster) with the core packages installed. -
What are the core packages? @@ -59,7 +51,16 @@ This is the most basic command which creates a Kubernetes Cluster (Kind cluster) * **Gitea** server is the in-cluster Git server that ArgoCD can be configured to sync resources from. You can sync from local file systems to this. * **Ingress-nginx** is used as a method to access in-cluster resources such as ArgoCD UI and Gitea UI. - See the [Architecture](#Architecture) section for more information on further information on how core packages are installed and configured. + #### Core package versions + + | Name | Version | + | -------- | ------- | + | Argo CD | v2.10.7 | + | Gitea | v9.5.1 | + | Nginx | v1.8.1 | + + The default manifests for the core packages are available [here](pkg/controllers/localbuild/resources). + See the [contribution doc](./CONTRIBUTING.md) for more information on how core packages are installed and configured.
@@ -69,19 +70,36 @@ Once idpbuilder finishes provisioning cluster and packages, you can access GUIs * ArgoCD: https://argocd.cnoe.localtest.me:8443/ * Gitea: https://gitea.cnoe.localtest.me:8443/ -You can obtain credentials for these packages by running the following commands: +You can obtain credentials for them by running the following command: ```bash -# argoCD password. username is admin. -kubectl -n argocd get secret argocd-initial-admin-secret \ - -o go-template='{{ range $key, $value := .data }}{{ printf "%s: %s\n" $key ($value | base64decode) }}{{ end }}' - -# gitea admin credentials -kubectl get secrets -n gitea gitea-admin-secret \ - -o go-template='{{ range $key, $value := .data }}{{ printf "%s: %s\n" $key ($value | base64decode) }}{{ end }}' +./idpbuilder get secrets ``` -#### Example commands +
+ The "get secrets" command + + The `get secrets` command retrieves the following: + - ArgoCD initial admin password. + - Gitea admin user credentials. + - Any secrets labeled with `cnoe.io/cli-secret=true`. + + You can think of the command as executing the following kubectl commands: + + ```bash + kubectl -n argocd get secret argocd-initial-admin-secret + kubectl get secrets -n gitea gitea-admin-secret + kubectl get secrets -A -l cnoe.io/cli-secret=true + ``` + In addition, secrets labeled with `cnoe.io/package-name` can be specified with the `-p` flag. For example, for Gitea: + + ```bash + ./idpbuilder get secrets -p gitea + ``` + +
+ +### Example commands **For more advanced use cases, check out the [examples](./examples) directory.** @@ -91,29 +109,31 @@ You can specify the kubernetes version by using the `--kube-version` flag. Suppo ./idpbuilder create --kube-version v1.27.3 ``` -If you need to expose more ports between the docker container and the kubernetes host, you can use the `--extra-ports` flag. For example: +If you want to specify your own kind configuration file, use the `--kind-config` flag. ``` -./idpbuilder create --extra-ports 22:32222 +./idpbuilder create --build-name local --kind-config ./my-kind.yaml` ``` -If you want to specify your own kind configuration file, use the `--kind-config` flag. +If you want to specify ArgoCD configmap. ``` -./idpbuilder create --build-name local --kind-config ./my-kind.yaml` +./idpbuilder create --package-custom-file=argocd:pkg/k8s/test-resources/input/argocd-cm.yaml ``` -You can also specify the name of build. This name is used as part of the cluster, namespace, and git repositories. +Run the following commands for available flags and subcommands: ``` -./idpbuilder create --build-name localdev +./idpbuilder --help +./idpbuilder create --help ``` ### Custom Packages Idpbuilder supports specifying custom packages using the flag `--package-dir` flag. This flag expects a directory containing ArgoCD application files. -Let's take a look at [this example](examples/basic). This example defines two custom package directories to deploy to the cluster. +Examples of using custom packages are available in the [example](./examples) directory. +Let's take a look at [this example](examples/basic). This defines two custom package directories to deploy to the cluster. To deploy these packages, run the following commands from this repository's root. @@ -167,105 +187,34 @@ You can also view the updated Application spec by going to this address: https:/ The second package directory defines two normal ArgoCD applications referencing a remote repository. They are applied as-is. -## Architecture - -idpbuilder is made of two phases: CLI and Kubernetes controllers. - -![idpbuilder.png](docs/images/idpbuilder.png) - -### CLI - -When the idpbuilder binary is executed, it starts with the CLI phase. - -1. This is the phase where command flags are parsed and translated into relevant Go structs' fields. Most notably the [`LocalBuild`](https://github.com/cnoe-io/idpbuilder/blob/main/api/v1alpha1/localbuild_types.go) struct. -2. Create a Kind cluster, then update the kubeconfig file. -3. Once the kind cluster is started and relevant fields are populated, Kubernetes controllers are started: - * `LocalbuildReconciler` responsible for bootstrapping the cluster with absolute necessary packages. Creates Custom Resources (CRs) and installs embedded manifests. - * `RepositoryReconciler` responsible for creating and managing Gitea repository and repository contents. - * `CustomPackageReconciler` responsible for managing custom packages. -4. They are all managed by a single Kubernetes controller manager. -5. Once controllers are started, CRs corresponding to these controllers are created. For example for Backstage, it creates a GitRepository CR and ArgoCD Application. -6. CLI then waits for these CRs to be ready. - -### Controllers - -During this phase, controllers act on CRs created by the CLI phase. Resources such as Gitea repositories and ArgoCD applications are created. - -#### LocalbuildReconciler - -`LocalbuildReconciler` bootstraps the cluster using embedded manifests. Embedded manifests are yaml files that are baked into the binary at compile time. -1. Install core packages. They are essential services that are needed for the user experiences we want to enable: - * Gitea. This is the in-cluster Git server that hosts Git repositories. - * Ingress-nginx. This is necessary to expose services inside the cluster to the users. - * ArgoCD. This is used as the packaging mechanism. Its primary purpose is to deploy manifests from gitea repositories. -2. Once they are installed, it creates `GitRepository` CRs for core packages. This CR represents the git repository on the Gitea server. -3. Create ArgoCD applications for the apps. Point them to the Gitea repositories. From here on, ArgoCD manages the core packages. - -Once core packages are installed, it creates the other embedded applications: Backstage and Crossplane. -1. Create `GitRepository` CRs for the apps. -2. Create ArgoCD applications for the apps. Point them to the Gitea repositories. -#### RepositoryReconciler - -`RepositoryReconciler` creates Gitea repositories. -The content of the repositories can either be sourced from Embedded file system or local file system. - -#### CustomPackageReconciler - -`CustomPackageReconciler` parses the specified ArgoCD application files. If they specify repository URL with the scheme `cnoe://`, -it creates `GitRepository` CR with source specified as local, then creates ArgoCD application with the repository URL replaced. - -For example, if an ArgoCD application is specified as the following. - -```yaml -apiVersion: argoproj.io/v1alpha1 -kind: Application -spec: - source: - repoURL: cnoe://busybox -``` - -Then, the actual object created is this. - -```yaml -apiVersion: argoproj.io/v1alpha1 -kind: Application -spec: - source: - repoURL: http://my-gitea-http.gitea.svc.cluster.local:3000/giteaAdmin/idpbuilder-localdev-my-app-busybox.git -``` - -## Developer notes - -If you want to contribute and extend the existing project, make sure that you have installed go (>= 1.21) and cloned this project. -Next, you can build it `make` or launch the `main.go` within your IDE or locally `./idpbuilder`. - -You can override the kind configuration generated by the idpbuilder. For that purpose, look to the -console to grab the config and save it in a file: -```text -########################### Our kind config ############################ -# Kind kubernetes release images https://github.com/kubernetes-sigs/kind/releases -kind: Cluster -apiVersion: kind.x-k8s.io/v1alpha4 -... -######################### config end ############################ -``` -Next, import it `./idpbuilder create --kindConfig ` - -### Default manifests installed by idpbuilder - -The default manifests for the core packages are available [here](pkg/controllers/localbuild/resources). -These are generated by scripts. If you want to make changes to them, see below. - -#### ArgoCD - -ArgoCD manifests are generated using a bash script available [here](./hack/argo-cd/generate-manifests.sh). -This script runs kustomize to modify the basic installation manifests provided by ArgoCD. Modifications include: - -1. Prevent notification and dex pods from running. This is done to keep the number of pods running low by default. -2. Use the annotation tracking instead of the default label tracking. Annotation tracking allows you to avoid [problems caused by the label tracking method](https://argo-cd.readthedocs.io/en/stable/user-guide/resource_tracking/). In addition, this configuration is required when using Crossplane. +## Contributing + +If you'd like to contribute to the project or know the architecture and internals of this project, check out the [contribution doc](./CONTRIBUTING.md). + +## Running in Codespaces + +1. Create a Codespaces instance. ![img](examples/ref-implementation/images/codespaces-create.png) +2. Wait for it to be ready. It may take several minutes. +3. Get the latest release of idpbuilder: + ```bash + version=$(curl -Ls -o /dev/null -w %{url_effective} https://github.com/cnoe-io/idpbuilder/releases/latest) + version=${version##*/} + curl -L -o ./idpbuilder.tar.gz "https://github.com/cnoe-io/idpbuilder/releases/download/${version}/idpbuilder-$(uname | awk '{print tolower($0)}')-$(uname -m | sed 's/x86_64/amd64/').tar.gz" + tar xzf idpbuilder.tar.gz + ``` +4. Run idpbuilder: + ``` + idpbuilder create --protocol http \ + --host ${CODESPACE_NAME}-8080.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN} \ + --port 8080 --use-path-routing + ``` +5. Because Codespaces gives a single externally routable host name for an instance, idpbuilder must deploy with path based routing. + This means ArgoCD and Gitea UIs are given with the following commands. + * ArgoCD: `echo https://${CODESPACE_NAME}-8080.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN}/argocd` + * Gitea: `echo https://${CODESPACE_NAME}-8080.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN}/gitea` +6. Note that not all examples work with path based routing. ## Extending the IDP builder -We are actively working to include more patterns and examples of extending idpbuilder to get started easily. - +We are actively working to include more patterns and examples of extending idpbuilder to get started easily. \ No newline at end of file