Merge "Documentation: use Material for MkDocs"

.github/workflows/jekyll-gh-pages.yml → .github/workflows/mkdocs-gh-pages.yml

@@ -1,4 +1,4 @@
-name: Build static HTML documentation with Jekyll and GitHub pages
+name: Build static HTML documentation with Material for MkDocs and GitHub pages
 
 on:
   push:
@@ -14,25 +14,22 @@ permissions:
 
 concurrency:
   group: "pages"
-  cancel-in-progress: false
+  cancel-in-progress: true
 
 jobs:
   build:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
         uses: actions/checkout@v3
-      - name: Move README and CONTRIBUTING into the documentation
-        run: |
-          cp "${GITHUB_WORKSPACE}/README.md" "${GITHUB_WORKSPACE}/doc/README.md"
-          cp "${GITHUB_WORKSPACE}/CONTRIBUTING.md" "${GITHUB_WORKSPACE}/doc/developer/CONTRIBUTING.md"
       - name: Setup Pages
         uses: actions/configure-pages@v3
-      - name: Build with Jekyll
-        uses: actions/jekyll-build-pages@v1
+      - name: Setup python and install material for mkdocs
+        uses: actions/setup-python@v4
         with:
-          source: ./doc/
-          destination: ./_site
+          python-version: 3.12
+      - run: pip install -r mkdocs-requirements.txt
+      - run: mkdocs build --site-dir ./_site
       - name: Upload artifact
         uses: actions/upload-pages-artifact@v2
 

Makefile

@@ -73,6 +73,10 @@ vet: ## Run go vet against code.
 sc: staticcheck ## Run staticcheck checks https://staticcheck.dev/docs/
 	$(LOCALBIN)/staticcheck ./...
 
+.PHONY: doc-serve
+doc-serve: mkdocs
+	$(LOCALBIN)/mkdocs/bin/mkdocs serve
+
 .PHONY: test
 test: manifests generate fmt vet envtest ## Run tests.
 	KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test ./... -coverprofile cover.out
@@ -161,12 +165,19 @@ KUSTOMIZE ?= $(LOCALBIN)/kustomize
 CONTROLLER_GEN ?= $(LOCALBIN)/controller-gen
 ENVTEST ?= $(LOCALBIN)/setup-envtest
 OPERATOR_SDK ?= $(LOCALBIN)/operator-sdk
+MKDOCS ?= $(LOCALBIN)/mkdocs/bin/mkdocs
 
 ## Tool Versions
 KUSTOMIZE_VERSION ?= v5.3.0
 CONTROLLER_TOOLS_VERSION ?= v0.13.0
 OPERATOR_SDK_VERSION ?= 1.32.0
 STATICCHECK_VERSION ?= 2023.1.6
+MKDOCS_VERSION ?= 1.5.3
+
+.PHONY: mkdocs
+mkdocs: $(MKDOCS) ## Install material for mkdocs locally if necessary
+$(MKDOCS): $(LOCALBIN)
+	( test -f $(LOCALBIN)/mkdocs/bin/mkdocs && [[ "$(shell $(LOCALBIN)/mkdocs/bin/mkdocs -V)" =~ "$(MKDOCS_VERSION)" ]] ) || ( python -m venv $(LOCALBIN)/mkdocs && $(LOCALBIN)/mkdocs/bin/pip install --upgrade -r mkdocs-requirements.txt )
 
 KUSTOMIZE_INSTALL_SCRIPT ?= "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/kustomize/${KUSTOMIZE_VERSION}/hack/install_kustomize.sh"
 .PHONY: kustomize

README.md

@@ -78,7 +78,7 @@ The current project status is: **Alpha - NOT PRODUCTION READY**
 Should you have any questions or feedback concerning the SF-Operator, you can:
 
 * [Join our Matrix channel](https://matrix.to/#/#softwarefactory-project:matrix.org)
-* Send an email to [[email protected]]([email protected])
+* Send an email to [[email protected]](mailto:[email protected])
 * [File an issue](https://github.com/softwarefactory-project/sf-operator/issues/new) for bugs and feature suggestions
 
 ## Contributing

doc/README.md

@@ -0,0 +1,90 @@
+# SF-Operator: a Zuul-based CI infrastructure for OpenShift
+
+<a href="https://zuul-ci.org" ><img src ="https://zuul-ci.org/gated.svg" /></a>
+<a href="https://softwarefactory-project.io/zuul/t/local/buildsets?project=software-factory%2Fsf-operator&pipeline=post&skip=0" ><img src="https://softwarefactory-project.io/zuul/api/tenant/local/badge?project=software-factory/sf-operator&pipeline=post" /></a>
+<a href="https://matrix.to/#/#softwarefactory-project:matrix.org"><img src="https://img.shields.io/badge/matrix-%23softwarefactory--project-8A2BE2" alt="Matrix Channel" /></a>
+<a href="https://github.com/softwarefactory-project/sf-operator/tags" ><img src="https://img.shields.io/github/v/tag/softwarefactory-project/sf-operator" /></a>
+<img src="https://img.shields.io/badge/project%20status-ALPHA-FF2060" alt="Testing only; use in production at your own risks" />
+
+## About
+
+SF-Operator is an Operator that simplifies the deployment and operation of Software Factory instances on the OpenShift Container Platform. An instance of Software Factory is composed of [Zuul](https://zuul-ci.org) and its dependencies ([NodePool](https://zuul-ci.org/docs/nodepool/latest/), Zookeeper, MariaDB, Log Server).
+
+It is the natural evolution of the [Software Factory project](https://softwarefactory-project.io): the 3.8.x release of Software Factory saw the containerization of every major service, but was still delivered as RPM packages, in the form of a custom CentOS 7 distribution.
+SF-Operator builds upon this containerization effort to move from a distro-centric approach to a cloud-native deployment.
+This is also an opportunity to focus on the leanest service suite needed to provide a working gated CI infrastructure; hence a scope reduced to Zuul and its dependencies only.
+
+SF-Operator is built mostly in Go upon the [Operator Framework](https://operatorframework.io), with the aim of reaching the highest capability level that can be achieved, in order to bring a modern, scalable gated CI alternative to OpenShift users, with the least friction from operation as possible.
+
+Furthermore, SF-Operator takes advantage of some of the specificities of OpenShift as a container orchestration platform:
+
+* Improved routes API
+* Integration with OLM for streamlined operator and operands' lifecycle management
+* If [enabled in OpenShift](https://docs.openshift.com/container-platform/4.13/monitoring/enabling-monitoring-for-user-defined-projects.html#enabling-monitoring-for-user-defined-projects), SF-Operator comes with default monitoring and alerting configurations that can be used out of the box. The default alerting rules are honed from years of maintaining and running several large Zuul deployments at scale for [Fedora](https://fedora.softwarefactory-project.io/zuul/status), [Ansible](https://ansible.softwarefactory-project.io/zuul/status) and [RDO](https://review.rdoproject.org/zuul/status).
+* If [enabled](https://docs.openshift.com/container-platform/4.13/logging/cluster-logging.html), OpenShift provides application logs aggregation with its logging subsystem out of the box.
+
+Finally, we also provide a Command Line Interface (CLI) called sfconfig to simplify common tasks related to the operator, management of the operands, development and testing.
+
+## Status
+
+The current project status is: **Alpha - NOT PRODUCTION READY**
+
+## Capability Levels
+
+* Level 1 - Basic Install - **8/10**
+    - Zuul Scheduler: ✅
+    - Zuul Executor: ✅
+    - Zuul Web: ✅
+    - Zuul Merger: ❌
+    - Nodepool Launcher: ✅
+    - Nodepool Builder: ✅
+    - Zookeeper: ✅
+    - MariaDB: ✅
+    - Log Server: ✅
+    - Internal Config Repository, bootstrapped pipelines and default jobs: ✅
+* Level 2 - Seamless upgrades - **2/2**
+    - Operator: ✅
+    - Operands: ✅
+* Level 3 - Full Lifecycle - **1/5**
+    - SF 3.8.x migration ❌
+    - Backup: ❌
+    - Restore: ❌
+    - Rolling deployments: ❌
+    - Reconfiguration: ✅
+* Level 4 - Deep Insights - **1/3**
+    - Operator metrics: ❌
+    - Operand metrics: ✅
+    - Alerts: ❌ (WIP)
+* Level 5 - Auto pilot - **0/3**
+    - Auto-scaling : ❌
+    - Auto-healing: ❌
+    - Auto-tuning: ❌
+
+## Getting Started
+
+* [Installing the Operator ](https://softwarefactory-project.github.io/sf-operator/operator/getting_started.html)
+* [Deploying Zuul and dependencies with SF-Operator](https://softwarefactory-project.github.io/sf-operator/deployment/getting_started.html)
+
+## Documentation
+
+* [Operator documentation](https://softwarefactory-project.github.io/sf-operator/operator/): for OpenShift cluster administrators, this documentation covers installing SF-Operator and managing the operator's lifecycle.
+* [Deployment documentation](https://softwarefactory-project.github.io/sf-operator/deployment/): this documentation covers the essentials for people or teams who intend to deploy and manage Zuul and its dependencies through the SF-Operator.
+* [Developer documentation](https://softwarefactory-project.github.io/sf-operator/developer/): this documentation describes how to set up a development and testing environment to develop the SF-Operator.
+* [End User documentation](https://softwarefactory-project.github.io/sf-operator/user/): for users of a Software Factory instance. This documentation mostly describes the `Software Factory's config repository` usage (configuration-as-code).
+* [CLI refererence](https://softwarefactory-project.github.io/sf-operator/reference/cli/)
+
+## Getting Help
+
+Should you have any questions or feedback concerning the SF-Operator, you can:
+
+* [Join our Matrix channel](https://matrix.to/#/#softwarefactory-project:matrix.org)
+* Send an email to [[email protected]](mailto:[email protected])
+* [File an issue](https://github.com/softwarefactory-project/sf-operator/issues/new) for bugs and feature suggestions
+
+## Contributing
+
+Refer to [CONTRIBUTING.md](https://github.com/softwarefactory-project/sf-operator/blob/master/CONTRIBUTING.md).
+
+## Licence
+
+Sf-operator is distributed under the [Apache License](https://www.apache.org/licenses/LICENSE-2.0.txt).

doc/deployment/backing_services.md

@@ -2,7 +2,6 @@
 
 Here you will find information about the backing services used by Zuul and Nodepool.
 
-## Table of Contents
 
 1. [Philosophy](#philosophy)
 1. [MariaDB](#mariadb)

doc/deployment/certificates.md

@@ -1,6 +1,5 @@
 # Set up TLS on service routes
 
-## Table of Contents
 
 1. [Using a trusted Certificate Authority](#using-a-trusted-certificate-authority)
 1. [Using Let's Encrypt](#using-lets-encrypt)
@@ -21,11 +20,13 @@ The operator watches a `Secret` named `sf-ssl-cert` in the `SoftwareFactory` Cus
 When this `Secret`'s data hold a Certificate, Key and CA Certificate (following a specific scheme) then
 the sf-operator is able to reconfigure all managed `Route`'s TLS to use the TLS material stored in the secret.
 
-> Make sure that the certificate `CN` matches the `fqdn` setting of the `SoftwareFactory` Custom Resource.
+!!! note
+    Make sure that the certificate `CN` matches the `fqdn` setting of the `SoftwareFactory` Custom Resource.
 
 The [sf-operator](./../reference/cli/index.md) CLI can be used to configure these secrets.
 
-> The [`SF configure TLS`](./../reference/cli/index.md#configure-tls) subcommand will validate the SSL certificate/key before updating the `Secret`.
+!!! note
+    The [`SF configure TLS`](./../reference/cli/index.md#configure-tls) subcommand will validate the SSL certificate/key before updating the `Secret`.
 
 ```sh
 sf-operator SF configure TLS --CA /tmp/ssl/localCA.pem \
@@ -56,7 +57,8 @@ Once `Ready`, all managed `Route` will present the new certificate.
 The SF Operator offers an option to request a certificate from `Let's Encrypt` using the `ACME http01`
 challenge. The deployment `FQDN` must be publicly resolvable.
 
-> This overrides the custom X509 certificates that might have been set following the [steps above](#using-a-trusted-certificate-authority).
+!!! note
+    This overrides the custom X509 certificates that might have been set following the [steps above](#using-a-trusted-certificate-authority).
 
 1. test your deployment with Let's Encrypt's staging server:
 

doc/deployment/config_repository.md

@@ -1,6 +1,5 @@
 # Config Repository
 
-## Table of Contents
 
 1. [Concept](#concept)
 1. [Repository content](#repository-content)
@@ -15,7 +14,8 @@
 
 ## Concept
 
-> If you are already familiar with the config repository in Software Factory, you can skip this section and go straight to [setting up the repository](#setting-up-the-repository).
+!!! note
+    If you are already familiar with the config repository in Software Factory, you can skip this section and go straight to [setting up the repository](#setting-up-the-repository).
 
 In Software Factory, Zuul and Nodepool's configurations are stored in a special git repository called the **config repository**. Zuul is also pre-configured to run the following CI pipelines and jobs against changes on this repository, in a reserved tenant (`internal`):
 
@@ -44,7 +44,8 @@ Any other file or folder will be ignored.
 
 As of the current version of the SF-Operator, Gerrit and GitLab are the only supported hosting options for the config repository.
 
-> You can follow the [developer's documentation to deploy a test Gerrit instance](../developer/howtos/index.md#gerrit) if needed.
+!!! note
+    You can follow the [developer's documentation to deploy a test Gerrit instance](../developer/howtos/index.md#gerrit) if needed.
 
 ### Hosting on Gerrit
 
@@ -76,10 +77,11 @@ ssh -p29418 <gerrit_host> gerrit set-members --add [email protected] "Service Use
 
 ##### Repository ACLs and Labels
 
-> Access controls and labels management with Gerrit is out of the scope of this documentation. Please refer to
-Gerrit's documentation for further details, for example
-[here](https://gerrit-review.googlesource.com/Documentation/access-control.html) for ACLs
-or [here](https://gerrit-review.googlesource.com/Documentation/config-labels.html) for labels.
+!!! note
+    Access controls and labels management with Gerrit is out of the scope of this documentation. Please refer to
+    Gerrit's documentation for further details, for example
+    [here](https://gerrit-review.googlesource.com/Documentation/access-control.html) for ACLs
+    or [here](https://gerrit-review.googlesource.com/Documentation/config-labels.html) for labels.
 
 The repository must be set with specific ACLs to allow Zuul to interact with it:
 
@@ -121,24 +123,27 @@ For further information check the [Gerrit section](https://zuul-ci.org/docs/zuul
 
 #### Configuring the Zuul connection
 
-In order for Zuul to start listening to Gerrit events, add a `gerritconn` property in your deployed **SoftwareFactory**'s Spec:
-
-> The `username` is the name of the [bot account that was set up in the previous section](#gerrit-bot-account).
+In order for Zuul to start listening to Gerrit events, add a `gerritconn` property in your deployed **SoftwareFactory**'s Spec. Edit the spec with:
 
 ```sh
 kubectl edit sf my-sf
+```
+
+```yaml
 [...]
 spec:
   zuul:
     gerritconns:
       - name: gerrit
-        username: zuul
+        username: zuul # (1)
         hostname: <gerrit_ssh_hostname>
         puburl: "<gerrit_url>"
 [...]
 ```
 
-You can check the [CRD's OpenAPI schema](config/crd/bases/sf.softwarefactory-project.io_softwarefactories.yaml) for specification details.
+1. The `username` is the name of the [bot account that was set up in the previous section](#gerrit-bot-account).
+
+You can check the [CRD's OpenAPI schema](./crds.md) for specification details.
 
 At that step you can continue the setting by [configuring the location of the config repository](#set-the-config-repository-location).
 
@@ -156,8 +161,9 @@ is sufficient.
 Please refer to the [upstream's Zuul documentation for more information about the GitLab's driver and how to define
 an API Token and WebHook token](https://zuul-ci.org/docs/zuul/latest/drivers/gitlab.html#gitlab).
 
-> Note that the WebHook URL to configure on the project or on the project's group setting must
-be `https://<fqdn>/zuul/api/connection/<zuul-connection-name>/payload`
+!!! note
+    The WebHook URL to configure on the project or on the project's group setting must
+    be `https://<fqdn>/zuul/api/connection/<zuul-connection-name>/payload`
 
 The `gate` pipeline defined for the `config` repository workflow relies in the `gateit` label for the pipeline
 trigger rule. Thus, a GitLab label named `gateit` must be defined in the `Settings` the `config` repository.

doc/deployment/crds.md

@@ -2,17 +2,25 @@
 
 This document gives details about the Custom Resource Definitions (CRDs) that the SF-Operator installs on a cluster.
 
-## Table of Contents
+The specs are constantly evolving during alpha development, and should be considered
+unstable but the ultimate source of truth for documentation about their properties.
 
-1. [CRDs](#crds)
 
-## CRDs
+1. [SoftwareFactory](#crds)
+1. [LogServer]()
 
-The Custom Resources installed by the SF-Operator are described in the following OpenAPI specs:
+## SoftwareFactory
 
-* [SoftwareFactory](https://github.com/softwarefactory-project/sf-operator/blob/master/config/crd/bases/sf.softwarefactory-project.io_softwarefactories.yaml)
-* [LogServer](https://github.com/softwarefactory-project/sf-operator/blob/master/config/crd/bases/sf.softwarefactory-project.io_logservers.yaml)
+This custom resource describes a Software Factory instance.
 
-The spec is constantly evolving during alpha development, and should be considered
-unstable but the ultimate source of truth for documentation about its properties.
+```yaml
+--8<-- "config/crd/bases/sf.softwarefactory-project.io_softwarefactories.yaml"
+```
 
+## LogServer
+
+This custom resource describes a standalone Log Server (SSH/rsync endpoint + HTTP server).
+
+```yaml
+--8<-- "config/crd/bases/sf.softwarefactory-project.io_logservers.yaml"
+```

doc/deployment/delete.md

@@ -2,4 +2,4 @@
 
 The [sf-operator](./../reference/cli/index.md) cli command provides several levels of deletion.
 
-See the subcommand [`SF wipe`'s documentation](./../reference/cli/index.md#wipe) for more details. 
+See the subcommand [`SF wipe`'s documentation](./../reference/cli/index.md#wipe) for more details. 

doc/deployment/getting_started.md

@@ -1,6 +1,5 @@
 # Getting Started with a deployment
 
-## Table of Contents
 
 1. [Prerequisites](#prerequisites)
 1. [Installation](#installation)
@@ -19,12 +18,14 @@ In order to deploy a Software Factory with the SF-Operator on an OpenShift clust
 
 We recommend using a dedicated namespace to deploy your Software Factory. Furthermore, only one instance of a Software Factory must be deployed per namespace due to name and label collisions.
 
-> Currently, the namespace must allow `privileged` containers to run. Indeed the `zuul-executor` container requires
-extra privileges because of [bubblewrap](https://github.com/containers/bubblewrap).
+!!! note
+    Currently, the namespace must allow `privileged` containers to run. Indeed the `zuul-executor` container requires
+    extra privileges because of [bubblewrap](https://github.com/containers/bubblewrap).
 
 In this example we will create a dedicated namespace called **sf**. Then the next three commands below configure privileged access on this namespace; modify the commands as needed if using a different namespace.
 
-> Note that these commands might need to be run by a user with enough privileges to create and modify namespaces and policies.
+!!! note
+    Note that these commands might need to be run by a user with enough privileges to create and modify namespaces and policies.
 
 ```sh
 kubectl create namespace sf

doc/deployment/index.md

@@ -1,9 +1,12 @@
+---
+title: About
+---
+
 # Deployment documentation
 
 Welcome to the deployment documentation. You can find here what you need to know when it comes to deploying
 and managing a Software Factory Custom Resource through SF-Operator.
 
-## Table of Contents
 
 1. [Getting started](./getting_started.md)
 1. [Setting up the config repository](./config_repository.md)