diff --git a/docs/autotest.md b/docs/autotest.md index fcf5efdd6..dcb0f3fe4 100644 --- a/docs/autotest.md +++ b/docs/autotest.md @@ -144,3 +144,70 @@ It is also possible to run just the sanity tests until one of them fails: $ REPO_ROOT=`pwd` ginkgo '-focus=sanity' -failFast ./test/e2e/ ... ``` + +## Testing on an existing cluster + +This can be done by emulating what `make start` does when setting up a +QEMU-based cluster. Here is a (not necessarily complete) list: +- Prepare a directory outside of `_work` with the following files. +- Create `ssh.<0 to number of nodes -1>` scripts such that each script + logs into the corresponding node or executes commands. For example: +``` console +$ cat igk/ssh.0 +#!/bin/sh + +ssh igk-1 "$@" +``` +- Create `ssh.` symlinks to the corresponding `ssh.` file. +- Create a `kubernetes.version` file with `.` versions, for example 1.19. +- Create `kube.config` and ensure that a local kubectl command can use it + to connect to the cluster. SSH portforwarding may be necessary for remote + clusters. The config must grant admin permissions. +- Ensure that `ssh. ` works for the following commands: + - `kubectl get nodes` (only needed for `ssh.0`) + - `sudo pvs` + - `sudo ndctl list -NR` +- Label all worker nodes with PMEM with `storage=pmem` and + `feature.node.kubernetes.io/memory-nv.dax=true`. +- Delete all namespaces. +- On OpenShift 4.6 and 4.7: create the `scheduler-policy` config map with a fixed + host port (default: `TEST_SCHEDULER_EXTENDER_NODE_PORT=32000`) and reconfigure + `scheduler/cluster` as explained + in [OpenShift scheduler configuration](install.md#openshift-scheduler-configuration). + There is one crucial difference for the config map: in the `managedResources` array, + it must also include an entry for `second.pmem-csi.intel.com/scheduler` (use by + operator-lvm-production). The corresponding service will be created when deploying + PMEM-CSI as part of the tests. +- Symlink `_work/` to the directory. +- Push a pmem-csi-driver image to a registry that the cluster can pull from. + The normal `make push-images` and `make test_e2e` work when overriding the test config + with a `test/test-config.d/xxx_external-cluster.sh` file that has the following content. + This is just an example, quay.io also works. +``` +TEST_LOCAL_REGISTRY=docker.io/pohly +TEST_PMEM_REGISTRY=docker.io/pohly +TEST_BUILD_PMEM_REGISTRY=docker.io/pohly +TEST_LOCAL_REGISTRY_SKIP_TLS=false +``` +- Run `CLUSTER= TEST_HAVE_OLM=false make test_e2e TEST_E2E_SKIP="raw.conversion"`. On OpenShift, + use `TEST_HAVE_OLM=true`. The only direct TCP connection is the one for the API server, + so with port forwarding a single developer machine can test multiple different remote + clusters. + +## Using ndctl on an OS which does not provide it + +If `ndctl` is not available for the OS but containers can be run, then +the following workaround is possible. Create a wrapper script: +``` ShellSession +sudo tee /usr/local/bin/ndctl <