Skip to content

Commit

Permalink
Merge pull request #58 from shizunge/readme
Browse files Browse the repository at this point in the history
Add labels to change behavior for particular services
  • Loading branch information
shizunge authored Sep 27, 2024
2 parents 1de5422 + 6d38153 commit 74e7e53
Show file tree
Hide file tree
Showing 6 changed files with 453 additions and 31 deletions.
28 changes: 22 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ You can configure the most behaviors of *Gantry* via environment variables.
| Environment Variable | Default | Description |
|-----------------------|---------|-------------|
| GANTRY_SERVICES_EXCLUDED | | A space separated list of services names that are excluded from updating. |
| GANTRY_SERVICES_EXCLUDED_FILTERS | | A space separated list of [filters](https://docs.docker.com/engine/reference/commandline/service_ls/#filter), e.g. `label=project=project-a`. Exclude services which match the given filters from updating. Note that multiple filters will be logical **ANDED**. |
| GANTRY_SERVICES_EXCLUDED_FILTERS | `label=gantry.services.excluded=true` | A space separated list of [filters](https://docs.docker.com/engine/reference/commandline/service_ls/#filter), e.g. `label=project=project-a`. Exclude services which match the given filters from updating. Note that multiple filters will be logical **ANDED**. The default value allows you to add label `gantry.services.excluded=true` to services to exclude them from updating. |
| GANTRY_SERVICES_FILTERS | | A space separated list of [filters](https://docs.docker.com/engine/reference/commandline/service_ls/#filter) that are accepted by `docker service ls --filter` to select services to update, e.g. `label=project=project-a`. Note that multiple filters will be logical **ANDED**. |
| GANTRY_SERVICES_SELF | | This is optional. When running as a docker service, *Gantry* will try to find the service name of itself automatically, and update itself firstly. The manifest inspection will be always performed on the *Gantry* service to avoid an infinity loop of updating itself. This can be used to ask *Gantry* to update another service firstly. |

Expand All @@ -78,17 +78,17 @@ You can configure the most behaviors of *Gantry* via environment variables.
|-----------------------|---------|-------------|
| GANTRY_MANIFEST_CMD | buildx | Valid values are `buildx`, `manifest`, and `none`.<br>Set which command for manifest inspection. Also see FAQ section [when to set `GANTRY_MANIFEST_CMD`](docs/faq.md#when-to-set-gantry_manifest_cmd).<ul><li>[`docker buildx imagetools inspect`](https://docs.docker.com/engine/reference/commandline/buildx_imagetools_inspect/)</li><li>[`docker manifest inspect`](https://docs.docker.com/engine/reference/commandline/manifest_inspect/)</li></ul>Set to `none` to skip checking the manifest. As a result of skipping, `docker service update` always runs. In case you add `--force` to `GANTRY_UPDATE_OPTIONS`, you also want to disable the inspection. |
| GANTRY_MANIFEST_NUM_WORKERS | 1 | The maximum number of `GANTRY_MANIFEST_CMD` that can run in parallel. |
| GANTRY_MANIFEST_OPTIONS | | [Options](https://docs.docker.com/engine/reference/commandline/buildx_imagetools_inspect/#options) added to the `docker buildx imagetools inspect` or [options](https://docs.docker.com/engine/reference/commandline/manifest_inspect/#options) to `docker manifest inspect`, depending on `GANTRY_MANIFEST_CMD` value. |
| GANTRY_MANIFEST_OPTIONS | | [Options](https://docs.docker.com/engine/reference/commandline/buildx_imagetools_inspect/#options) added to the `docker buildx imagetools inspect` or [options](https://docs.docker.com/engine/reference/commandline/manifest_inspect/#options) to `docker manifest inspect`, depending on `GANTRY_MANIFEST_CMD` value, for all services. Also see [Labels](#labels) about adding options to a particular service. |

### To add options to services update

| Environment Variable | Default | Description |
|-----------------------|---------|-------------|
| GANTRY_ROLLBACK_ON_FAILURE | true | Set to `true` to enable rollback when updating fails. Set to `false` to disable the rollback. |
| GANTRY_ROLLBACK_OPTIONS | | [Options](https://docs.docker.com/engine/reference/commandline/service_update/#options) added to the `docker service update --rollback` command. |
| GANTRY_UPDATE_JOBS | false | Set to `true` to update replicated-job or global-job. Set to `false` to disable updating jobs. |
| GANTRY_ROLLBACK_OPTIONS | | [Options](https://docs.docker.com/engine/reference/commandline/service_update/#options) added to the `docker service update --rollback` command for all services. Also see [Labels](#labels) about adding options to a particular service. |
| GANTRY_UPDATE_JOBS | false | Set to `true` to update replicated-job or global-job. Set to `false` to disable updating jobs. *Gantry* adds additional options to `docker service update` when there is [no running tasks](docs/faq.md#how-to-update-services-with-no-running-tasks). |
| GANTRY_UPDATE_NUM_WORKERS | 1 | The maximum number of updates that can run in parallel. |
| GANTRY_UPDATE_OPTIONS | | [Options](https://docs.docker.com/engine/reference/commandline/service_update/#options) added to the `docker service update` command. |
| GANTRY_UPDATE_OPTIONS | | [Options](https://docs.docker.com/engine/reference/commandline/service_update/#options) added to the `docker service update` command for all services. Also see [Labels](#labels) about adding options to a particular service. |
| GANTRY_UPDATE_TIMEOUT_SECONDS | 300 | Error out if updating of a single service takes longer than the given time. |

### After updating
Expand Down Expand Up @@ -120,12 +120,28 @@ If the images of services are hosted on multiple registries that are required au

You need to tell *Gantry* to use a named config rather than the default one when updating a particular service. The named configurations are set via either `GANTRY_REGISTRY_CONFIG`, `GANTRY_REGISTRY_CONFIG_FILE` or `GANTRY_REGISTRY_CONFIGS_FILE`. This can be done by adding the following label to the service `gantry.auth.config=<config-name>`. *Gantry* creates [Docker configuration files](https://docs.docker.com/engine/reference/commandline/cli/#configuration-files) and adds `--config <config-name>` to the Docker command line for the corresponding services.

> NOTE: When `GANTRY_REGISTRY_CONFIG`, `GANTRY_REGISTRY_CONFIG_FILE` or `GANTRY_REGISTRY_CONFIGS_FILE` is used, *Gantry* automatically adds `--with-registry-auth` to `docker service update` commands. Without `--with-registry-auth`, the service will be updated to an image without digest. See this [comment](https://github.com/shizunge/gantry/issues/53#issuecomment-2348376336).
> NOTE: *Gantry* automatically adds `--with-registry-auth` to the `docker service update` command for a sevice, when it finds the label `gantry.auth.config=<config-name>` on the service. Without `--with-registry-auth`, the service will be updated to an image without digest. See this [comment](https://github.com/shizunge/gantry/issues/53#issuecomment-2348376336).
> NOTE: You can use `GANTRY_REGISTRY_CONFIGS_FILE` together with other authentication environment variables.
> NOTE: *Gantry* uses `GANTRY_REGISTRY_PASSWORD` and `GANTRY_REGISTRY_USER` to obtain Docker Hub rate when `GANTRY_REGISTRY_HOST` is empty or `docker.io`. You can also use their `_FILE` variants. If either password or user is empty, *Gantry* reads the Docker Hub rate for anonymous users.
## Labels

Labels can be added to services to modify the behavior of *Gantry* for particular services. When *Gantry* sees the following labels on a service, it will modify the Docker command line only for that service. The value on the label overrides the global environment variables.

| Labels | Description |
|---------|-------------|
| `gantry.auth.config=<config-name>` | See [Authentication](#authentication). |
| `gantry.services.excluded=true` | Exclude the services from updating if you are using the default [`GANTRY_SERVICES_EXCLUDED_FILTERS`](#to-select-services). |
| `gantry.manifest.cmd=<command>` | Override [`GANTRY_MANIFEST_CMD`](#to-check-if-new-images-are-available) |
| `gantry.manifest.options=<string> ` | Override [`GANTRY_MANIFEST_OPTIONS`](#to-check-if-new-images-are-available) |
| `gantry.rollback.on_failure=<boolean>` | Override [`GANTRY_ROLLBACK_ON_FAILURE`](#to-add-options-to-services-update) |
| `gantry.rollback.options=<string>` | Override [`GANTRY_ROLLBACK_OPTIONS`](#to-add-options-to-services-update) |
| `gantry.update.jobs=<boolean>` | Override [`GANTRY_UPDATE_JOBS`](#to-add-options-to-services-update) |
| `gantry.update.options=<string>` | Override [`GANTRY_UPDATE_OPTIONS`](#to-add-options-to-services-update) |
| `gantry.update.timeout_seconds=<number>` | Override [`GANTRY_UPDATE_TIMEOUT_SECONDS`](#to-add-options-to-services-update) |

## FAQ

[FAQ](docs/faq.md)
Expand Down
95 changes: 72 additions & 23 deletions src/lib-gantry.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,25 +15,66 @@
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#

# read_env returns empty string if ENV_VALUE is set to empty, in which case we want to use the DEFAULT_VALUE.
_read_env_default() {
local ENV_NAME="${1}"
local DEFAULT_VALUE="${2}"
local READ_VALUE=
READ_VALUE=$(read_env "${ENV_NAME}" "${DEFAULT_VALUE}")
local VALUE="${READ_VALUE}"
[ -z "${VALUE}" ] && VALUE="${DEFAULT_VALUE}"
echo "${VALUE}"
}

# Read a number from an environment variable. Log an error when it is not a number.
gantry_read_number() {
local VNAME="${1}"
local ENV_NAME="${1}"
local DEFAULT_VALUE="${2}"
if ! is_number "${DEFAULT_VALUE}"; then
log ERROR "DEFAULT_VALUE must be a number. Got \"${DEFAULT_VALUE}\"."
return 1
fi
local READ_VALUE VALUE
READ_VALUE=$(read_env "${VNAME}" "${DEFAULT_VALUE}")
VALUE="${READ_VALUE}"
[ -z "${VALUE}" ] && VALUE="${DEFAULT_VALUE}"
local VALUE=
VALUE=$(_read_env_default "${ENV_NAME}" "${DEFAULT_VALUE}")
if ! is_number "${VALUE}"; then
log ERROR "${VNAME} must be a number. Got \"${READ_VALUE}\"."
local READ_VALUE=
READ_VALUE=$(read_env "${ENV_NAME}" "${DEFAULT_VALUE}")
log ERROR "${ENV_NAME} must be a number. Got \"${READ_VALUE}\"."
return 1;
fi
echo "${VALUE}"
}

_get_label_from_service() {
local SERVICE_NAME="${1}"
local LABEL="${2}"
local VALUE=
if ! VALUE=$(docker service inspect -f "{{index .Spec.Labels \"${LABEL}\"}}" "${SERVICE_NAME}" 2>&1); then
log ERROR "Failed to obtain the value of label ${LABEL} from service ${SERVICE_NAME}. ${VALUE}"
return 1
fi
echo "${VALUE}"
}

# Read a number from an environment variable. Log an error when it is not a number.
_read_env_or_label() {
local SERVICE_NAME="${1}"
local ENV_NAME="${2}"
local LABEL="${3}"
local DEFAULT_VALUE="${4}"
local LABEL_VALUE=
LABEL_VALUE=$(_get_label_from_service "${SERVICE_NAME}" "${LABEL}")
if [ -n "${LABEL_VALUE}" ]; then
log DEBUG "Use value \"${LABEL_VALUE}\" from label ${LABEL} on the service ${SERVICE_NAME}."
echo "${LABEL_VALUE}"
return 0
fi
local VALUE=
VALUE=$(_read_env_default "${ENV_NAME}" "${DEFAULT_VALUE}")
echo "${VALUE}"
}


_login_registry() {
local USER="${1}"
local PASSWORD="${2}"
Expand Down Expand Up @@ -567,17 +608,15 @@ _get_config_from_service() {
local SERVICE_NAME="${1}"
local AUTH_CONFIG_LABEL="gantry.auth.config"
local AUTH_CONFIG=
if ! AUTH_CONFIG=$(docker service inspect -f "{{index .Spec.Labels \"${AUTH_CONFIG_LABEL}\"}}" "${SERVICE_NAME}" 2>&1); then
log ERROR "Failed to obtain authentication config from service ${SERVICE_NAME}. ${AUTH_CONFIG}"
AUTH_CONFIG=
fi
AUTH_CONFIG=$(_get_label_from_service "${SERVICE_NAME}" "${AUTH_CONFIG_LABEL}")
[ -z "${AUTH_CONFIG}" ] && return 0
echo "--config ${AUTH_CONFIG}"
}

_skip_jobs() {
local UPDATE_JOBS="${GANTRY_UPDATE_JOBS:-"false"}"
local SERVICE_NAME="${1}"
local UPDATE_JOBS=
UPDATE_JOBS=$(_read_env_or_label "${SERVICE_NAME}" "GANTRY_UPDATE_JOBS" "gantry.update.jobs" "false")
if is_true "${UPDATE_JOBS}"; then
return 1
fi
Expand All @@ -590,10 +629,12 @@ _skip_jobs() {
}

_get_image_info() {
local MANIFEST_OPTIONS="${GANTRY_MANIFEST_OPTIONS:-""}"
local MANIFEST_CMD="${1}"
local IMAGE="${2}"
local DOCKER_CONFIG="${3}"
local SERVICE_NAME="${1}"
local MANIFEST_OPTIONS=
MANIFEST_OPTIONS=$(_read_env_or_label "${SERVICE_NAME}" "GANTRY_MANIFEST_OPTIONS" "gantry.manifest.options" "")
local MANIFEST_CMD="${2}"
local IMAGE="${3}"
local DOCKER_CONFIG="${4}"
local MSG=
local RETURN_VALUE=0
if echo "${MANIFEST_CMD}" | grep -q -i "buildx"; then
Expand Down Expand Up @@ -628,8 +669,9 @@ _get_image_info() {
# echo the image if we found a new image.
# return the number of errors.
_inspect_image() {
local MANIFEST_CMD="${GANTRY_MANIFEST_CMD:-"buildx"}"
local SERVICE_NAME="${1}"
local MANIFEST_CMD=
MANIFEST_CMD=$(_read_env_or_label "${SERVICE_NAME}" "GANTRY_MANIFEST_CMD" "gantry.manifest.cmd" "buildx")
local IMAGE_WITH_DIGEST=
if ! IMAGE_WITH_DIGEST=$(_get_service_image "${SERVICE_NAME}" 2>&1); then
log ERROR "Failed to obtain image from service ${SERVICE_NAME}. ${IMAGE_WITH_DIGEST}"
Expand Down Expand Up @@ -668,7 +710,7 @@ _inspect_image() {
DOCKER_CONFIG=$(_get_config_from_service "${SERVICE}")
[ -n "${DOCKER_CONFIG}" ] && log DEBUG "Adding options \"${DOCKER_CONFIG}\" to docker commands for ${SERVICE_NAME}."
local IMAGE_INFO=
if ! IMAGE_INFO=$(_get_image_info "${MANIFEST_CMD}" "${IMAGE}" "${DOCKER_CONFIG}"); then
if ! IMAGE_INFO=$(_get_image_info "${SERVICE_NAME}" "${MANIFEST_CMD}" "${IMAGE}" "${DOCKER_CONFIG}"); then
log DEBUG "Skip updating ${SERVICE_NAME} because there is a failure to obtain the manifest from the registry of image ${IMAGE}."
return 1
fi
Expand Down Expand Up @@ -768,6 +810,7 @@ _get_service_update_additional_options() {
OPTIONS="${OPTIONS} --replicas=0"
fi
fi
# Add `--with-registry-auth` if needed.
local WITH_REGISTRY_AUTH=
WITH_REGISTRY_AUTH="$(_get_with_registry_auth "${DOCKER_CONFIG}")"
[ -n "${WITH_REGISTRY_AUTH}" ] && OPTIONS="${OPTIONS} ${WITH_REGISTRY_AUTH}"
Expand All @@ -778,16 +821,19 @@ _get_service_rollback_additional_options() {
local SERVICE_NAME="${1}"
local DOCKER_CONFIG="${2}"
local OPTIONS=
# Add `--with-registry-auth` if needed.
local WITH_REGISTRY_AUTH=
WITH_REGISTRY_AUTH="$(_get_with_registry_auth "${DOCKER_CONFIG}")"
[ -n "${WITH_REGISTRY_AUTH}" ] && OPTIONS="${OPTIONS} ${WITH_REGISTRY_AUTH}"
echo "${OPTIONS}"
}

_rollback_service() {
local ROLLBACK_ON_FAILURE="${GANTRY_ROLLBACK_ON_FAILURE:-"true"}"
local ROLLBACK_OPTIONS="${GANTRY_ROLLBACK_OPTIONS:-""}"
local SERVICE_NAME="${1}"
local ROLLBACK_ON_FAILURE=
ROLLBACK_ON_FAILURE=$(_read_env_or_label "${SERVICE_NAME}" "GANTRY_ROLLBACK_ON_FAILURE" "gantry.rollback.on_failure" "true")
local ROLLBACK_OPTIONS=
ROLLBACK_OPTIONS=$(_read_env_or_label "${SERVICE_NAME}" "GANTRY_ROLLBACK_OPTIONS" "gantry.rollback.options" "")
local DOCKER_CONFIG="${2}"
if ! is_true "${ROLLBACK_ON_FAILURE}"; then
return 0
Expand All @@ -812,14 +858,17 @@ _rollback_service() {
# return 0 when there is no error or failure.
# return 1 when there are error(s) or failure(s).
_update_single_service() {
local SERVICE_NAME="${1}"
local UPDATE_TIMEOUT_SECONDS=
if ! UPDATE_TIMEOUT_SECONDS=$(gantry_read_number GANTRY_UPDATE_TIMEOUT_SECONDS 300); then
UPDATE_TIMEOUT_SECONDS=$(_read_env_or_label "${SERVICE_NAME}" "GANTRY_UPDATE_TIMEOUT_SECONDS" "gantry.update.timeout_seconds" "300")
if ! is_number "${UPDATE_TIMEOUT_SECONDS}"; then
log ERROR "UPDATE_TIMEOUT_SECONDS must be a number. Got \"${UPDATE_TIMEOUT_SECONDS}\"."
local ERROR_SERVICE="GANTRY_UPDATE_TIMEOUT_SECONDS-is-not-a-number"
_static_variable_add_unique_to_list STATIC_VAR_SERVICES_UPDATE_INPUT_ERROR "${ERROR_SERVICE}"
return 1
fi
local UPDATE_OPTIONS="${GANTRY_UPDATE_OPTIONS:-""}"
local SERVICE_NAME="${1}"
local UPDATE_OPTIONS=
UPDATE_OPTIONS=$(_read_env_or_label "${SERVICE_NAME}" "GANTRY_UPDATE_OPTIONS" "gantry.update.options" "")
local IMAGE="${2}"
local INPUT_ERROR=0
[ -z "${SERVICE_NAME}" ] && log ERROR "Updating service: SERVICE_NAME must not be empty." && INPUT_ERROR=1 && SERVICE_NAME="unknown-service-name"
Expand Down Expand Up @@ -927,7 +976,7 @@ gantry_initialize() {

gantry_get_services_list() {
local SERVICES_EXCLUDED="${GANTRY_SERVICES_EXCLUDED:-""}"
local SERVICES_EXCLUDED_FILTERS="${GANTRY_SERVICES_EXCLUDED_FILTERS:-""}"
local SERVICES_EXCLUDED_FILTERS="${GANTRY_SERVICES_EXCLUDED_FILTERS:-"label=gantry.services.excluded=true"}"
local SERVICES_FILTERS="${GANTRY_SERVICES_FILTERS:-""}"
local SERVICES=
if ! SERVICES=$(_get_services_filted "${SERVICES_FILTERS}"); then
Expand Down
48 changes: 48 additions & 0 deletions tests/gantry_filters_spec.sh
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,54 @@ Describe 'filters'
The stderr should satisfy spec_expect_no_message "${FAILED_TO_REMOVE_IMAGE}.*${IMAGE_WITH_TAG}"
End
End
Describe "test_SERVICES_EXCLUDED_FILTERS_default" "container_test:true"
TEST_NAME="test_SERVICES_EXCLUDED_FILTERS_default"
IMAGE_WITH_TAG=$(get_image_with_tag "${SUITE_NAME}")
SERVICE_NAME="gantry-test-$(unique_id)"
MAX_SERVICES_NUM=10
test_SERVICES_EXCLUDED_FILTERS_default() {
local TEST_NAME="${1}"
local SERVICE_NAME="${2}"
local MAX_SERVICES_NUM="${3}"
reset_gantry_env "${SERVICE_NAME}"
local LABEL="gantry.services.excluded"
for NUM in $(seq 0 "${MAX_SERVICES_NUM}"); do
local SERVICE_NAME_NUM="${SERVICE_NAME}-${NUM}"
docker service update --quiet --label-add "${LABEL}=true" "${SERVICE_NAME_NUM}"
done
# Do not set GANTRY_SERVICES_EXCLUDED_FILTERS, check the default one is working.
run_gantry "${TEST_NAME}"
}
BeforeEach "common_setup_new_image_multiple ${TEST_NAME} ${IMAGE_WITH_TAG} ${SERVICE_NAME} ${MAX_SERVICES_NUM}"
AfterEach "common_cleanup_multiple ${TEST_NAME} ${IMAGE_WITH_TAG} ${SERVICE_NAME} ${MAX_SERVICES_NUM}"
It 'run_test'
When run test_SERVICES_EXCLUDED_FILTERS_default "${TEST_NAME}" "${SERVICE_NAME}" "${MAX_SERVICES_NUM}"
The status should be success
The stdout should satisfy display_output
The stderr should satisfy display_output
The stderr should satisfy spec_expect_no_message "${SKIP_UPDATING_ALL}"
The stderr should satisfy spec_expect_no_message "${SKIP_UPDATING}.*${SERVICE_NAME}"
The stderr should satisfy spec_expect_no_message "${PERFORM_UPDATING}.*${SERVICE_NAME}"
The stderr should satisfy spec_expect_no_message "${NUM_SERVICES_SKIP_JOBS}"
The stderr should satisfy spec_expect_no_message "${NUM_SERVICES_INSPECT_FAILURE}"
The stderr should satisfy spec_expect_no_message "${NUM_SERVICES_NO_NEW_IMAGES}"
The stderr should satisfy spec_expect_no_message "${NUM_SERVICES_UPDATING}"
The stderr should satisfy spec_expect_no_message "${UPDATED}.*${SERVICE_NAME}"
The stderr should satisfy spec_expect_no_message "${NO_UPDATES}.*${SERVICE_NAME}"
The stderr should satisfy spec_expect_no_message "${ROLLING_BACK}.*${SERVICE_NAME}"
The stderr should satisfy spec_expect_no_message "${FAILED_TO_ROLLBACK}.*${SERVICE_NAME}"
The stderr should satisfy spec_expect_no_message "${ROLLED_BACK}.*${SERVICE_NAME}"
The stderr should satisfy spec_expect_message "${NO_SERVICES_UPDATED}"
The stderr should satisfy spec_expect_no_message "${NUM_SERVICES_UPDATED}"
The stderr should satisfy spec_expect_no_message "${NUM_SERVICES_UPDATE_FAILED}"
The stderr should satisfy spec_expect_no_message "${NUM_SERVICES_ERRORS}"
The stderr should satisfy spec_expect_message "${NO_IMAGES_TO_REMOVE}"
The stderr should satisfy spec_expect_no_message "${REMOVING_NUM_IMAGES}"
The stderr should satisfy spec_expect_no_message "${SKIP_REMOVING_IMAGES}"
The stderr should satisfy spec_expect_no_message "${REMOVED_IMAGE}.*${IMAGE_WITH_TAG}"
The stderr should satisfy spec_expect_no_message "${FAILED_TO_REMOVE_IMAGE}.*${IMAGE_WITH_TAG}"
End
End
Describe "test_SERVICES_EXCLUDED_FILTERS_bad" "container_test:false"
TEST_NAME="test_SERVICES_EXCLUDED_FILTERS_bad"
IMAGE_WITH_TAG=$(get_image_with_tag "${SUITE_NAME}")
Expand Down
Loading

0 comments on commit 74e7e53

Please sign in to comment.