Improve make config behaviour (#133)

## Description - This change fixes the unexpected behaviour of the `make config` command which fails when the `asdf` tool isn't correctly set up - `asdf` is no longer installed as part of the repository template, however it is listed as a required dependency - A fix to the `check-file-format` git pre-commit hook is also included as part of this PR ## Context - Fixes #124 - Fixes #126 - Fixes #102 ## Type of changes - [x] Refactoring (non-breaking change) - [ ] New feature (non-breaking change which adds functionality) - [x] Breaking change (fix or feature that would change existing functionality) - [x] Bug fix (non-breaking change which fixes an issue) ## Checklist - [x] I am familiar with the [contributing guidelines](../docs/CONTRIBUTING.md) - [x] I have followed the code style of the project - [ ] I have added tests to cover my changes - [x] I have updated the documentation accordingly - [ ] This PR is a result of pair or mob programming --- ## Sensitive Information Declaration To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including [PII (Personal Identifiable Information) / PID (Personal Identifiable Data)](https://digital.nhs.uk/data-and-information/keeping-data-safe-and-benefitting-the-public) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter. - [x] I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.
nhs-england-tools · Oct 4, 2023 · 5f28100 · 5f28100
1 parent 3928625
commit 5f28100
Show file tree

Hide file tree

Showing 12 changed files with 53 additions and 73 deletions.
diff --git a/.tool-versions b/.tool-versions
@@ -1,22 +1,11 @@
-nodejs 18.17.1 # Always check AWS and Azure runtime support
-python 3.11.4 # Always check AWS and Azure runtime support
-poetry 1.6.1
-yarn 1.22.19
-terraform 1.5.6
-pre-commit 3.3.3
+# This file is for you! Please, updated to the versions agreed by your team.
+
+terraform 1.5.7
+pre-commit 3.4.0
 
 # ==============================================================================
 # The section below is reserved for Docker image versions.
 
-# alpine, SEE: https://hub.docker.com/_/alpine/tags
-# docker/alpine 3.18.3@sha256:c5c5fda71656f28e49ac9c5416b3643eaa6a108a8093151d6d1afc9463be8e33
-
-# nodejs, SEE: https://hub.docker.com/_/node/tags
-# docker/node 18.17.1-alpine3.18@sha256:982b5b6f07cd9241c9ebb163829067deac8eaefc57cfa8f31927f4b18943d971
-
-# python, SEE: https://hub.docker.com/_/python/tags
-# docker/python 3.11.4-alpine3.18@sha256:0135ae6442d1269379860b361760ad2cf6ab7c403d21935a8015b48d5bf78a86
-
 # terraform, SEE: https://hub.docker.com/r/hashicorp/terraform/tags
 # docker/hashicorp/terraform 1.5.6@sha256:180a7efa983386a27b43657ed610e9deed9e6c3848d54f9ea9b6cb8a5c8c25f5
 

diff --git a/Makefile b/Makefile
@@ -25,8 +25,6 @@ clean:: # Clean-up project resources (main) @Operations
 config:: # Configure development environment (main) @Configuration
 	# TODO: Use only 'make' targets that are specific to this project, e.g. you may not need to install Node.js
 	make \
-		nodejs-install \
-		python-install \
 		terraform-install
 
 # ==============================================================================

diff --git a/README.md b/README.md
@@ -7,11 +7,10 @@ Start with an overview or a brief description of what the project is about and w
 
 Welcome to our repository template designed to streamline your project setup! This robust template provides a reliable starting point for your new projects, covering an essential tech stack and encouraging best practices in documenting.
 
-This repository template aims to foster a user-friendly development environment by ensuring that every included file is concise and adequately self-documented. By adhering to this standard, we can promote increased clarity and maintainability throughout your project's lifecycle. Bundled within this template are resources that pave the way for seamless repository creation. Our supported tech stack includes:
+This repository template aims to foster a user-friendly development environment by ensuring that every included file is concise and adequately self-documented. By adhering to this standard, we can promote increased clarity and maintainability throughout your project's lifecycle. Bundled within this template are resources that pave the way for seamless repository creation. Currently supported technologies are:
 
 - Terraform
-- Python
-- Node.js
+- Docker
 
 Make use of this repository template to expedite your project setup and enhance your productivity right from the get-go. Enjoy the advantage of having a well-structured, self-documented project that reduces overhead and increases focus on what truly matters - coding!
 
@@ -21,12 +20,12 @@ Make use of this repository template to expedite your project setup and enhance
   - [Table of Contents](#table-of-contents)
   - [Setup](#setup)
     - [Prerequisites](#prerequisites)
-    - [Installation](#installation)
+    - [Configuration](#configuration)
   - [Usage](#usage)
     - [Testing](#testing)
-  - [Architecture](#architecture)
+  - [Design](#design)
     - [Diagrams](#diagrams)
-    - [Configuration](#configuration)
+    - [Modularity](#modularity)
   - [Contributing](#contributing)
   - [Contacts](#contacts)
   - [Licence](#licence)
@@ -44,35 +43,30 @@ cd nhs-england-tools/repository-template
 
 ### Prerequisites
 
-The following software packages or their equivalents are expected to be installed
-
-- [GNU make](https://www.gnu.org/software/make/) **v3.82 or later: OS X users, take note below**
-- [Docker](https://www.docker.com/)
+The following software packages, or their equivalents, are expected to be installed:
 
-Note that the version of GNU Make available by default on OS X is earlier than this.  You will need to upgrade it, or certain `make` tasks will fail.  On OS X, you will need [homebrew](https://brew.sh/) installed, then to install `make`, like so:
-
-```shell
-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
-brew install make
-```
+- [docker](https://www.docker.com/) container runtime or a compatible tool, e.g. [podman](https://podman.io/),
+- [asdf](https://asdf-vm.com/) version manager,
+- [GNU make](https://www.gnu.org/software/make/) 3.82 or later,
+- [GNU coreutils](https://www.gnu.org/software/coreutils/) and [GNU binutils](https://www.gnu.org/software/binutils/) may be required to build dependencies like Python, which may need to be compiled during installation. For macOS users, this has been scripted and automated by the `dotfiles` project; please see this [script](https://github.com/nhs-england-tools/dotfiles/blob/main/assets/20-install-base-packages.macos.sh) for details.
 
-You will then see instructions to fix your `$PATH` variable to make the newly installed version available.
-If you are using [dotfiles](https://github.com/nhs-england-tools/dotfiles), this is all done for you.
+> [!NOTE]<br>
+> The version of GNU make available by default on macOS is earlier than 3.82. You will need to upgrade it or certain `make` tasks will fail. On macOS, you will need [homebrew](https://brew.sh/) installed, then to install `make`, like so:
+>
+> ```shell
+> brew install make
+> ```
+>
+> You will then see instructions to fix your `$PATH` variable to make the newly installed version available. If you are using [dotfiles](https://github.com/nhs-england-tools/dotfiles), this is all done for you.
 
-### Installation
+### Configuration
 
-Install and configure toolchain dependencies
+Installation and configuration of the toolchain dependencies
 
 ```shell
 make config
 ```
 
-If this repository is
-
-```shell
-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/org/repo/branch/install.sh)"
-```
-
 ## Usage
 
 After a successful installation, provide an informative example of how this project can be used. Additional code snippets, screenshots and demos work well in this space. You may also link to the other documentation resources, e.g. the [User Guide](./docs/user-guide.md) to demonstrate more use cases and to show more features.
@@ -81,15 +75,15 @@ After a successful installation, provide an informative example of how this proj
 
 There are `make` tasks for you to configure to run your tests.  Run `make test` to see how they work.  You should be able to use the same entry points for local development as in your CI pipeline.
 
-## Architecture
+## Design
 
 ### Diagrams
 
 The [C4 model](https://c4model.com/) is a simple and intuitive way to create software architecture diagrams that are clear, consistent, scalable and most importantly collaborative. This should result in documenting all the system interfaces, external dependencies and integration points.
 
 ![Repository Template](./docs/diagrams/Repository_Template_GitHub_Generic.png)
 
-### Configuration
+### Modularity
 
 Most of the projects are built with customisability and extendability in mind. At a minimum, this can be achieved by implementing service level configuration options and settings. The intention of this section is to show how this can be used. If the system processes data, you could mention here for example how the input is prepared for testing - anonymised, synthetic or live data.
 

diff --git a/scripts/config/pre-commit.yaml b/scripts/config/pre-commit.yaml
@@ -10,7 +10,8 @@ repos:
   hooks:
   - id: check-file-format
     name: Check File Format
-    entry: check=staged-changes./scripts/githooks/check-file-format.sh
+    entry: ./scripts/githooks/check-file-format.sh
+    args: ["check=staged-changes"]
     language: script
     pass_filenames: false
 - repo: local

diff --git a/scripts/docker/docker.lib.sh b/scripts/docker/docker.lib.sh
@@ -11,8 +11,9 @@ set -euo pipefail
 #   $ source ./docker.lib.sh
 #
 # Arguments (provided as environment variables):
-#   DOCKER_IMAGE=ghcr.io/org/repo   # Docker image name
-#   DOCKER_TITLE="My Docker image"  # Docker image title
+#   DOCKER_IMAGE=ghcr.io/org/repo             # Docker image name
+#   DOCKER_TITLE="My Docker image"            # Docker image title
+#   TOOL_VERSIONS=$project_dir/.tool-versions # Path to the tool versions file
 
 # ==============================================================================
 # Functions to be used with custom images.
@@ -164,7 +165,7 @@ function docker-get-image-version-and-pull() {
 
   # Get the image full version from the '.tool-versions' file,
   # match it by name and version regex, if given.
-  local versions_file="$(git rev-parse --show-toplevel)/.tool-versions"
+  local versions_file="${TOOL_VERSIONS:=$(git rev-parse --show-toplevel)/.tool-versions}"
   local version="latest"
   if [ -f "$versions_file" ]; then
     line=$(grep "docker/${name} " "$versions_file" | sed "s/^#\s*//; s/\s*#.*$//" | grep "${match_version:-'.*'}")
@@ -217,13 +218,13 @@ function _create-effective-dockerfile() {
 function _replace-image-latest-by-specific-version() {
 
   local dir=${dir:-$PWD}
-  local versions_file=$(git rev-parse --show-toplevel)/.tool-versions
+  local versions_file="${TOOL_VERSIONS:=$(git rev-parse --show-toplevel)/.tool-versions}"
   local dockerfile="${dir}/Dockerfile.effective"
   local build_datetime=${BUILD_DATETIME:-$(date -u +'%Y-%m-%dT%H:%M:%S%z')}
 
   if [ -f "$versions_file" ]; then
     # First, list the entries specific for Docker to take precedence, then the rest but exclude comments
-    content=$(grep " docker/" "$versions_file"; grep -v " docker/" "$versions_file" | grep -v "^#")
+    content=$(grep " docker/" "$versions_file"; grep -v " docker/" "$versions_file" ||: | grep -v "^#")
     echo "$content" | while IFS= read -r line; do
       [ -z "$line" ] && continue
       line=$(echo "$line" | sed "s/^#\s*//; s/\s*#.*$//" | sed "s;docker/;;")

diff --git a/scripts/docker/docker.mk b/scripts/docker/docker.mk
@@ -45,6 +45,7 @@ docker-example-build: # Build Docker example @ExamplesAndTests
 	cd scripts/docker/examples/python
 	DOCKER_IMAGE=repository-template/docker-example-python
 	DOCKER_TITLE="Repository Template Docker Python Example"
+	TOOL_VERSIONS="$(shell git rev-parse --show-toplevel)/scripts/docker/examples/python/.tool-versions.example"
 	docker-build
 
 docker-example-lint: # Lint Docker example @ExamplesAndTests

diff --git a/scripts/docker/examples/python/.tool-versions.example b/scripts/docker/examples/python/.tool-versions.example
@@ -0,0 +1,2 @@
+# python, SEE: https://hub.docker.com/_/python/tags
+# docker/python 3.11.4-alpine3.18@sha256:0135ae6442d1269379860b361760ad2cf6ab7c403d21935a8015b48d5bf78a86
diff --git a/scripts/docker/examples/python/assets/hello_world/requirements.txt b/scripts/docker/examples/python/assets/hello_world/requirements.txt
@@ -1,2 +1,12 @@
-flask==2.3.2
-Flask-WTF==1.1.1
+blinker==1.6.2
+click==8.1.7
+Flask-WTF==1.2.0
+Flask==2.3.3
+itsdangerous==2.1.2
+Jinja2==3.1.2
+MarkupSafe==2.1.3
+pip==23.1.2
+setuptools==65.5.1
+Werkzeug==3.0.0
+wheel==0.41.1
+WTForms==3.0.1
diff --git a/scripts/docker/tests/.tool-versions.test b/scripts/docker/tests/.tool-versions.test
@@ -0,0 +1,2 @@
+# python, SEE: https://hub.docker.com/_/python/tags
+# docker/python 3.11.4-alpine3.18@sha256:0135ae6442d1269379860b361760ad2cf6ab7c403d21935a8015b48d5bf78a86
diff --git a/scripts/docker/tests/docker.test.sh b/scripts/docker/tests/docker.test.sh
@@ -74,6 +74,7 @@ function test-docker-build() {
 function test-docker-image-from-signature() {
 
   # Arrange
+  TOOL_VERSIONS="$(git rev-parse --show-toplevel)/scripts/docker/tests/.tool-versions.test"
   cp Dockerfile Dockerfile.effective
   # Act
   _replace-image-latest-by-specific-version

diff --git a/scripts/init.mk b/scripts/init.mk
@@ -29,14 +29,6 @@ shellscript-lint-all: # Lint all shell scripts in this project, do not fail on e
 		file=$${file} scripts/shellscript-linter.sh ||:
 	done
 
-nodejs-install: # Install Node.js @Installation
-	make _install-dependency name="nodejs"
-	make _install-dependency name="yarn" version=latest
-
-python-install: # Install Python @Installation
-	make _install-dependency name="python"
-	make _install-dependency name="poetry" version=latest
-
 githooks-config: # Trigger Git hooks on commit that are defined in this repository @Configuration
 	make _install-dependency name="pre-commit"
 	pre-commit install \
@@ -48,14 +40,6 @@ githooks-run: # Run git hooks configured in this repository @Operations
 		--config scripts/config/pre-commit.yaml \
 		--all-files
 
-asdf-install: # Install asdf @Installation
-	if [ -d "${HOME}/.asdf" ]; then
-		( cd "${HOME}/.asdf"; git pull )
-	else
-		git clone --depth=1 https://github.com/asdf-vm/asdf.git "${HOME}/.asdf" ||:
-	fi
-	asdf plugin update --all
-
 _install-dependency: # Install asdf dependency - mandatory: name=[listed in the '.tool-versions' file]; optional: version=[if not listed]
 	asdf plugin add ${name} ||:
 	asdf install ${name} $(or ${version},)
@@ -73,7 +57,6 @@ clean:: # Remove all generated and temporary files (common) @Operations
 
 config:: # Configure development environment (common) @Configuration
 	make \
-		asdf-install \
 		githooks-config
 
 help: # Print help @Others
@@ -158,15 +141,12 @@ HELP_SCRIPT = \
 
 ${VERBOSE}.SILENT: \
 	_install-dependency \
-	asdf-install \
 	clean \
 	config \
 	githooks-config \
 	githooks-run \
 	help \
 	list-variables \
-	nodejs-install \
-	python-install \
 	runner-act \
 	shellscript-lint-all \
 	version-create-effective-file \
diff --git a/scripts/terraform/terraform.mk b/scripts/terraform/terraform.mk
@@ -90,6 +90,7 @@ ${VERBOSE}.SILENT: \
 	terraform-example-provision-aws-infrastructure \
 	terraform-fmt \
 	terraform-init \
+	terraform-install \
 	terraform-plan \
 	terraform-shellscript-lint \
 	terraform-validate \
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		# python, SEE: https://hub.docker.com/_/python/tags
		# docker/python 3.11.4-alpine3.18@sha256:0135ae6442d1269379860b361760ad2cf6ab7c403d21935a8015b48d5bf78a86