Skip to content

Commit

Permalink
Fix typos and refactor for pipx cluster use
Browse files Browse the repository at this point in the history
  • Loading branch information
@f-PLT
f-PLT committed May 22, 2024
1 parent cdfd467 commit 2ed9ba2
Show file tree
Hide file tree
Showing 3 changed files with 114 additions and 30 deletions.
44 changes: 37 additions & 7 deletions .make/base.make
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ SHELL := /usr/bin/env bash
BUMP_TOOL := bump-my-version
APP_VERSION := 0.0.0
DOCKER_COMPOSE ?= docker compose
POETRY_VERSION := 1.8.2
AUTO_INSTALL ?=

# Conda variables
Expand Down Expand Up @@ -163,7 +162,7 @@ poetry-install-auto: ## Install Poetry in activated Conda environment, or with p
pip install --user pipx; \
pipx ensurepath; \
fi; \
pipx install poetry==$(POETRY_VERSION); \
pipx install poetry; \
else \
echo "Installing poetry with Conda"; \
make -s _conda-poetry-install; \
Expand All @@ -178,8 +177,21 @@ poetry-install: ## Install standalone Poetry using pipx and create Poetry env. W
if [ "$(AUTO_INSTALL)" = "true" ]; then \
ans="y";\
else \
echo""; \
echo -n "Would you like to install Poetry using pipx? (This will also install pipx if needed) [y/N]: "; \
pipx --version; \
if [ $$? != "0" ]; then \
echo""; \
echo -e "\e[1;39;41m-- WARNING --\e[0m The following pip has been found and will be used to install pipx: "; \
echo " -> "$$(which pip); \
echo""; \
echo "If you do not have write permission to that environment, you will need to either activate"; \
echo "a different environment, or create a virtual one (ex. venv) to install pipx into it."; \
echo "See documentation for more information."; \
echo""; \
echo -n "Would you like to install pipx and Poetry? [y/N]: "; \
else \
echo""; \
echo -n "Would you like to install Poetry using pipx? [y/N]: "; \
fi; \
read ans; \
fi; \
case $$ans in \
Expand All @@ -191,7 +203,7 @@ poetry-install: ## Install standalone Poetry using pipx and create Poetry env. W
pipx ensurepath; \
fi; \
echo "Installing Poetry"; \
pipx install poetry==$(POETRY_VERSION); \
pipx install poetry; \
make -s poetry-create-env; \
;; \
*) \
Expand Down Expand Up @@ -254,9 +266,27 @@ poetry-remove-env: ## Remove current project's Poetry managed environment.
esac; \
fi; \

.PHONY: poetry-uninstall
poetry-uninstall: poetry-remove-env ## Uninstall pipx-installed Poetry and the created environment
@if [ "$(AUTO_INSTALL)" = "true" ]; then \
ans="y";\
else \
echo""; \
echo -n "Would you like to uninstall pipx-installed Poetry? [y/N]: "; \
read ans; \
fi; \
case $$ans in \
[Yy]*) \
pipx uninstall poetry; \
;; \
*) \
echo "Skipping uninstallation."; \
echo " "; \
;; \
esac; \

.PHONY: poetry-uninstall-pipx
poetry-uninstall-pipx: poetry-remove-env ## Uninstall pipx-installed Poetry, the created env and pipx
poetry-uninstall-pipx: poetry-remove-env ## Uninstall pipx-installed Poetry, the created environment and pipx
@if [ "$(AUTO_INSTALL)" = "true" ]; then \
ans="y";\
else \
Expand Down Expand Up @@ -302,7 +332,7 @@ install-with-lab: poetry-install-auto ## Install the application and it's dev de
install-package: poetry-install-auto ## Install the application package only
@poetry install

## -- Versioning targets ------------------------------------------------------------------------------------------- ##
## -- Versioning targets -------------------------------------------------------------------------------------------- ##

# Use the "dry" target for a dry-run version bump, ex.
# make bump-major dry
Expand Down
2 changes: 1 addition & 1 deletion Makefile.private.example
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,5 @@ DOCKER_COMPOSE := docker compose
# a 'true' value will automatically install/remove without asking beforehand.
AUTO_INSTALL := false

## -- Private targets -------------------------------------------------------------------------------------------------##
## -- Private targets ------------------------------------------------------------------------------------------------##

98 changes: 76 additions & 22 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ The following Makefile files are project or user specific:
The different targets and their description can be examined by executing the command
`make targets`

Ex.

![](img/make_targets.png)

## Installation
Expand Down Expand Up @@ -62,6 +64,39 @@ clusters). This is why the `pipx` option for Poetry is also enabled in this proj
**Unless you really know what you are doing, it is not recommended to install Poetry
as a standalone tool (with pipx) while also using Conda environments.**

There are 2 recommended ways to manage `Poetry` if it is not already configured on
your system. Using `Conda`, or installing `Poetry` as a standalone tool using `pipx`

It is not recommended to install `Poetry` in the same environment that will be managed
by `Poetry` in order to minimize dependency conflicts.

#### Pipx particularities

**WARNING** : If you want to install `Poetry` using `pipx`, and pipx is not already
installed on your system, it will be installed using `pip`.

Write access to that environment is required (use `which pip` to learn which
environment is active if not sure).

Particularly on remote compute clusters (SLURM), default system `pip` will probably not
be writable by users. One way around it is to create a generic virtual environment
using `venv` like so (preferably in your $HOME):

```
python -m venv $HOME/.venv
source $HOME/.venv/bin/activate
pip install pipx
pipx ensurepath
pipx install poetry
```

This virtual environment can be deactivated afterward, as Poetry will still be
available to the user.

```
deactivate
```

### Conda targets

If you need or want to install Conda:
Expand All @@ -79,16 +114,22 @@ To remove the conda environment:
make conda-clean-env
```

Make sure to activate the configured environment before installing this package.
To install `Poetry` using `Conda` (after conda environment as been created:

### Poetry targets
```
make conda-poetry-install
```

To remove poetry from the Conda environment:

There are 2 possibilities available in how to manage `Poetry` if it is not already
configured on your system.
```
make conda-poetry-uninstall
```

**It is not recommended to install `Poetry` in the same environment that will be managed
by `Poetry`; It should be installed by `Conda` (as in, `conda install poetry`) or
on the system using `pipx`, in order to minimize dependency conflicts.**
Make sure to activate the conda environment before moving on the
[install targets](#install-targets).

### Poetry targets

The following target will first try to install `Poetry` in the active `Conda`
environment; if it can't find `Conda`, it will proceed to install via `pipx`
Expand All @@ -97,33 +138,45 @@ environment; if it can't find `Conda`, it will proceed to install via `pipx`
make poetry-install-auto
```

To install `Poetry` using `Conda`:
The following target will allow environment management directly with a standalone
`Poetry` installation through `pipx`. It will also create a virtual environment managed
by `Poetry` that uses Python 3.10.

```
make conda-poetry-install
make poetry-install
```

Using `pipx` will instead allow environment management directly with `Poetry`. The
following target will also make `Poetry` use a Python 3.10 environment for this
project.
If you already have Poetry installed and configured, or want to recreate it later,
an environment for the project can also be created using:

```
make poetry-install-pipx
make poetry-create-env
```

This will also create a virtual environment managed by `Poetry`.
and removed using:

A standalone environment can also be created later using the `make poetry-create-env`
command, and removed with the `make poetry-remove-env` command.
```
make poetry-remove-env
```

Information about the currently active environment used by Poetry,
whether Conda or Poetry, can be seen using the `make poetry-env-info` command.
Information about the currently active environment used by `Poetry`,
whether `Conda` or `Poetry`, can be seen using:

Both install methods can also be cleaned up:
```
make poetry-env-info
```

To remove `Poetry` that was installed with `pipx` (be sure to execute this command in
the environment where pipx is installed):

```
make poetry-uninstall
```

To uninstall both `Poetry` and `pipx` (again, be sure to execute this command in
the environment where pipx is installed):

```
make conda-poetry-uninstall
# or
make poetry-uninstall-pipx
```

Expand All @@ -132,7 +185,8 @@ make poetry-uninstall-pipx
If you have an active `Conda` environment and install `Poetry` using `pipx`,
you will have to use `poetry run python <your_command_or_script_path>` instead of
`python <your_command_or_script_path>`, (which is normal when using Poetry) as
`python` will use Conda's active environment.
Conda's active environment will define the available default `python` executable.


### Install targets

Expand Down

0 comments on commit 2ed9ba2

Please sign in to comment.