diff --git a/README.md b/README.md index fa95444..eb0e320 100644 --- a/README.md +++ b/README.md @@ -91,7 +91,7 @@ The template is a [GitHub template repository --- ``` -Now, give your repository a name, choose the **visibility** (Public or Private) and click **Create repository from template**. Do not select **include all branches**. +Now, give your repository a name, choose the **visibility** (Public or Private) and click **Create repository from template**. ```{figure} docs/images/github-template-create.png --- @@ -174,15 +174,32 @@ For example, see this template as a live demo > 🌟 [worldbank.github.io/template](http://worldbank.github.io/template) (Live Demo) -### Adding Content +### Add content The template is created as a [Jupyter Book](https://jupyterbook.org/intro.html) - an open-source project to build beautiful, publication-quality books and documents from computational content. Let's see below how to add, execute and publish new content for your project. -#### Table of Contents +#### Updating the Jupyter Book `_config.yml` metadata -When ready to publish the *documentation* on [GitHub Pages](https://pages.github.com/), all you need to do is edit the [table of contents](#table-of-contents) and add and/or update content you would like to display. [Jupyter Book](https://jupyterbook.org) supports content written as [Markdown](https://daringfireball.net/projects/markdown/), [Jupyter](https://jupyter.org) notebooks and [reStructuredText](https://docutils.sourceforge.io/rst.html) files and the `docs/_toc.yml` file controls the [table of contents](#table-of-contents) of your book. +To configure your Jupyter Book for your project, you’ll need to update the `_config.yml` file. This file controls various aspects of the Jupyter Book, including the project title, description, and relevant URLs. Below is a template to update this file to reflect the project’s details. -The template comes with the [table of contents](#table-of-contents) below as an example. +```yaml +# Book settings +title: +author: + +repository: +url: https://github.com// + +# Jupyter Book options +execute: + execute_notebooks: "auto" # Automatically execute notebooks during the build process +``` + +#### Update table of contents + +When ready to publish the *documentation* on [GitHub Pages](https://pages.github.com/), all you need to do is edit the [table of contents](https://github.com/worldbank/template/blob/main/docs/_toc.yml) and add and/or update content you would like to display. [Jupyter Book](https://jupyterbook.org) supports content written as [Markdown](https://daringfireball.net/projects/markdown/), [Jupyter](https://jupyter.org) notebooks and [reStructuredText](https://docutils.sourceforge.io/rst.html) files and the `docs/_toc.yml` file controls the [table of contents](https://github.com/worldbank/template/blob/main/docs/_toc.yml) of your book. + +The template comes with the [table of contents](https://github.com/worldbank/template/blob/main/docs/_toc.yml) below as an example. ```yaml @@ -191,72 +208,127 @@ root: README parts: -- caption: Documentation + - caption: Examples numbered: True chapters: - - file: notebooks/world-bank-api.ipynb -- caption: Additional Resources - chapters: - - url: - title: Development Data Partnership - - url: - title: World Bank DEC - - url: - title: World Bank DIME - + - file: notebooks/world-bank-api.ipynb + - file: notebooks/world-bank-package.ipynb + - file: notebooks/nasa-apod.ipynb + - file: notebooks/bibliography.ipynb ``` ```{seealso} [Jupyter Book Structure and organize content](https://jupyterbook.org/en/stable/basics/organize.html) ``` -#### Dependencies +#### Add executable content + +[Jupyter Notebooks](https://jupyter.org) can be beautifully rendered and downloaded from your book. By default, the template will render any files listed on the [table of contents](#update-table-of-contents) that have a notebook structure. The template comes with a Jupyter notebook example, `notebooks/world-bank-api.ipynb`, to illustrate. + +```{important} + +By default, Jupyter notebooks are **not** executed. However, you can configure[Jupyter Book](https://jupyterbook.org) to run notebooks during the build process (on GitHub), allowing **code outputs** and **interactive visualizations** to be generated and included in the *documentation* automatically. When enabled, Jupyter notebooks are executed by [GitHub Actions](https://github.com/features/actions) each time a commit is made to the `main` branch. For this to work, it’s crucial to ensure that all necessary [dependencies](##use-pyproject-toml-for-python-package-management) are included in the repository. If you want to prevent a specific notebook from being executed, you can [exclude it from execution](https://jupyterbook.org/en/stable/content/execute.html#exclude-files-from-execution). +``` + +```{seealso} +[Jupyter Book Write executable content](https://jupyterbook.org/en/stable/content/executable/index.html) +``` + +### Use `pyproject.toml` for Python Package Management + +While the template recommends using [Conda](https://conda.io/projects/conda/en/latest/index.html) (or [Mamba](https://github.com/mamba-org/mamba)) as the environment manager and managing dependencies through an `environment.yml` file, there is an alternative approach that leverages `pyproject.toml`. This can be particularly advantageous if your project is a Python package or if you want to simplify and standardize the management of Python-specific dependencies. + +#### Why use `pyproject.toml`? The next step is ensure your code is maintainable, reliable and reproducible by including any dependencies and requirements, such as packages, configurations, secrets (template) and additional instructions. -The template suggests to use [conda](https://docs.conda.io/) (or [mamba](https://mamba.readthedocs.io/en/latest/)) as environment manager and, as [conventional](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html), the environment is controlled by the `environment.yml` file. +1. **Standardization**: `pyproject.toml` is a modern, standardized format defined by [PEP 518](https://peps.python.org/pep-0518/) and [PEP 621](https://peps.python.org/pep-0621/) that centralizes project configuration in Python projects, including build requirements and dependencies. -The `environment.yml` file is where you specify any packages available on the [Anaconda repository](https://anaconda.org) as well as from the Anaconda Cloud (including [conda-forge](https://conda-forge.org)) to install for your project. Ensure to include the pinned version of packages required by your project (including by Jupyter notebooks). +2. **Python Packaging**: If your project is to be distributed as a package, `pyproject.toml` is the preferred way to define build tools (like `setuptools` or `poetry`) and metadata for your package (like name, version, dependencies, etc.). It allows tools like `pip` and `build` to install and package your project more effectively. -```yaml -channels: - - conda-forge - - defaults -dependencies: - - python=3.9 - - bokeh=2.4.3 - - pandas=1.4.3 - - pip: - - requests==2.28.1 -``` +3. **Compatibility with Tools**: The `pyproject.toml` file is compatible with multiple Python packaging and dependency management tools such as `poetry` and `pip`. This allows for smoother integration with CI/CD pipelines, PyPI, and other environments. -To (re)create the environment on your installation of [conda](https://conda.io) via [anaconda](https://docs.anaconda.com/anaconda/install/), [miniconda](https://docs.conda.io/projects/continuumio-conda/en/latest/user-guide/install/) or preferably [miniforge](https://github.com/conda-forge/miniforge), you only need to pass the `environment.yml` file, which will install requirements and guarantee that whoever uses your code has the necessary packages (and correct versions). By default, the template uses [Python 3.9](https://www.python.org). +4. **Separation of Concerns**: While Conda manages both system-level and Python-specific packages, using `pyproject.toml` helps isolate Python dependencies. This is useful if your project uses primarily Python packages and you want finer control over Python versioning and dependency resolution. -```shell -conda env create -n -f environment.yml -``` +#### Example: Using `pyproject.toml` -In case your project uses Python, it is *strongly* recommended to distribute it as a [package](https://packaging.python.org/). +This `pyproject.toml` file specifies the dependencies and other metadata for your Python package. You can install these packages using `pip`, ensuring that your Python environment is properly managed. You can still use Conda for system-level packages (such as `libc`, `gdal`, etc.), while using `pyproject.toml` for Python package management. -```{important} -The template contains an example - the [datalab](https://github.com/worldbank/template/tree/main/src/datalab) Python package - and will automatically find and install any `src` packages as long as `pyproject.yml` is kept up-to-date. -``` +1. **`pyproject.toml` Example**: -```{seealso} -[Conda Managing Environments](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html) -``` + ```toml + [build-system] + requires = ["hatchling>=1.21.0", "hatch-vcs>=0.3.0"] + build-backend = "hatchling.build" -#### Jupyter Notebooks + [project] + name = "template" + description = "A data science project" + readme = { file = "README.md", content-type = "text/markdown" } + license = { file = "LICENSE" } + authors = [ + { name = "Your Name", email = "your.email@example.com" } + ] + dynamic = ["version"] -[Jupyter Notebooks](https://jupyter.org) can be beautifully rendered and downloaded from your book. By default, the template will render any files listed on the [table of contents](#table-of-contents) that have a notebook structure. The template comes with a Jupyter notebook example, `notebooks/world-bank-api.ipynb`, to illustrate. + python = ">=3.9" + dependencies = [ + "pandas>=1.4.3,<2", + ] + [project.optional-dependencies] + docs = [ + "docutils==0.17.1", + "jupyter-book>=1,<2", + ] -```{important} -Optionally, [Jupyter Book](https://jupyterbook.org) can execute notebooks during the build (on GitHub) and display **code outputs** and **interactive visualizations** as part of the *documentation* on the fly. In this case, Jupyter notebooks will be executed by [GitHub Actions](https://github.com/features/actions) during build on each commit to the `main` branch. Thus, it is important to include all [requirements and dependencies](#dependencies) in the repository. In case you would like to ignore a notebook, you can [exclude files from execution](https://jupyterbook.org/en/stable/content/execute.html#exclude-files-from-execution). -``` + [tool.hatch.build.targets.sdist] + include = [ + "src/**/*" + ] + + [tool.hatch.version] + source = "vcs" + ``` + +2. **Keep the Conda Environment for System-level Packages**: + You can continue to use `environment.yml` to specify non-Python dependencies or packages not available on PyPI, such as `mamba` or `gdal`. + + ```yaml + channels: + - conda-forge + dependencies: + - python=3.9 + - mamba + - gdal + ``` + +3. **Installation**: + To create an environment, you would first install the Conda dependencies and then use `pip` to install Python-specific dependencies from `pyproject.toml`. Alternatively, you can skip Conda and use `pip` for the entire setup. + + ```shell + # Create Conda environment + conda env create -f environment.yml -n + + # Activate the environment + conda activate + + # Install Python dependencies + pip install . + ``` + +#### Distributing Your Project as a Python Package + +If your project uses [Python](https://python.org), it’s highly recommended to distribute it as a [package](https://packaging.python.org/en/latest/tutorials/packaging-projects/). By including a `pyproject.toml` file, the packaging process becomes more streamlined. Additionally: + +- The `pyproject.toml` file acts as a single source of truth for your Python dependencies and project metadata. +- Any packages in the `src/` folder will be automatically discovered and installed. +- Using `pyproject.toml` future-proofs your setup by aligning with modern packaging standards. +-You can combine Conda for system-level dependencies with `pyproject.toml` for Python dependencies, using Conda for environments and Pip/Poetry for Python packages. ```{seealso} -[Jupyter Book Write executable content](https://jupyterbook.org/en/stable/content/executable/index.html) +- [Writing your pyproject.toml](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/) +- [Conda Managing Environments](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html) ``` ## Code of Conduct diff --git a/notebooks/environment.yml b/notebooks/environment.yml deleted file mode 100644 index 500c72c..0000000 --- a/notebooks/environment.yml +++ /dev/null @@ -1,10 +0,0 @@ -channels: - - conda-forge - - defaults -dependencies: - - python=3.10 - - bokeh - - pandas - - pip - - pip: - - requests==2.28.1 diff --git a/pyproject.toml b/pyproject.toml index 50b1a51..9da3295 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -42,8 +42,10 @@ skip = 'docs/_build,docs/references.bib,*.png,*.gz,*.whl' ignore-regex = '^\s*"image\/png":\s.*' ignore-words-list = "gost," -[tool.hatch.build.targets.wheel] -packages = ["src/*"] +[tool.hatch.build.targets.sdist] +include = [ + "src/**/*" +] [tool.hatch.version] source = "vcs"