Skip to content

Commit

Permalink
Lint spacing
Browse files Browse the repository at this point in the history
  • Loading branch information
tomalrussell committed Oct 28, 2024
1 parent 1503e5a commit 09365f5
Show file tree
Hide file tree
Showing 4 changed files with 56 additions and 42 deletions.
13 changes: 10 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ workflow can be used to analyse environmental risks to infrastructure
networks using global open data. It is a work in progress.

Goals:

- Automated pipeline for reproducible analysis anywhere in the world
- Maps per-country and of larger areas
- Charts/stats of exposure per admin region, per hazard type, scenario, epoch
Expand All @@ -20,13 +21,15 @@ Goals:
- Estimate indirect effects of disruption - people affected, economic activity disrupted

Non-goals:

- Using closed data, which may be appropriate for other projects or use-cases
- Detailed operational/engineering level simulation
- Long-term planning

## Installation

Install `open-gira` by cloning the repository:

```bash
git clone https://github.com/nismod/open-gira.git
```
Expand All @@ -39,11 +42,13 @@ manage these `conda` packages.

Having installed one of the suggested package managers, to create the
`open-gira` conda environment:

```bash
micromamba create -f environment.yml -y
```

And to activate the environment:

```bash
micromamba activate open-gira
```
Expand All @@ -52,9 +57,10 @@ micromamba activate open-gira

Some rules use the `wget` utility to download files.

On Linux or MacOS, you may already have the `wget` utility available. If not,
it should be possible to install with your usual package manager (e.g. apt,
On Linux or MacOS, you may already have the `wget` utility available. If not,
it should be possible to install with your usual package manager (e.g. apt,
MacPorts, brew), or else using micromamba:

```bash
micromamba install wget
```
Expand All @@ -80,6 +86,7 @@ Note that all subsequent commands given in the documentation assume that the
Workflow steps are tested using small sample datasets.

To run the tests:

```bash
python -m pytest tests
```
Expand Down Expand Up @@ -164,5 +171,5 @@ Programme. The views expressed here do not necessarily reflect the UK
government's official policies.

This research has also been supported by funding from the World Bank
Group, and the UK Natural Environment Research Council (NERC) through
Group, and the UK Natural Environment Research Council (NERC) through
the UK Centre for Greening Finance and Investment (CGFI).
40 changes: 21 additions & 19 deletions docs/src/user-guide/usage/network-creation/rail.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,32 +21,34 @@ from user-supplied sources (e.g. rehabilitation costs).
## Configuration

To specify a desired network:

- Review and amend the spreadsheets in `bundled_data/transport`, these supply
information that is used to gap-fill or extend what can be determined from OSM alone.
- Review and amend `config/config.yaml`:
- The `infrastructure_datasets` map should contain a key pointing to an `.osm.pbf`
file URL for desired area. There are currently entries for the planet,
for (some definition of) continents and several countries. We use
the [geofabrik](http://download.geofabrik.de/) service for continent and
country-level OSM extracts.
- Check the OSM filter file pointed to by `network_filters.rail`.
This file specifies which [elements](https://wiki.openstreetmap.org/wiki/Elements)
(nodes, ways or relations) to keep (or reject) from the multitude of data
in an OSM file. See the filter expressions section
[here](https://docs.osmcode.org/osmium/latest/osmium-tags-filter.html)
for more information on the syntax of these files.
- Check and amend `keep_tags.rail`. This list of strings specifies which
`tags` (attributes) to retain on the filtered elements we extract from
the `.osm.pbf` file.
- Review `slice_count`. This controls the degree of parallelism possible.
With it set to 1, there is no spatial slicing (we create the network in
a single chunk). To speed network creation for large domains, it can be
set to a larger square number. The first square number greater than your
number of available CPUs is a good heuristic.
- The `infrastructure_datasets` map should contain a key pointing to an `.osm.pbf`
file URL for desired area. There are currently entries for the planet,
for (some definition of) continents and several countries. We use
the [geofabrik](http://download.geofabrik.de/) service for continent and
country-level OSM extracts.
- Check the OSM filter file pointed to by `network_filters.rail`.
This file specifies which [elements](https://wiki.openstreetmap.org/wiki/Elements)
(nodes, ways or relations) to keep (or reject) from the multitude of data
in an OSM file. See the filter expressions section
[here](https://docs.osmcode.org/osmium/latest/osmium-tags-filter.html)
for more information on the syntax of these files.
- Check and amend `keep_tags.rail`. This list of strings specifies which
`tags` (attributes) to retain on the filtered elements we extract from
the `.osm.pbf` file.
- Review `slice_count`. This controls the degree of parallelism possible.
With it set to 1, there is no spatial slicing (we create the network in
a single chunk). To speed network creation for large domains, it can be
set to a larger square number. The first square number greater than your
number of available CPUs is a good heuristic.

## Creation

And to create the network, by way of example:

```bash
snakemake --cores all -- results/egypt-latest_filter-rail/edges.gpq
```
Expand Down
40 changes: 21 additions & 19 deletions docs/src/user-guide/usage/network-creation/road.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,32 +20,34 @@ sources (e.g. rehabilitation costs).
## Configuration

To specify a desired network:

- Review and amend the spreadsheets in `bundled_data/transport`, these supply
information that is used to gap-fill or extend what can be determined from OSM alone.
- Review and amend `config/config.yaml`:
- The `infrastructure_datasets` map should contain a key pointing to an `.osm.pbf`
file URL for desired area. There are currently entries for the planet,
for (some definition of) continents and several countries. We use
the [geofabrik](http://download.geofabrik.de/) service for continent and
country-level OSM extracts.
- Check the OSM filter file pointed to by `network_filters.road`.
This file specifies which [elements](https://wiki.openstreetmap.org/wiki/Elements)
(nodes, ways or relations) to keep (or reject) from the multitude of data
in an OSM file. See the filter expressions section
[here](https://docs.osmcode.org/osmium/latest/osmium-tags-filter.html)
for more information on the syntax of these files.
- Check and amend `keep_tags.road`. This list of strings specifies which
`tags` (attributes) to retain on the filtered elements we extract from
the `.osm.pbf` file.
- Review `slice_count`. This controls the degree of parallelism possible.
With it set to 1, there is no spatial slicing (we create the network in
a single chunk). To speed network creation for large domains, it can be
set to a larger square number. The first square number greater than your
number of available CPUs is a good heuristic.
- The `infrastructure_datasets` map should contain a key pointing to an `.osm.pbf`
file URL for desired area. There are currently entries for the planet,
for (some definition of) continents and several countries. We use
the [geofabrik](http://download.geofabrik.de/) service for continent and
country-level OSM extracts.
- Check the OSM filter file pointed to by `network_filters.road`.
This file specifies which [elements](https://wiki.openstreetmap.org/wiki/Elements)
(nodes, ways or relations) to keep (or reject) from the multitude of data
in an OSM file. See the filter expressions section
[here](https://docs.osmcode.org/osmium/latest/osmium-tags-filter.html)
for more information on the syntax of these files.
- Check and amend `keep_tags.road`. This list of strings specifies which
`tags` (attributes) to retain on the filtered elements we extract from
the `.osm.pbf` file.
- Review `slice_count`. This controls the degree of parallelism possible.
With it set to 1, there is no spatial slicing (we create the network in
a single chunk). To speed network creation for large domains, it can be
set to a larger square number. The first square number greater than your
number of available CPUs is a good heuristic.

## Creation

And to create the network, by way of example:

```bash
snakemake --cores all -- results/egypt-latest_filter-road-primary/edges.gpq
```
5 changes: 4 additions & 1 deletion docs/src/user-guide/usage/utilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
## Removing intermediate files

You can remove intermediate files by running the `clean` rule.

```bash
snakemake --cores 1 -R clean
```
Expand All @@ -26,6 +27,7 @@ This will write `.gpkg` files beside their source `.geoparquet`.

`matplotlib` plots can be interactive (zoom, pan, etc.), but not as static
images. Some rules produce pickled plot files. To view these, use:

```
python workflow/scripts/unpickle_plot.py <path_to_pickled_plot>
```
Expand All @@ -34,11 +36,12 @@ python workflow/scripts/unpickle_plot.py <path_to_pickled_plot>

The bash script `archive_results.sh` can be used to back up analysis results.
Here's an example usage:

```bash
./archive_results.sh results/ /mnt/backup/open-gira
```

This will create a timestamped folder in path of the second argument, e.g.
`/mnt/backup/open-gira/2023-07-24T101756+0100`, containing an archive of results
(excluding input files downloaded from the internet) and a README describing the
state of the repository at the time of archive creation.
state of the repository at the time of archive creation.

0 comments on commit 09365f5

Please sign in to comment.