feat: generate h3 polygons and create webmap #46

ioalexei · 2024-05-17T10:37:57Z

Addresses issue #25

Adds files:

points_to_h3.py
create_webmap.py
templates/maplibre_template.html

Points to H3

Input:

Input file: point layer with GVI scores
Output file
(Optional) H3 cell resolution (see here)

Output:

File containing h3 polygons at the specified resolution with mean GVI score as an attribute

Example usage:

python3 -m src.points_to_h3 data/processed/gvi_score_layer.gpkg data/processed/h3_polygons.gpkg 11

Create webmap

Input:

Input file: H3 polygon file
Output: HTML file to display MapLibre webmap
(Optional) filename for HTML file
(Optional) default zoom level for webmap

Output:

HTML file that contains javascript to generate a webmap showing the H3 polygons, styled by mean GVI score

Example usage:

python3 -m src.create_webmap ./data/processed/h3_polygons.gpkg ./outputs gvi_webmap.html 10

This reverts commit 9b33217.

- add h3pandas to dependencies - add app launch to end of file - use single aggregation (mean) instead of multiple - change order of arguments based on defaults

- adding Typer help and defaults - adding code to generate colours and breakpoints for map - formatting

- fix interpolation breaks - fix hard coded template path

Hard code colours into dataframe/json rather than using MapLibre interpolation

remove temp file

- add pop to show gvi score when polygon is clicked

- Round GVI scores to 2 decimal places for neater pop-ups

- Reduce bounding box buffer from 0.5 degrees to 0.25 - Set default zoom to 12

Update sample .env file to include placeholder for Maptiler api key

Fix line lengths

Formatting

Points to hex to webmap

…view into main

danbjoseph · 2024-06-17T22:04:14Z

When I generate the web map and then open the file in a browser the map is blank and there is an error is in the console. is it possible to have it generate the overlay but just not show the basemap if there's no API key? can you add details to the Readme about the API key (where to get it, where to put it)?
hex grids look good, although trying to generate with resolution 9 results in an error

add info on MapTiler API key

ioalexei · 2024-06-24T15:40:15Z

Added a note to the ReadMe re. MapTiler API key - will look into the other comments and come back to you

Add a leading `0` to cell resolution if it is a single digit

The Jinja template will use an empty basemap if there is no Maptiler API key in the .env file

ioalexei · 2024-07-08T10:33:23Z

Single digit cell resolution error fixed with 2d039c9
Webmap rendering in absence of Maptiler API key fixed with 592679f

Note: ruff flags h3pandas as unused but it's used in line 79 of the hex creation file ( gdf.h3.geo_to_h3())

danbjoseph · 2024-08-12T18:53:02Z

when running:

python -m src.create_webmap data/processed/ThreeRivers/Three_Rivers_h3_polygons_10.gpkg data/processed/ThreeRivers/Three_River_gvi_webmap.html 10

i get the below 2 UserWarning (at the start) and a FileNotFoundError (see at the end where the error log shows it replaced the space with a slash and merges the 10 into the file path). it works if I remove the default zoom argument and for the output provide only a directory path (with no filename).

(.venv) ➜  street-view-green-view git:(pr/46) python -m src.create_webmap data/processed/ThreeRivers/Three_Rivers_h3_polygons_10.gpkg data/processed/ThreeRivers/Three_River_gvi_webmap.html 10
/Users/danbjoseph/GitHub/americanredcross/street-view-green-view/src/create_webmap.py:59: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.

  str(gdf.dissolve().centroid.x.values[0])
/Users/danbjoseph/GitHub/americanredcross/street-view-green-view/src/create_webmap.py:61: UserWarning: Geometry is in a geographic CRS. Results from 'centroid' are likely incorrect. Use 'GeoSeries.to_crs()' to re-project geometries to a projected CRS before this operation.

  + str(gdf.to_crs("4326").dissolve().centroid.y.values[0])
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/danbjoseph/GitHub/americanredcross/street-view-green-view/src/create_webmap.py:135 in     │
│ main                                                                                             │
│                                                                                                  │
│   132 │   template = environment.get_template("maplibre_template.html")                          │
│   133 │                                                                                          │
│   134 │   # Generate the HTML file from the template, filling dynamic values                     │
│ ❱ 135 │   with open(                                                                             │
│   136 │   │   os.path.join(output_dir, filename), mode="w", encoding="utf-8"                     │
│   137 │   ) as message:                                                                          │
│   138 │   │   message.write(                                                                     │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │             bbox = <POLYGON ((-85.355 41.675, -85.907 41.675, -85.907 42.215, -85.355        │ │
│ │                    42.215, -...>                                                             │ │
│ │           bounds = (                                                                         │ │
│ │                    │   -85.90720454077227,                                                   │ │
│ │                    │   41.67471472908509,                                                    │ │
│ │                    │   -85.35469609820089,                                                   │ │
│ │                    │   42.215034264698346                                                    │ │
│ │                    )                                                                         │ │
│ │       bounds_str = '[-85.90720454077227,41.67471472908509],                                  │ │
│ │                    [-85.35469609820089,42.215034264698346]'                                  │ │
│ │           centre = '-85.63005910941862, 41.940565538549514'                                  │ │
│ │       centre_str = '[-85.63005910941862, 41.940565538549514]'                                │ │
│ │             cmap = <matplotlib.colors.ListedColormap object at 0x1627dbe60>                  │ │
│ │      environment = <jinja2.environment.Environment object at 0x164408bc0>                    │ │
│ │         filename = '10'                                                                      │ │
│ │              gdf = │   │   │      h3_10  gvi_score                                           │ │
│ │                    geometry  gvi_norm html_color                                             │ │
│ │                    0    8a274b8c2437fff      15.65  POLYGON ((-85.64586 41.92636, -85.64673  │ │
│ │                    41.926...  0.306524    #34618d                                            │ │
│ │                    1    8a274b8c2507fff      14.97  POLYGON ((-85.64385 41.92817, -85.64473  │ │
│ │                    41.927...  0.291979    #365d8d                                            │ │
│ │                    2    8a274b8c251ffff      16.66  POLYGON ((-85.64485 41.92727, -85.64573  │ │
│ │                    41.927...  0.328128    #31678e                                            │ │
│ │                    3    8a274b8c2587fff       7.17  POLYGON ((-85.64812 41.92762, -85.649    │ │
│ │                    41.92738...  0.125134    #472d7b                                          │ │
│ │                    4    8a274b8c258ffff       7.84  POLYGON ((-85.64749 41.92654, -85.64837  │ │
│ │                    41.926...  0.139465    #46307e                                            │ │
│ │                    ..               ...        ...                                           │ │
│ │                    ...       ...        ...                                                  │ │
│ │                    282  8a274b8f1b9ffff       6.40  POLYGON ((-85.60709 41.96486, -85.60797  │ │
│ │                    41.964...  0.108663    #482677                                            │ │
│ │                    283  8a274b8f332ffff       4.26  POLYGON ((-85.6371 41.95591, -85.63798   │ │
│ │                    41.9556...  0.062888    #48186a                                           │ │
│ │                    284  8a274b8f3377fff       9.37  POLYGON ((-85.63647 41.95483, -85.63734  │ │
│ │                    41.954...  0.172193    #443b84                                            │ │
│ │                    285  8a274b8f3acffff       8.70  POLYGON ((-85.63383 41.95556, -85.63471  │ │
│ │                    41.955...  0.157861    #453781                                            │ │
│ │                    286  8a274b8f3adffff      10.11  POLYGON ((-85.63546 41.95573, -85.63634  │ │
│ │                    41.955...  0.188021    #424086                                            │ │
│ │                                                                                              │ │
│ │                    [287 rows x 5 columns]                                                    │ │
│ │       gdf_bounds = array([-85.65720454,  41.92471473, -85.6046961 ,  41.96503426])           │ │
│ │                i = 1.0                                                                       │ │
│ │       input_file = PosixPath('data/processed/ThreeRivers/Three_Rivers_h3_polygons_10.gpkg')  │ │
│ │   legend_colours = ['#440154', '#31688e', '#35b779', '#fde725']                              │ │
│ │       legend_gvi = [1, 13, 25, 37]                                                           │ │
│ │  legend_gvi_norm = array([0.        , 0.33333333, 0.66666667, 1.        ])                   │ │
│ │   legend_label_1 = 1                                                                         │ │
│ │   legend_label_2 = 17                                                                        │ │
│ │   legend_label_3 = 32                                                                        │ │
│ │   legend_label_4 = 48                                                                        │ │
│ │   legend_patch_1 = '#440154'                                                                 │ │
│ │   legend_patch_2 = '#31688e'                                                                 │ │
│ │   legend_patch_3 = '#35b779'                                                                 │ │
│ │   legend_patch_4 = '#fde725'                                                                 │ │
│ │ maptiler_api_key = None                                                                      │ │
│ │       output_dir = PosixPath('data/processed/ThreeRivers/Three_River_gvi_webmap.html')       │ │
│ │         template = <Template 'maplibre_template.html'>                                       │ │
│ │       zoom_level = 12                                                                        │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
FileNotFoundError: [Errno 2] No such file or directory: 'data/processed/ThreeRivers/Three_River_gvi_webmap.html/10'

danbjoseph · 2024-08-12T19:10:29Z

also, i added a MapTiler API key to the .env file but it doesn't seem to be reading it in.

ioalexei · 2024-08-13T09:54:27Z

2e3586c fixes the Maptiler API issue by adding load_dotenv() to the create_webmap script and correcting an error in the HTML template

the error above is because the function expects zoom level to be the 4th of 4 arguments (input data, output folder, output filename, zoom level) - because only 3 were passed, it interpreted zoom level as the file name. I've made a mistake in the example in the documentation.

Is it better to not ask for a separate folder and filename and just take a path to a file for the output? Or should I just update the readme to give a code sampe that will work with this pattern

danbjoseph · 2024-08-13T12:37:33Z

yeah, i think a path to the file (instead of separate path and filename) matches the other steps?

ioalexei · 2024-08-13T16:16:35Z

Updated with ioalexei@b42316d

danbjoseph · 2024-08-29T00:12:32Z

this is looking good, except for it to render the hex layer when there isn't a MapTiler API key, it seems that the .env file has to have...

MAPTILER_API_KEY = "None"

if the user copies the template and doesn't touch the line then it will be MAPTILER_API_KEY = "MY_MAPTILER_API_KEY" and it doesn't work? or if they delete the line and the MAPTILER_API_KEY key isn't present, it doesn't work?

danbjoseph · 2024-09-12T00:25:40Z

if there is no MAPTILER_API_KEY line it works (shows the hex grid without a base map)

MAPTILER_API_KEY = "MAPTILER_API_KEY"  # doesn't work 
MAPTILER_API_KEY = ""  # doesn't work

i guess we could just put a commented note in the .env.example file to remove the line if not used?

ioalexei · 2024-09-12T09:09:08Z

Added a new commit to account for a blank variable - but think it makes sense to put a note in the example .env file too as it might be hard to account of all the ways it could be invalid

danbjoseph · 2024-09-16T18:13:56Z

functionality seems good to me for a merge
as @ioalexei notes, ruff flags h3pandas as unused but it's used in line 79 of the hex creation file ( gdf.h3.geo_to_h3()) - is there an edit that will let the check pass?

ioalexei and others added 21 commits May 16, 2024 13:37

Create points_to_h3.py

9b33217

Revert "Create points_to_h3.py"

a0b7166

This reverts commit 9b33217.

Create points_to_h3.py

9350375

Updates to points_to_h3

3a90717

- add h3pandas to dependencies - add app launch to end of file - use single aggregation (mean) instead of multiple - change order of arguments based on defaults

Add webmap code and template

4621eba

Tidying and formatting webmap script

c983a7f

- adding Typer help and defaults - adding code to generate colours and breakpoints for map - formatting

Fix interpolation breaks

3947b57

- fix interpolation breaks - fix hard coded template path

Fix code to generate polygon colours

dab6ad4

Hard code colours into dataframe/json rather than using MapLibre interpolation

Delete gvi_webmap.html

9508e16

remove temp file

Applying ruff checks

e517e9f

Update maplibre_template.html

be3e141

- add pop to show gvi score when polygon is clicked

Update create_webmap.py

35afb61

- Round GVI scores to 2 decimal places for neater pop-ups

Generate legend from GVI scores

b2bbf6a

Update create_webmap.py

a7fcaa6

- Reduce bounding box buffer from 0.5 degrees to 0.25 - Set default zoom to 12

Update .env.example

e4d864e

Update sample .env file to include placeholder for Maptiler api key

Update create_webmap.py

98fdc08

Fix line lengths

Update points_to_h3.py

05c662e

Formatting

Merge pull request #1 from ioalexei/points-to-hex

79755ef

Points to hex to webmap

Merge branch 'main' of https://github.com/ioalexei/street-view-green-…

f07b66d

…view into main

visualize step described in readme

e2da466

remove reference to folder not in repo

3f75210

danbjoseph and others added 3 commits June 17, 2024 18:07

alphabetize new dependency

d93dc6b

Merge branch 'main' into main

543d150

Update README.md

86aa021

add info on MapTiler API key

ioalexei added 4 commits July 8, 2024 10:59

Fix single digit resolution error

2d039c9

Add a leading `0` to cell resolution if it is a single digit

Use empy basemap if no Maptiler key provided

592679f

The Jinja template will use an empty basemap if there is no Maptiler API key in the .env file

Merge branch 'points-to-hex-to-webmap' into main

889600d

Formatting

0d556f5

fix env variable and update template

2e3586c

Combine output path and filename

b42316d

danbjoseph changed the title ~~Generate h3 polygons and create webmap~~ feat: generate h3 polygons and create webmap Aug 26, 2024

docs: filename and path is one argument

baaa5ef

add logiv for missing env variable

ce00146

fix for blank env variable

5cb7516

danbjoseph added 3 commits September 13, 2024 10:37

Merge branch 'main' into main

64dc3c8

fix: ruff format

4b5aec8

fix: ruff fix import block is un-sorted or un-formatted

10c9347

danbjoseph requested review from jayqi and dragonejt September 16, 2024 18:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: generate h3 polygons and create webmap #46

feat: generate h3 polygons and create webmap #46

ioalexei commented May 17, 2024 •

edited

Loading

danbjoseph commented Jun 17, 2024

ioalexei commented Jun 24, 2024

ioalexei commented Jul 8, 2024 •

edited

Loading

danbjoseph commented Aug 12, 2024 •

edited

Loading

danbjoseph commented Aug 12, 2024

ioalexei commented Aug 13, 2024

danbjoseph commented Aug 13, 2024 •

edited

Loading

ioalexei commented Aug 13, 2024

danbjoseph commented Aug 29, 2024

danbjoseph commented Sep 12, 2024

ioalexei commented Sep 12, 2024

danbjoseph commented Sep 16, 2024

feat: generate h3 polygons and create webmap #46

Are you sure you want to change the base?

feat: generate h3 polygons and create webmap #46

Conversation

ioalexei commented May 17, 2024 • edited Loading

Points to H3

Create webmap

danbjoseph commented Jun 17, 2024

ioalexei commented Jun 24, 2024

ioalexei commented Jul 8, 2024 • edited Loading

danbjoseph commented Aug 12, 2024 • edited Loading

danbjoseph commented Aug 12, 2024

ioalexei commented Aug 13, 2024

danbjoseph commented Aug 13, 2024 • edited Loading

ioalexei commented Aug 13, 2024

danbjoseph commented Aug 29, 2024

danbjoseph commented Sep 12, 2024

ioalexei commented Sep 12, 2024

danbjoseph commented Sep 16, 2024

ioalexei commented May 17, 2024 •

edited

Loading

ioalexei commented Jul 8, 2024 •

edited

Loading

danbjoseph commented Aug 12, 2024 •

edited

Loading

danbjoseph commented Aug 13, 2024 •

edited

Loading