California precincts in TopoJSON format
When developing geotuple.org, I was looking for detailed voting district (precinct) boundaries and came across this repository from the Los Angeles Times Data Viz team. They had already done an excellent job of organizing all the information (published here), but I thought I could tidy up the geographic data just a bit for my needs.
The original shapefile (not provided in this repo) is 95M while the resulting TopoJSON comes in at 10M.
$ du -hc merged.*
1.3M merged.dbf
4.0K merged.prj
93M merged.shp
204K merged.shx
95M total
$ du -h ca*
10M ca_precincts_topo.json
However, as the resulting file has been simplified, a more equitable comparison is a reduction of about 50%.
The main objective is to simplify the polygon geometry, by reducing #coordinates, and to take advantage of the topology encoding provided by TopoJSON. Secondly, to load the data into a PostGIS db.
I used the following tools:
This inital step combines the original county shapefiles (not provided in this repo) into one:
$ for f in shapefiles/*.shp; do ogr2ogr -update -append merged.shp $f -f "ESRI Shapefile"; done;
First of all, we want to project to a coordinate system based on a consistent unit (as opposed to decimal degrees). For California, an appropriate equal-area projection is "California Albers" (EPSG=3310) using meters:
$ ogr2ogr -f "ESRI Shapefile" merged2.shp -t_srs EPSG:3310 merged.shp
In order for TopoJSON to do it's "thing", we need to have coincident polygon boundaries. This is not always the case, as seen in this example between the Fresno and Madera counties along the San Joaqin River:
Overcoming this issue involves a fairly length process, centering on the ArcGIS Eliminate command (the specific commands used are detailed in the arcgis/eliminate_process.xml file).
This collapses the gap (sliver) as shown here:
The resulting shapefile can be found in arcgis/arc_elim_shapefile.zip.
First convert the output of the ArcGIS process to GeoJSON:
$ ogr2ogr -f "GeoJSON" merged2_geo.json -select "orig_fid pct16" arc_elim.shp
Finally, create the long awaited TopoJSON, while in the process further simplifying the geometry:
$ geo2topo merged2_geo.json > merged2_topo.json
$ toposimplify -p 1 -f merged2_topo.json > merged3_topo.json
$ topoquantize 1e6 merged3_topo.json > ca_precincts_topo.json
For reference, see: Command-Line Cartography, Part 3
The first step is to extract the relevant precinct voting information from the original csv file (not provided in this repo):
$ head -24569 final-results/all_precinct_results.csv > all_precinct_results2.csv
We can then import this file into the table precinct_results by executing the SQL script import_precincts.sql:
# \i postgis/import_precincts.sql
# SELECT COUNT(*) FROM precinct_results;
count
-------
24568
(1 row)
Secondly, we import the precinct boundaries into the table precinct using OGR (optionally projecting back to WGS84):
$ ogr2ogr -f PostgreSQL PG:"dbname='YOUR_NAME' user='YOUR_USER' password='YOUR_PW'" -nln precinct -select "pct16" -lco GEOM_TYPE="geography" -lco GEOMETRY_NAME=geog -s_srs EPSG:3310 -t_srs "EPSG:4326" ca_precincts_topo.json
We can now access the voting data spatially via a join:
# SELECT b.pres_trump_per FROM precinct a INNER JOIN precinct_results b ON a.pct16=b.pct16 WHERE a.pct16 = '037-7300001A';
pres_trump_per
----------------
100
(1 row)
- Attempting to convert the unsimplified shapefile to TopoJSON failed:
$ ogr2ogr -f "GeoJSON" merged.geojson -t_srs EPSG:3310 merged.shp
$ geo2topo pct16=merged.geojson > merged.topojson
buffer.js:495
throw new Error('"toString()" failed');
...
Seemingly because of hitting a limitation in nodejs (presumably due to too many coordinates).
Simplifying the input fixes the problem:
$ ogr2ogr -f "GeoJSON" merged.geojson -simplify 5 -t_srs EPSG:3310 merged2.shp