Merge pull request #38 from buckhx/dev

Csv shapes

Author: buckhx

.gitignore

@@ -3,6 +3,7 @@
 /resources/*.go
 !/resources/stub.go
 *.sh
+*.dig
 
 # Compiled Object files, Static and Dynamic libs (Shared Objects)
 *.o
@@ -29,3 +30,9 @@ _testmain.go
 *.test
 *.prof
 *.swp
+*.pid
+*.out
+*.log
+*.csv
+*.txt
+*.gz

README.md

@@ -4,23 +4,48 @@
 
 A simple tool for solving common geospatial workflows.
 
-* wms: Web Mapping Server (tile server)
-* mbt: Tile builder for vector tiles into the mbtiles spec
+* [wms](#wms): Web Mapping Server (tile server)
+* [mbt](#mbt): Tile builder for vector tiles into the mbtiles spec
+* fence: A geofence utility
+* dig: A geocoder based on OSM data
+
+This project is under heavy development and APIs/CLIs are subject to change until further notice.
+
+## Installation
+
+Install diglet into /usr/local/bin (assuming you have permissions to) like so
+
+    curl -sSL https://raw.githubusercontent.com/buckhx/diglet/master/scripts/install.py | sudo python - /usr/local/bin
+    
+If you don't have permission at /usr/local/bin, try something like this where you extend your PATH
+
+    BINDIR=~/diglet
+    mkdir -p $BINDIR
+    export PATH=$PATH:$BINDIR
+    curl -sSL https://raw.githubusercontent.com/buckhx/denv/master/scripts/install.py | python - $BINDIR
+
+If you just want the binary, the final arg to the install py is a directory to download into
+
+Or do it manually by going to the releases page and download the diglet artifact https://github.com/buckhx/diglet/releases/latest
+
+Currently only building 64 bit linux for simplicity, but will build for more archs as things stabilize.
+Instructions for [building](#building)
 
 # wms
 
 A real-time tile server in a single binary
 
 Here are some neat things that diglet wms does
 
-* Backend changes are pushed to the front end in real time
-  * Currently changes are registered from the kernel (inotify/kqueue/ReadDirectoryChangesW)
-* Sniffs the tile format 
-  * (pbf, json, gz, jpg, png, etc...)
-* Source specific hooks in the works
-  * (on PostGIS insert -> build mbtiles)
+* Read-through mmap LRU cache
+* Easy peasy-lemon squeezy HTTPS
+* Sniffs the tile format and set Content-Type: (pbf, json, gz, jpg, png, etc...)
 * HTTP/JSON-RPC/WS endpoints
-* All packaged up in an itty-bitty binary
+
+Some things in the works
+
+* Backend changes are pushed to the front end in real time (via websockeys)
+* Source specific hooks (on PostGIS insert -> build mbtiles)
 
 ## Usage
 
@@ -70,9 +95,25 @@ Get information about the specific tileset. This information is populated from t
 Get the tile at the given coordinates and return the contents as the response body. 
 Passing json=true as a will return the tile as a json object with it's coordinates 
 
+###Gallery
+
+    GET /tileset/gallery/{tileset-slug}
+
+A simple gallery to view your tiles with. Only supports vector tiles for now. ?lat={}&lon={}&zoom{} will zoom to desired location.
+
 #mbt
 
-More info to come, but here's the current help message
+Will build tiles from either geojson or a csv.
+
+If a csv is used, either --csv-lat/--csv-lon or --csv shape must be set to read the coordinates correctly.
+The csv-shape is a list of list of [[],[],[]] and will only render a single, exterior ring polygon per line.
+Csv also requires a named header.
+
+Geojson is fair-game, no support for GeometryCollection or Topojson
+
+--filter if included will only include these columns in this properties. Includes all if not added
+
+Valid extentsions for mbtiles are .mbtiles or .mbt
 
 ```
 NAME:
@@ -92,15 +133,20 @@ OPTIONS:
    --layer-name "features"				Name of the layer for the features to be added to
    --desc, --description "Generated from Diglet"	Value inserted into the description entry of the mbtiles
    --extent "4096"					Extent of tiles to be built. Default is 4096
-   --max, --max-zoom "10"				Maximum zoom level to build tiles for. Not Implemented.
-   --min, --min-zoom "5"				Minimum zoom level to build tiles from. Not Implemented.
-   --filter 						Only include fields keys in this comma delimited list.	EXAMPLE --filter name,date,case_number,id	NOTE all fields are lowercased and non-word chars replaced with '_'
-   --csv-lat "latitude"					
-   --csv-lon "longitude"				
-   --csv-geometry "geometry"				Column containing geometry in geojson-like 'coordinates' form
+   --max, --max-zoom "10"				Maximum zoom level to build tiles for
+   --min, --min-zoom "5"				Minimum zoom level to build tiles from
+   --filter 						Only include fields keys in this comma delimited list
+   --csv-lat						Column containing latitude					
+   --csv-lon						Column containint longitude
+   --csv-shape						Column containing geometry in geojson-like 'coordinates' form
    --csv-delimiter ","	
 ```
 
+#fence & dig
+
+I did a geofencing experiment and may or may not include that in the standard build.
+Also did some geocoding work from OSM. Hit's ~500 r/s, but unstable so won't be included for now.
+
 ## Releases
 
 Diglet uses a go library for reading mbtiles that depends on some C code, this makes
@@ -114,6 +160,25 @@ We'll include a script and or directions for building on your own.
 
 #### Major releases will have platform specific binaries
 
+## Building
+
+Here are the basics, but you can inspect the .travis.yml for specifics
+
+Prereqs
+
+* [go installed](https://golang.org/doc/install#install), preferably 1.4+
+* gcc access
+* go get github.com/buckhx/diglet
+* cd into diglet 
+
+```
+CGO_ENABLED=1
+go test -v ./...
+go get ./...
+go generate
+go build -ldflags '-extldflags "-static"'
+```
+
 ## Some food for thought
 
 ##### A tile is a girls best friend
@@ -139,6 +204,9 @@ Here are dependencies and their licenese
 * buckhx/diglet: MIT
 * buckhx/mbtiles: MIT
 * codegangsta/cli: MIT
+* qedus/osmpbf: MIT
 * gorilla: BSD 3-Clause
 * mattn/go-sqlite3: MIT
 * go-std-lib: BSD 3-Clause
+
+I got lazy and haven't maintained the licenses...

burrow/routes.go

@@ -179,6 +179,18 @@ type RequestContext struct {
 	Params     MethodParams
 }
 
+func (ctx *RequestContext) Render(template string) (string, error) {
+	params := ctx.HTTPReader.URL.Query()
+	vars := make(map[string]string, len(ctx.Params)+len(params))
+	for k, v := range ctx.Params {
+		vars[k] = v.GetString()
+	}
+	for k, v := range params {
+		vars[k] = v[0] //only take first
+	}
+	return RenderTemplate(template, vars)
+}
+
 func (r *RouteHandler) MountRoutes(methods []Method) {
 	dict := make(map[string]Method)
 	for _, method := range methods {

burrow/util.go

@@ -3,11 +3,16 @@ package burrow
 import (
 	"encoding/binary"
 	"fmt"
+	"github.com/cbroglie/mustache"
 	"log"
 	"strconv"
 	"strings"
 )
 
+func RenderTemplate(template string, vars ...interface{}) (string, error) {
+	return mustache.Render(template, vars...)
+}
+
 func info(format string, vals ...interface{}) {
 	log.Printf("Burrow info: "+format, vals...)