There's been a lot of discussion about using adjacency graphs, but probably the most sensible approach is to create some kind of vector representation of a segment that is a function if its adjacency and/or features.

Initial milestone here - just incorporate something like node2vec into the pipeline and train it on any individual city's graph.