Generic Map Plugin

[jansule]

Dear Superset community,

we are working on a visualization plugin for displaying arbitrary ECharts on a map. To do so, we are developing a generic approach that makes use of the existing EChart plugins in Superset and renders them on distinct geolocations on a map. Interactions on the ECharts themselves will be preserved.

See here a (very rough) example implementation of our concept:

Example Explore View

Example Dashboard Panel

We plan on implementing following main features:

• Generic chart configuration (all ECharts supported by Superset)
• Map configuration incl. support for specifying an arbitrary number of raster/vector layers

For our approach, we make use of the established OpenLayers library and base the layer configuration on the open standards of the Open Geospatial Consortium (OGC), i.e. WMS and WFS. This ensures stability and longevity.

At the moment, the plugin is being developed as a third-party plugin. However, we think this plugin exceeds our project specific use-case and has potential value for the broader community (as can be seen in #9281). Hence, we are evaluating the interest of becoming a core plugin of Superset.

Would it be of interest for you to add such a plugin as core plugin to Superset?

We are looking forward to getting feedback from you.

[jansule]

Is there any update from your side what you are thinking about it?

Please let us know, if you have any questions or if we shall explain some aspects in more detail.

[rusackas]

Hi @jansule sorry for the radio silence... this is getting a fair amount of upvotes, and I share their enthusiasm.

First off, I think this looks like a great plugin to add... I think a lot of people will find interesting use cases for it. If you're willing to develop it further, I think we'd be quite supportive! Particularly if it has good tests for making sure it stays stable, and hopefully is able to support recent features that the ECharts tend to have (e.g. drill to detail, etc).

There was also a thread (which I can dig up if you're interested) about augmenting or replacing the MapBox plugin with something less licensed/proprietary than MapBox (which requires an access token). I see you're well underway on that effort in this plugin, so I wonder if there's a clever way to tackle both efforts in one plugin. Or maybe that's asking too much... it just crossed my mind ;)

One curiosity: Is the geometry column effectively lat/long? I'm also curious if all the ECharts are rendered in a single Canvas element, or if there are any scalability/performance concerns that might come with this.

In general, I'd love to learn more and I, for one, would love to see such a PR!

[Lecaethomas]

Hi, for an easier implementation/conversion why wouldn't you use MapLibre.js? It sounds like you would not have to recreate your whole code as it is almost all the same as mapbox. (Maybe I arrive a bit late on the thread)

[rusackas]

It's definitely in consideration, but nobody has stepped up to volunteer the PRs to make the migration. We'd love to see such a contribution as part of 5.0 if you (or anyone reading this) is interested.

[jansule]

Hi @rusackas thanks for your reply! We are happy to hear that our idea might be interesting for you.

If you're willing to develop it further, I think we'd be quite supportive!

Thanks, that sounds great!

Since we created this discussion, we continued the development of the plugin and managed to create a (hopefully) usable prototype, which is definitely still rough around the edges.

In general, I'd love to learn more and I, for one, would love to see such a PR!

We will try to draft a PR in January, since there is still quite some work to do. To give you a brief heads up, see below an overview of the current features and some screenshots.

Regarding your questions:

hopefully is able to support recent features that the ECharts tend to have (e.g. drill to detail, etc)

We are currently using the <ECharts /> component of superset to render the charts, so basically all features supported by that component should also be supported by our plugin. Our plugin is basically a wrapper that takes a chart configuration and merges this configuration with data for each location.

There was also a thread (which I can dig up if you're interested) about augmenting or replacing the MapBox plugin with something less licensed/proprietary than MapBox (which requires an access token). I see you're well underway on that effort in this plugin, so I wonder if there's a clever way to tackle both efforts in one plugin

Our plugin supports the configuration of default layers that will automatically be included on every chart that will be created. Currently, this configuration takes place in MainPresets.js, so this can be configured for each superset instance, separately. Also, layers can be freely configured for each chart in the explore view. I'm not sure though, if this plugin might serve as a replacement for MapBox, but I'm sure it can at least be used as inspiration for an additional plugin.

Is the geometry column effectively lat/long?

Currently, the geometry column is a GeoJSON string in EPSG:3857, but with some effort this could be extended.

I'm also curious if all the ECharts are rendered in a single Canvas element, or if there are any scalability/performance concerns that might come with this

We are creating single canvas elements for each chart and place them on the map. We created a dedicated OpenLayers layer class for this. So far, rendering the charts is definitely our bottleneck. We tested with a dataset containing around 50000 rows distributed over around 50 different locations. There, initial rendering takes a few seconds (~ <10sec), afterwards everything runs smooth, though.

---

Features:

• Map extent configuration: either fit the whole dataset, or manual setting of extent
• Layer configuration: add, edit, move, remove layers
• Supported layer types: WMS (raster), WFS (vector), XYZ (raster)
• Styling of vector layers via GeoStyler
• Editable layer attributions
• Chart background configuration: background color, background opacity, border radius
• Scale dependent chart size configuration: automatic computation per zoom level based on function (fixed, linear, exponential) with manual adjustments

Screenshots:

Vector layer styling

Scale dependent chart size

[rusackas]

Wow, this is looking amazing!!! Than you for all the thorough answers, and now I'm excited for January more than ever! :)

CC @villebro @lauderbaugh @yousoph who are going to love this :)

[villebro]

Love this! Can we start a Slack channel for this to make collaboration easier?

[rusackas]

Good idea... @jansule I don't see you on Slack but if you're on there, please DM either Ville or Me to say hello, and we can start a channel!

[rumbin]

Sounds really exciting!
I wonder how this approach compares to the ones that focused on implementing kepler.gl:

• Integrate Kepler.gl as a new visualization type #6403
• Kepler.gl and Apache Superset 1.3.2 integration keplergl/kepler.gl#1656

@jansule could you share your thoughts on this, in terms of functionality and use cases?

[jansule]

Sure. I am not very familiar with kepler.gl but from a first glance the main differences seem to be the following:

• We focus on cartodiagrams - putting charts on the map. kepler.gl seems to cover other types of thematic maps (e.g. choropleth maps) and it looks like there is currently no support for cartodiagrams. I could be wrong, though. For our solution, we make use of the existing superset echarts plugins and do not implement our own charts.
• Map configuration will be done via superset controls and not within the plugin.

Cartodiagrams are generally useful for displaying multivariate data and thereby adds another possibility to the set of already supported thematic maps.

I can imagine that a similar approach could be implemented with kepler.gl (or deck.gl), but I don't know how much effort that would be. We are more familiar with OpenLayers, which definitely adds to the equation.

[jansule]

Another point is that by using OpenLayers, we support the official OGC standards WMS (raster layers) and WFS (vector layers). So we are not bound to a centralized service for serving map data. From a brief look at the documentation of kepler.gl, it looks like it uses mapbox under the hood.

[artemonsh]

Is there any progress on this? Would love to use it!

[villebro]

@artemonsh you can join the channel where this is being discussed: https://apache-superset.slack.com/archives/C04J3EHE1QA

[gannebamm]

Hi,
What exactly is meant by XYZ (raster)? Is WCS meant? Will the plugin enable charting of those raster values?

[jansule]

What exactly is meant by XYZ (raster)?

XYZ works for services were tiles can be requested like this: https://{a-c}.tile.thunderforest.com/cycle/{z}/{x}/{y}.png.

This is done by OpenStreetMap, google and others. See also: https://openlayers.org/en/latest/apidoc/module-ol_source_XYZ-XYZ.html

Is WCS meant?

No.

Will the plugin enable charting of those raster values?

Currently, the layers will only be used as base layers which will be placed below the charting layer. So they will not be used for charting. However, this might be subject of future change.

[jansule]

Finally, I managed to create a PR for this: #25869

[rusackas]

Sorry this PR is still open... we're still appreciative. Everything is competing for attention 😓

[villebro]

FYI @jansule and I are actively working on this (latest pushes to the branch have taken place today)

[rusackas]

Fantastic news! I've been feeling like a deadbeat on this issue.