Responsive image maps

[prushforth]

This issue may be better raised on the HTML spec itself, but I was hoping we could discuss a bit before that so that I don't get something too badly wrong out of ignorance.

I have been working on implementing MapML support in GeoServer, and it works quite nicely. I was able to include a <picture> element with a number of child <source> elements, allowing the initial view of the map to work whether or not script runs. In doing this, I am able to include a number of links to images that are generated by WMS GetMap requests only on demand (which one is decided by the user agent).

I noticed today that someone has long ago created a GeoServer plugin on demand, serializes a GeoServer GetMap request as a <map> element with a set of appropriate child <area> elements. The plugin html response doesn't generate the <img usemap="..."> element, only the <map> and <area>s, however, I presume so that the <map> html document can be server-side included in another HTML response.

So I was thinking about how to use this functionality in my <picture><source><img> usage above, and I realized that I would have to either specify a different <map> for each chosen image URL, or I would need to be able to transform the <area coords="..."> values to conform to the chosen image. All of this just can't be done declaratively, as far as I know, and I know that WICG members have asked for responsive image maps before, so was wondering how we might suggest to specify a declarative way to do that, and potentially add it to this repo/spec.

Currently, in the web-map custom element 'polyfill', child <area is="map-area"> elements are progressively enhanced by my code, and the coordinates are transformed into the map projection coordinate system (TCRS), which is nice, but it still doesn't satisfy the above (responsive image maps) non-progressive scenario.

Any discussion appreciated.

[prushforth]

I saw that someone over there said that <area coords="..."> should respond to CSS transform: scale(...), however I tested and could not reproduce that. The spec has a note that says:

Browser zoom features and transforms applied using CSS or SVG do not affect the coordinates.

So perhaps transform is a dead end after all, but I think that would solve some of the issues for using maps with media queries and the picture element.

[Malvoz]

Oddly, it seems that percentages (which I believe would solve this) were once valid for coordinate values, MDN says:

In HTML4, the values are numbers of pixels or percentages, if a percent sign (%) is appended; in HTML5, the values are numbers of CSS pixels.

If that is accurate, I wonder what the reasons were for removing it?

wondering how we might suggest to specify a declarative way to do that, and potentially add it to this repo/spec.

In whatwg/html#389 which you've linked to, there already are suggestions to solve it, but (at the time) a lack of implementor interest.

I suppose there'd have to be implementor interest in the MapML/HTML Map Element proposals first before they can count as motivators for changes to the coords attribute, but I don't think that should stop you from expressing your interest in that issue.

So perhaps transform is a dead end

Just for me to understand - if that had worked, what would the intention going forward be? To spec that browsers must apply transforms to area coords within MapML documents? Or were you just looking into it for perhaps demonstrational purposes? Perhaps there are other CSS hacks to look into then?

[prushforth]

Or were you just looking into it for perhaps demonstrational purposes?

I was hoping to provide a non-scripted responsive fallback for areas in my GeoServer plugin, which serves the web-map polyfill in its HTML Layer Preview response (I am working on a video demonstrating this plugin now, together with the fallback. I'll post a link here when I'm done. It's kind of hard to explain without pictures). There is currently a long-standing plugin for GeoServer which generates a <map> and child <area> elements for a given GetMap request (where the response format requested is text/html), including pixel-based coordinates for features (<area coords>), so I was hoping to use that functionality to transmit <area> elements that represent geographic features, as a responsive fallback

I'm trying to build the case / evidence that today's <map>, <area> and associated <img> elements, as already implemented by browsers, provide the perfect processing semantics for future enhancement.

To spec that browsers must apply transforms to area coords within MapML documents?

No, I hope to not change or make any proposal to change the <area> element, despite some allusion to doing so in this spec. I would prefer to leave <area> alone if possible. OTOH I think the <feature> element as specified is appropriate in MapML. In the map-area polyfill element, I transform <area is="map-area"> elements into layers, each with a single <feature> in it, IIRC.

Perhaps there are other CSS hacks to look into then?

Did you have something in mind? I would hack it if I could (without scripting it).

[Malvoz]

Did you have something in mind?

Using CSS, the only thing I can think of is to override the coords attribute to absolutely position <area> elements with percentage offsets i.e. left: 10%; top: 20% (which should act as coords if percentage values were allowed). You could then set the height/width of the area element and use clip-path to set shapes.

This would allow for changing the <area>'s position, shape and other styles based on media query conditions. All the styles for responsive maps could go inside a @supports() statement and thus legacy browsers will ignore those styles and use the coords attribute relative to a then unresponsive map.

Just an idea. Which might have to get a lot more elaborate than this, and since it's a hack for <area> it probably wont work in all browsers.

[prushforth]

Using CSS, the only thing I can think of is to override the coords attribute to absolutely position elements with percentage offsets i.e. left: 10%; top: 20% (which should act as coords if percentage values were allowed). You could then set the height/width of the area element and use clip-path to set shapes.

That is very creative! Although not the first time I've heard of clip-path, it is definitely a unique suggestion for its application, to me. I will tinker with it if I can find some time.

responsive-test.html.txt

[Malvoz]

I set up a pretty rough demo from tinkering with it myself, I placed the <area> elements over the icons (I did not make them span the text, which they probably should) and thus sort of mimicking what coordinates with % values could mean. It only works in Chrome it seems (indicating that they're deviating from the spec processing model?), and I didn't modify the coords attribute values but overall I think it's close to what you want it to do, see it here: https://malvoz.github.io/responsive-area-coords/

[prushforth]

@Malvoz looks interesting, and most importantly, functional. I don't understand it all yet, but from what I see, we would need to generate a top/left % scale for each area for each possible img src. Could get heavy for large sets of <area>s (unfortunately quite typical of geographic feature data), but possibly a motivating demo of the need for support for CSS transform of coords.

I will keep looking at it to understand it better. Thank you for having the interest to do that!

[Malvoz]

Oh and if you haven't noticed already, I had left a few explanatory comments in the HTML file for the essential properties that might clarify some of the things.

[prushforth]

@Malvoz, I did notice, and thanks you were right I needed them :-)

[prushforth]

@AmeliaBR this is the issue I mentioned that @Malvoz has been helping me with. Can we discuss here and figure out a recommendation / solution before pitching that over in the WHATWG issue?

I was thinking specifically about how to apply a css transform to the area coords would allow us to include "source" coordinates from the GeoServer feature and supply a transform inside a media query that matched the source@media value.

Maybe I'll let @Malvoz's code explain itself, but he got it to work with CSS shapes, I think. I couldn't get my example to transform, but I think that is per spec, and I'm not quite sure why.

There is an existing GeoServer plugin that will serialize geospatial features as <area> elements, and if we could do that once and supply n transforms for n <source@media> elements I think it would be pretty cool. I could make it work as a fallback with the GeoServer MapML Community Module as a responsive fallback. I'm sure I haven't explained what I'm interested in doing properly, but just have a look and tell us what you think. :-)

[Malvoz]

@prushforth I've made a small update to the demo page, and included a list of issues with my approach. The HTML/CSS has been simplified to its bare minimum, I think it should be more readable now.

[prushforth]

Thanks!