Puck lag mitigation

[ianthetechie]

As things stand today, the puck can lag a bit behind the user's current location. We've added a little dot that updates in real time with the user's location in both demo apps as a visualization of this (and to enable debugging of route snapping vs GPS drift). Here is a brain dump of what I know so far.

Underlying problems

A few underlying problems contribute to the observed lag.

(In)frequency of location updates

Most mobile devices will deliver a location update approximately once per second. This is not fast enough to create a smooth motion without some sort of additional work.

Camera animation

The MapLibre camera mode (track user with bearing) that draws the puck attempts to address this infrequency of updates with an animation. This sounds like a great idea in theory, but it means the user's location may be up to 1 second ahead of the puck. Or at least, that's what the intent of the MapLibre code is, as I understand it.

There are some bugs in this animation code though. I tried looking at it myself about 6 months ago, but wasn't able to find a glaringly obvious issue.

Fast updates cause problems

On iOS in particular (I forget if Android exhibits this), rapid updates to the user's location actually make the problem significantly worse. I believe this is because it constantly interrupts the animation with a new one, and given how easing etc. work, it actually makes it harder to catch up. If you deliver updates twice/second, for example, you would have a persistent lag that's even longer. And if your update timing is variable, the lag will not be consistent either.

Root cause 1: Since updates are typically not delivered at a frequency of > 1Hz, the user's location will lag behind by definition.

Root cause 2: in my analysis, something is subtly wrong with how MapLibre animates its camera, preventing delivery of sub-second location estimates (ex: Kalman filters, external vehicle sensors, etc.)

(Related) Inaccuracy of location updates

It's pretty regular that the device confidently reports very wrong GPS data. I think that some amount of filtering is going to be required by many applications. iOS is interesting in particular, because the data you get is already smoothed in some way. Android has a mediocre fused location client, and an extremely rough location API, but at least it does let you get more "raw" data which is more amenable to building corrective layers on top of.

This isn't a direct cause of puck lag normally, but is a related challenge that will need to be considered.

Approach to getting better tracking

I think that everyone agrees that a critical component of the solution has to be estimating the user's location based on sensor inputs. For example, via a Kalman filter, which can deliver an estimate of the user's location at a higher frequency. I tihnk the implementation will almost necessarily be somewhat specific to the device type and expected mode of travel, but we can certainly implement some of this in Ferrostar's core (see #23). This can me implemented in a LocationProvider; I expect we can probably also implement config parameters to enable this on the bundled "system-backed" versions.

On the MapLibre side, what we ultimately need is an API that allows us to do 60Hz (or similar) location updates. This could perhaps be accomplished by fixing the flaws in the animated approach and making movement linear. But some engineers I talked to from Meta are of the opinion that the ultimate solution is to skip animation and just deliver estimated positions at 60Hz from the location provider. They suggested that a custom layer may be the best approach for this. I do not know if both iOS and Android support this, but this is, in my opinion, the most promising approach. It would also solve the problem on iOS of the puck being a fixed position UIView which is almost trivial to knock off course.

[ahmedre]

Thanks Ian - One thing that Ferrostar already does that took me quite a while to figure out - setting the fastestInterval on the LocationEngineRequest to 0 is necessary to avoid dropping location events that are emitted by the engine too quickly. This improves the gap between the observed and actual drastically, though a gap still exists as is seen in Ferrostar's demo app, especially when zooming in.

Sharing some of my own notes from looking into this today:

I looked more into the animation on Android - whenever a new value comes in, it .cancel()s the AnimatorSet - unlike .end(), this actually "stops the animation where it is." The code then takes the "current value" and the "just emitted current location value" and starts a new (750ms) animation to go from this one to that one.

I then found the trackingAnimationDurationMultiplier flag, but sadly, even setting that to 0 still has a PacMan type effect of the actual location puck being behind what is drawn on Ferrostar (though it does disable the animation). One interesting thing when doing this - the delta between when LocationComponent receives the updated location and when SymbolLocationLayerRenderer receives the updated location is only ~100ms. This suggests that the corresponding delta is in the time it takes to apply the updated location in setGeoJson, though I could be missing something.

Consequently, I tried the specialized location layer (using .useSpecializedLocationLayer(true) to the LocationComponentActivationOptions builder call), but didn't seem to notice much of a difference with this either unfortunately.

[ianthetechie]

I looked more into the animation on Android - whenever a new value comes in, it .cancel()s the AnimatorSet - unlike .end(), this actually "stops the animation where it is." The code then takes the "current value" and the "just emitted current location value" and starts a new (750ms) animation to go from this one to that one.

Ha! I expected something like that, but never dug deep enough to find the smoking gun... Guess that's it ;)

One interesting thing when doing this - the delta between when LocationComponent receives the updated location and when SymbolLocationLayerRenderer receives the updated location is only ~100ms. This suggests that the corresponding delta is in the time it takes to apply the updated location in setGeoJson, though I could be missing something.

Huh.... Can you add some links to these in the source tree? Maybe if we can get enough of the pieces together we could ping Bart to get feedback on next steps/possible mitigations.

Thanks Ian - One thing that Ferrostar already does that took me quite a while to figure out - setting the fastestInterval on the LocationEngineRequest to 0 is necessary to avoid dropping location events that are emitted by the engine too quickly. This improves the gap between the observed and actual drastically, though a gap still exists as is seen in Ferrostar's demo app, especially when zooming in.

Sharing some of my own notes from looking into this today:

I looked more into the animation on Android - whenever a new value comes in, it .cancel()s the AnimatorSet - unlike .end(), this actually "stops the animation where it is." The code then takes the "current value" and the "just emitted current location value" and starts a new (750ms) animation to go from this one to that one.

I then found the trackingAnimationDurationMultiplier flag, but sadly, even setting that to 0 still has a PacMan type effect of the actual location puck being behind what is drawn on Ferrostar (though it does disable the animation). One interesting thing when doing this - the delta between when LocationComponent receives the updated location and when SymbolLocationLayerRenderer receives the updated location is only ~100ms. This suggests that the corresponding delta is in the time it takes to apply the updated location in setGeoJson, though I could be missing something.

Consequently, I tried the specialized location layer (using .useSpecializedLocationLayer(true) to the LocationComponentActivationOptions builder call), but didn't seem to notice much of a difference with this either unfortunately.

Interesting. I didn't even know this flag existed TBH! I wonder what it does differently. @Archdoog and I noted that the Android location puck implementation is pretty different from iOS (which uses a statically positioned UIVIew, which has its own set of issues), but looking this up it sounds like there are actually two modes within Android. The docs don't really say what it does though besides being faster 🤔