Real-time speed

[laurentg]

Unless I'm mistaken, there is no "real-time current speed" information in the format. Shouldn't we have this information on the format too?

We could also think on a predicted speed(s), if available.

[kpwebb]

Ok! I've updated the docs and proto buff message spec to handle both baseline and current conditions. Curious to hear your thoughts.

[laurentg]

On the BaselineStats message: are the hourOfWeek... fields a list of speeds for each 1h bin?

Why not having a dedicated message for each bin, such as this:

message SpeedQuartiles {
    required float averageSpeed = 1;
    optional float topQuartileSpeed = 2;
    optional float bottomQuartileSpeed = 3;
}
message BaselineStats {
    required SegmentDefinition segment = 1;
    required float averageSpeed = 2;
    repeated BinStats hourOfWeek = 3;
}

Just a small note: if we have both topQuartile and bottomQuartile, should we not have the averageSpeed be a medianSpeed for consistency (aka middleQuartile) ? Mathematically, you can have the topQuartile lower than the average (although you would need a few spurrious rockets in the samples). And for a routing point of view, basing computations on the median speed could make more sense than the average (although both would be rather similar anyway, the median being probably a bit lower), but I'm not an expert on this.

[laurentg]

About current condition, should the currentWindowStartTimestamp/ currentWindowEndTimestamp should be marked as "required" ? There could be an option to skip them in the request, as for some usage they could be unnecessary (thinking of route planners again).

Also, providing both start/end 64 bits timestamps require lots of bandwith, maybe only a start plus offset; and/or a 32 bits seconds timestamp, or even shorter, two offsets to a global message "current" timestamp.

[abyrd]

Agreed that if we are including quartiles, we should include the median rather than the mean.

I see the column-store format (arrays containing all measurements for all hours of the week) as mainly a space optimization. If we're not aiming for maximum compactness then it is indeed clearer to have one "struct" (compound protobuf message) per hour, with all the quantile fields present in each of those messages.

Here we have a dichotomy between historical data over all hours of the week, versus current data for only the segments where we have observations. This implies that data consumers will always handle all prediction and interpolation, which is unlikely, so there does need to be a way to express full-dataset predicted results where the inference/interpolation is done by the producer.

From what I've read "required" fields in Protobuf are essentially deprecated as they have been found to cause unexpected fragility, so I'd be in favor of just avoiding required fields.