Path matching, data extraction, templating.

[nyarly]

This is perhaps a very large topic, but I'm pulling it from a number of chats-in-passing. Most recently, I noticed #496 touching on it.

Generally speaking (and going back to basic design here), we want to group up many resources under path matchers to be processed by a single Handler, and with the varying parts delivered in a well typed parameter. @msrd0 was proposing that matching be handled by a Gotham derive rather than a Serde Deserialization.

On the one hand, I'd like to be convinced of the value there. Serde's a fine piece of engineering, and we do well leveraging it for parameter (and query) deserialization.

On the other, if we're contemplating changing the behaviors of routing, I'd like to revive one of the features I'd like to see: bidirectional routing. HTTP is driven by links, and having responses reach up and be coupled to the route formats by rendering explicit links is a pain. I'd like for routes to be named, for the routemap to be put into state, and for handlers to be able to call a method on the routemap with appropriate parameters provided to render a link that matches to the appropriate named route.

And, so long as I'm wishing, I wish that Gotham routes looked like (URL templates)[https://tools.ietf.org/html/rfc6570].

Maybe phasing in a new pattern for routes could bring all three features.

[msrd0]

@msrd0 was proposing that matching be handled by a Gotham derive rather than a Serde Deserialization.

This is not entirely correct. Serde doesn't neccessarily need to be removed. I do see some value in being able to use non-standard types like chrono::NaiveDate for a path parameter. However, serde is a library that maps a rust struct to some serialization primitives (and the other way around), and therefore afaik cannot be used to validate that some (de)serializable struct has the same parameters as the path template string. So ideally, we would write code like this

route.get(path!("/foo/{date: chrono::NaiveDate}")).to(|path| unimplemented!())

and have the path! macro generate a struct for the path parameters (possibly using serde). However, I don't think such a macro is currently possible.

What I previously experimented with was a macro that applies (possibly in addition to serde) to the struct that parses the path at runtime, and have that struct provide the template string. However, I didn't really like the solution.

I'd like to revive one of the features I'd like to see: bidirectional routing

That sounds very useful. Do you have a concrete idea in mind how that api could look? Ideally, we'd be able to at compile time generate such a route-lookup-struct that knows all routes and their path parameters, but again I don't think such a macro is currently possible.

This feature also would be a good step towards generating OpenAPI/Swagger specifications, which has been requested before (and I'd love to have this feature so I can stop hacking custom macros on top of gotham).

And, so long as I'm wishing, I wish that Gotham routes looked like (URL templates)[https://tools.ietf.org/html/rfc6570].

Given that rust also uses {} for format/template strings, it might make sense to change the syntax of our path templates. However, assuming we support all levels mentioned in that RFC, this will probably increase complexity a bit.

[nyarly]

Do you have a concrete idea in mind how that api could look?

I come to Gotham by way of Rails, so that's my inclination. I think something like

impl RouteMap {
  path_to(&self, name: &String, params: &BorrowBag) -> String
}

or something in that vein would be handy. A helper for full URLs might be worthwhile, but that tends to be fraught. Do we use a "canonical" domain? The domain from request URL?

[msrd0]

A helper for full URLs might be worthwhile, but that tends to be fraught.

The easiest way would probably be to make all URLs relative to gotham's root, ignoring the domain and possible reverse proxies that add/remove path segments. This is also what we'd need for an OpenAPI spec. Users could then easily add those details as they see fit.

However, for certain handlers, e.g. those that generate HTML pages, it would be much more helpful to have a path relative to the current URL. That means the RouteMap wouldn't be identical for all routes.

---

Regarding the path params, if we already have a (de)serializable struct for those, since we use that for the extractor, wouldn't it make sense to reuse that?

[nyarly]

have the path! macro generate a struct for the path parameters

I see what you're saying here. I appreciate the convenience, but even if that macro could be figured out, it seems like there's drawbacks with regards to the automatically generated struct, but I think they boil down to: how do I refer to the type?

In terms of design, I think there's value to investigating the idioms we'd like to see. A lot of APIs reuse fields (i.e. :user_id), and as I understand the routing API now, it's possible to

route.get("/pov/:user_id/:post_id")
         .with_path_extractor::<UserParam>()
         .with_path_extractor::<PostParam>()
         .to(my_handler);

right?

therefore afaik cannot be used to validate that some (de)serializable struct has the same parameters as the path template string

In principle, Serde might allow us to validate a 1:1 relationship between parameters in that path and fields in the struct. It will certainly fail to serialize (at request-time, unfortunately) if the path lacks parameters that match the extractors, though. Again, multiple extractors complicates this.

It would certainly be ideal if we could validate that all fields of the path get extracted at compile time. I don't have a clear idea how to accomplish that though.

[msrd0]

This code leads me to believe that above example compiles and runs without a warning, but doesn't work as expected

    fn with_path_extractor<NPE>(self) -> <Self as ReplacePathExtractor<NPE>>::Output
    where
        NPE: PathExtractor<Body> + Send + Sync + 'static,
    {
        self.replace_path_extractor()
    }

(https://docs.rs/gotham/0.6.0/src/gotham/router/builder/single.rs.html#659)

[nyarly]

:(

[nyarly]

At the very least, that needs better documentation.

[msrd0]

I just saw #145 suggesting to keep the template string separate but validate against the struct at compile time, possibly using a custom proc macro. This is what I came up after thinking about it for a few minutes: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=93fcee7935cc1e638fac2155fdad8f78

It won't be easy to sanely implement this, since serde allows renaming structs, making certain fields optional etc., which will mess with these compile time assertions. Maybe we could reduce these issues and the resulting complexity if we didn't use #[derive(Deserialize)] on the extractor struct itself but only for its fields.