Reducing common structs types or https://github.com/kube-rs/gateway-api-rs/issues/38

[dawid-nowak]

This approach aims to simplify the types and eliminate redundant ones. The approach works as follows:

1. We scan generated files and find similar items (structs or enums). Two structs are considered to be similar and processed further if their fields are exactly the same. The same method applies to the enums.
2. Once all similar structs and enums have been found, then we try get the name that will applied to similar structs from a file.
3. Once the new type names are settled we change the generated code and add a new file with the reduced types "common_types.rs"

We follow this process in phases:

1. In Phase 1, we only look at the structs with fields which are built from simple types (String, u32, u64)
2. In Phase 2, we also look at the structs with fields that have been changed in Phase 1
3. and repeat

Problems:

1. The documentation is stripped since the same type could be used in many different locations

[shaneutt]

First of all thank you for the PR. The time you put into this solution is very much appreciated!

I've done a high level review, and I see how the process works and the output, and it's impressive. I'll do a in-depth review soon but first I wanted to start off with some questions and comments regarding this approach from a couple dimensions:

UX

Obviously not having multiple types that do the same thing is a huge improvement, so that's great. In this draft however, users would currently have to use the "processed" feature/module. Ideally the users of this crate would have to do nothing special, nor have to understand any semantics of our crate that relate to how we generate code.

What are your thoughts on the feasibility of having these generators interleave the generated code with the code otherwise generated by kopium? That is to say, can we avoid the new processed module (and feature) and emit common_types.rs and the others right next to the APIs they belong to? 🤔

Maintainability & Kopium Interactions

Overall while I think the type-reducer has its complexities, those complexities are natural given the kind of work we're trying to do here. In my brief pass I didn't notice anything that stood out as an obvious workaround or anything like that. In fact, I was very pleased to see it's all built on top of a solid foundation on the syn crate, which is great.

I am curious though, after you've gone through the exercise of building this out: Do you have any new thoughts on the interaction with kopium here? How much of this work might actually be applicable there? Curious as to your thoughts 🤔

---

cc @clux as you may find this one interesting

[dawid-nowak]

Thanks for the kind words. Let me answer your questions:

UX
I think the "processed" name is not great but that could be easily changed. The Kubvernor Gateway implementation was initially based on "standard" APIs but I wanted to see whether "processed" types would improve the code. I have migrated the existing codebase to "processed" APIs and I think the results are optimistic (at least from my perspective).

Initially, I created a lot of types manually to handle duplication between the HTTP and GRPC Routes. With "processed" types I was able to automate that part somewhat. As a result, I was able to (re)move and refactor some duplicated code.
The one thing I found is that I would never mix the types. I would either rely on the types provided by "standard" APIs or on types provided by "processed" APIs.

It is very doable to emit "common" types and keep them side by side with "standard" types. In this approach I think we would need to provide a whole bunch of converters from "standard" types into "common" types. This would be very much akin to my original approach to implementing Kubvernor.

I think from the developer perspective such an approach would be confusing because the developers would need to keep selecting which type to use at a given moment in time and keep on converting from one type to another.

Maintainability & Kopium Interactions
This is an interesting one and it prompted me to learn more about the Kubernetes APIs.
Ultimately, I think the problem is that the yaml files describing CRDs are not using OpenAPI reference mechanisms to avoid duplication of types.

When one investigates the Go implementation, it is clear that there is a lot of code reuse there. GRPCRoute is dependent on types from HTTPRoute for example and there is shared_types module with type definitions shared among all CRDs.

This reuse disappears when yaml versions of the CRDs are generated. All information on how the types are reused is erased which makes it super hard for Kopium.

I think that if yaml/OpenAPI definitions relied on OpenAPI references to reuse shared definitions then it would be much easier for Kopium to generate Rust code very similar to the one in "processed" APIs. Indeed, this PR probably wouldn't even be necessary.

I don't know enough of the Kubernetes/Gateway API process to speculate whether it would be possible to change how yaml versions are generated.