[Need Help] How can I generate Swagger/OpenAPI v3 spec from .proto files and vice versa

[krishna-birla]

It seems that only binary protocol buffers are supported for interconversion between protobuf and swagger. How can I convert binary protocol buffers (.pb) to and from proto3 protocol buffers (.proto), is that even a sane question to ask.

Moreover, my final intent is to be able to interconvert between .proto and OpenAPI2/3 spec (.json/.yaml) files, either using annotations in the .proto file, some tags or otherwise.

Thanks in advance!

Edits:

Modifying the issue title a little to highlight the use case much better.

External links:

Stack Overflow | How to generate swagger3 (OpenAPI3) spec in (.json/.yaml) from protobuf (.proto) files?

[LorenzHW]

You can go from .json/.yaml to .proto with gnostic-grpc.

[krishna-birla]

Thanks for the quick response!!

This project is a very nice initiative, and I am super glad to see it incubate in GSOC! But there are a few issues that I think are fundamental given the description of the project is clear:

This tool converts an OpenAPI v3.0 API description into a description of a gRPC service that can be used to implement that API...

So

So a major feature added in OpenAPI3 over the previous version 2 was the ability to have multiple schemas for the same HTTP verb + path, using the oneOf, anyOf and allOf constructs. There are constructs in protobuf such as oneOf, using different combinations of these can simulate all OpenAPI3 constructs. The current version of the project does not support it.

The current conversion logic loses a lot of information from the spec (it is lossy), and hence converting back is neither possible nor supported (as of now). So generating servers and clients from these specs is not possible.

Moreover, some of my test examples were missing some tags from the annotations like body tag in results. A lot of swagger3 fields are today not supported. And above all, this is NOT an inter-conversion library, it does not convert to and from, where as the more practical use case is proto to swagger3 and not the other way around, since most implementations wrap an HTTP(S) server around gRPC server, and not the other way around.

But

But clearly this project is WIP, and what it already does is very helpful in some use cases. I would love to contribute to the project and be a part of it. This has good scope, it is trying to deal with a problem everyone has but no one talks about.

Meanwhile

Meanwhile, are there anymore "production" usable alternatives to solve my original problem?

[balnice]

@krishna-birla My team is also looking for the same and we've found 2 more alternatives for evaluation so far:

• openapi2proto only supports swagger/openapi spec 2.0
• openapi-generator only support protocol buffers version 3

Hope these help.

[krishna-birla]

Thanks @balnice
The second option looks promising, but the fundamental requirement of interconversion is still on hang. These libraries don't convert .proto to OpenAPI v3 spec, and only do it the other way around.

The most common and main use case is to wrap an HTTPS server around a gRPC server. This means that the developer writes the .proto files and the rest of the steps (i.e. generate gRPC server, generate gRPC client, generate OpenAPI v3 spec, generate HTTPS server, generate HTTPS client) should be automated.

The second repository you suggested definitely helps, but the question still remains open.

[shikag]

@krishna-birla Were you able to resolve this? I also have the same thing: Rest APIs in Java Spring Boot using Protobuf classes as arguments, and looking to generate Swagger UI.

Have you tried using protoc to generate .pb files from .proto and then using gnostic to generate OpenAPI V3 (something like reverse of this)

[krishna-birla]

@shikag It didn't work for me, I also thought that .pb is just intermediate code for protobuf, but it doesn't seem like it.

[shellsong]

@krishna-birla have you try this which can convert *.proto to swagger json schema

[krishna-birla]

@shellsong Yes, it supports OpenAPIv2, I am looking for v3.

[shikag]

Thanks!

[mpokryva]

You can run the generated OpenAPI v2 code through https://github.com/getkin/kin-openapi/tree/master/openapi2conv to get OpenAPI v3 code. This would work if you don't want to build something "cleaner," and you don't need any OpenAPI v3 features, just the syntax.

[krishna-birla]

I need the anyOf and oneOf features. That is the catch.

[mpokryva]

I see. One thing that should be easier to do would be to convert the v2 spec to v3 as I said above and then fork grpc-gateway and use gnostic's v3 proto spec to implement anyOf and oneOf annotations. See https://github.com/grpc-ecosystem/grpc-gateway/blob/master/protoc-gen-swagger/options/annotations.proto for the v2 equivalent. Not "clean," but it should work.

[timburks]

Hi @krishna-birla, do you have any end-to-end examples that illustrate your goals? Specifically I'm curious to see an OpenAPI spec that has examples of your usage of anyOf and oneOf, and even better if you have a corresponding .proto file - I'm guessing that since there's no automated conversion available, you might be manually mapping from proto to OpenAPI in handwritten code. Anyway, I think an example would be very helpful. Thanks!

[krishna-birla]

Hi @timburks
Thanks for the prompted reply.

A simple example could be something like this:
from

...
message CreateAnimalRequest {
  oneof Aminals {
    Cat cat = 1;
    Dog dog = 2;
    Lizard lizard = 3;
  }
}
...

to

...
requestBody:
        content:
          application/json:
            schema:
              oneOf:
              - $ref: '#/components/schemas/Cat'
              - $ref: '#/components/schemas/Dog'
              - $ref: '#/components/schemas/Lizard'
          application/yaml:
            schema:
              oneOf:
              - $ref: '#/components/schemas/Cat'
              - $ref: '#/components/schemas/Dog'
              - $ref: '#/components/schemas/Lizard'
...

The application/yaml and application/json parts can be taken from tags, so please ignore those.

Yes, I am forced to write a script of my own given the lack of a library, but I would prefer using one from openAPI or googleapis. I will be more than happy to contribute!