Question: Can L4 tcp proxy have same behaviour as udp proxy with use_per_packet_load_balancing=true?

[didimelli]

Title: Per-packet tcp proxy

Description:
My app must receive a very high amount of data (300MBps) in a single tcp connection.

I would like to be able to split the incoming data into individual "chunks" that my app understands (e.g. my app receives 10KB packets) and proxy the newly created chunks to N upstream servers. Typical tcp load balancer would not help, since they make a 1:1 connection between the client and an upstream server.

My app would only reason by individual packets, so ordering is not an issue. Moreover, my app does not need to reply to the client (apart from the tcp ack). Basically my app is a sink for binary data that is serialized into 10KB packets (with a custom structure, with headers, crcs and payload) and sent over a tcp connection.

Reading the docs, I understood that udp proxy, when configured with use_per_packet_load_balancing=true, does exactly that.
I realize tcp (connection-based) is very different from udp (connection-less), but I was wondering if something similar is achievable (maybe with some custom extension)?

[didimelli]

Looking at the documentation, it looks like a good solution might be this https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/network_filters/generic_proxy_filter. Would it be a good way to go?

[htuch]

I think you would need to implement a custom TCP protocol for this that demuxed the TCP stream across multiple backend connections. Generic proxy filter could be a way to go, depending on complexity. CC @ggreenway @wbpcode

[didimelli]

Not sure about how to measure "complexity", but if you refer to the protocol it should be pretty straight forward, something in the lines of:

• sync word (always fixed)
• some headers
• first crc (computed on sync word and headers)
• fixed length payload data
• second crc (computed on whole packet)

All packets would are fixed length. No response is sent back to the sender (only tcp connection needs to stay live).