Redefining vnet

[enobufs]

@Sean-Der @stv0g @Antonito and anyone who cares about "vnet":
I feel vnet package should evolve with a fresh eye. From what it is potentially capable of and how broadly its features/updates could affect other packages or even other projects, it may deserve a dedicated repository with its own versions.

The primary focus of this 'vnet' package was #13, for testing complex NAT traversal (UDP hole punching) scenarios for ICE (for which it worked pretty well, found many bugs!), then people's expectations has grown, current API does not seem to be sufficient enough to match the needs.

I'd really hope there would be more contributors to this feature. Drawing more attention from pion community for a new design may lead us to a better place, with healthier sense of ownership of the feature. So, why don't we start with discussing what new version of vnet should look like?

Things to think about:

• Redefine goals of the future 'vnet'
• Dedicated repo for this package
• Package name. Using 'vnet' (virtual network) in production (as @stv0g implied) feels wrong.
• API redefined (better interface with pluggability/extensibility)

Relates to #141, #27, #34, #168

[stv0g]

Hi @enobufs,

thanks for starting the discussion!

I'd really hope there would be more contributors to this feature.

While I am still relatively new to Pion, I am planning to start contributing more to the stun, turn, ice and transport packages as I need those for my own project.

Dedicated repo for this package

I am not too sure about whether it really needs its own package as I am wondering what will remain in the transport package?

So, why don't we start with discussing what new version of vnet should look like?

In my opinion, it should match 100 % the API of the standard net package with exception that all functions are converted to receiver functions for a Net interface. Nothing more, nothing less. I think it should be a drop-in replacement for net and we should be able to confidently grep through the Pion codebase and replace any occurence of a net Function with this interface.

There are some features missing to make this happen.
Mainly the TCP support, which is currently also the reason that TCP TURN and ICE are only partially supported in Pion.

In my PR #204, I introduced a new Interface with two new implementations (StdNet and VNet).
The first is just a wrapper for the methods from the standard net package while the latter the the virtual network.
Both the interface and those two implementations should and can be fully platform independent in my opinion.

However, I think there are also a few more platform specific features around networking, which this package should provide.
The main one, which I would like to tackle in the feature is monitoring the system for networking changes (added, removed interfaces and IP addresses). This would allow us to support ICE connection upgrades during the lifetime of an ICE session once better candidates become available.

On a side note: I am the author of Gont which has similar goals as the original vnet package but attempts to use the Kernels networking facilities rather than implementing a networking stack in userspace.
This has the big advantage, that the code which is tested needs no modifications. However, Gont relies on Linux network namespaces, and hence can only be used for testing on Linux which I feel might be a blocker for Pion.

[enobufs]

Hello @stv0g - my apologies for late reply.

I am planning to start contributing more to the stun, turn, ice and transport packages as I need those for my own project.

This is great! Your project Gont looks very interesting. I can see if this work allows us to use both of these tools, it would be a big win for the developers.

I am not too sure about whether it really needs its own package as I am wondering what will remain in the transport package?

I felt vnet solution deserve a dedicated repo but not strongly.

I think it should be a drop-in replacement for net and ...

This sounds very good.

Mainly the TCP support, which is currently also the reason that TCP TURN and ICE are only partially supported in Pion.

Agreed. It's been a desired feature in vnet. Do you have any good idea or tool to support TCP in vnet? Otherwise, I'd considering using pion/sctp to emulate TCP.

For vnet, simulating network impairment and NAT behavior in the unit tests were the main focus and emulation of performance could be less in priority. (I think performance aspect could be evaluated by a real kernel space solution like your Gont.)

Let me review #204. If you'd agree, we could aim for pion/transport v2, and I am hoping that you could lead this project for us to get there. I'll do my best to support this!

[stv0g]

Agreed. It's been a desired feature in vnet. Do you have any good idea or tool to support TCP in vnet? Otherwise, I'd considering using pion/sctp to emulate TCP.

Yes, Google's gVisor project has a Userspace TCP implementation which we probably can use: https://github.com/google/gvisor/tree/master/pkg/tcpip