Skip to content

Commit

Permalink
Change Connection behavior to require both a packet send and receive (#…
Browse files Browse the repository at this point in the history
  • Loading branch information
ncallaway authored Sep 6, 2020
1 parent e90a68e commit 3411ee7
Show file tree
Hide file tree
Showing 10 changed files with 268 additions and 64 deletions.
9 changes: 5 additions & 4 deletions docs/md_book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,12 @@
- [Intro](intro.md)
- [Important Notices](important.md)
- [Protocols](protocols.md)
- [Connections](connections.md)
- [Heartbeat](heartbeat.md)
- [Fragmentation](fragmentation.md)
- [Reliability](reliability/basics.md)
- [Basics](reliability/basics.md)
- [Reliability](reliability/reliability.md)
- [Ordering](reliability/ordering.md)
- [Basics](reliability/basics.md)
- [Reliability](reliability/reliability.md)
- [Ordering](reliability/ordering.md)
- [Congestion Avoidance](congestion_avoidence/congestion_avoidance.md)
- [Whit RTT](congestion_avoidence/rtt.md)
- [Whit RTT](congestion_avoidence/rtt.md)
63 changes: 63 additions & 0 deletions docs/md_book/src/connections.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Connections

As mentioned in [Protocols](protocols.md) laminar is built on top of UDP. UDP is a [connectionless protocol](https://en.wikipedia.org/wiki/Connectionless_communication), but for multiplayer games we very often want to connect to a particular endpoint and repeatedly send data back and forth.

Laminar itself needs to maintain a small amount of data for each endpoint it is communicating with. For example, Laminar maintains data about which sent packets the other endpoint has acknowledged as well as an estimated [Rount Trip Time](congestion_avoidence/rtt.md).

In order to support these common use-cases Laminar adds an extremely simple connection model on top of UDP.

Connections are considered established whenever we have both sent and received data to the same endpoint. This means, to establish a connection your server needs to respond to inbound messages it receives. A sample of this is presented

**client**

```rust
let socket = Socket::bind(SERVER_ADDRESS);
let (sender, receiver) = (socket.get_packet_sender(), socket.get_event_receiver());
thread::spawn(move || socket.start_polling());

sender.send(Packet::reliable_unordered(SERVER_ADDRESS, "Ping".as_bytes().to_vec()));

loop {
if let Ok(event) = receiver.recv() {
match event {
SocketEvent::Connect(addr) => {
println!("Connected to: {}", addr);
},
_
}
}
}
```

**server**

```rust
let socket = Socket::bind(SERVER_ADDRESS);
let (sender, receiver) = (socket.get_packet_sender(), socket.get_event_receiver());
thread::spawn(move || socket.start_polling());

loop {
if let Ok(event) = receiver.recv() {
match event {
SocketEvent::Packet(packet) => {
if packet.payload() == b"Ping" {
sender.send(Packet::reliable_unordered(
packet.addr(),
"Pong".as_bytes().to_vec(),
)).unwrap();
}
},
SocketEvent::Connect(addr) => {
println!("Connected to: {}", addr);
},
_
}
}
}
```

If we don't send the `Pong` in the server, then neither the client nor the server will display the "Connected to" message.

### Packet Flooding Mitigation

Laminar will optimistically track data for endpoints before connections are established. As soon as data is sent or received from a new endpoint Laminar will start tracking the endpoint. In order to prevent packet flooding attacks from causing Laminar to allocate too much memory, the number of unestablished connections that Laminar will optimistically track can be controlled with the `max_unestablished_connections` Config.
27 changes: 17 additions & 10 deletions docs/md_book/src/protocols.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,41 @@
# Networking protocols

So first and possibly the important one is which protocol to use and when. Let’s first take a look at TCP and UDP.
When building any networked application the first and possibly the important decision to make is which protocol to use and when. Laminar is built on top of UDP. Let’s first take a quick look at both TCP and UDP, then we'll explain why Laminar uses UDP.

## IP
All communication over the internet is happening over IP (Internet Protocol).
This protocol only passes packets across the network without any guarantee that it will arrive at the destination.

All communication over the internet is happening over IP (Internet Protocol).
This protocol only passes packets across the network without any guarantee that it will arrive at the destination.
Sometimes IP passes along multiple copies of the same packet and these packets make their way to the destination via different paths, causing packets to arrive out of order and in duplicate.

So to be able to communicate over the network we make use of existing protocols that provides some more certainty.
So to be able to communicate over the network we make use of existing protocols that provides some more certainty.
We will first take a look at TCP where after we checkout UPD.

## TCP/IP
TCP stands for “transmission control protocol”. IP stands for “internet protocol”.

TCP stands for “transmission control protocol”. IP stands for “internet protocol”.
Together they form the backbone for almost everything you do online, from web browsing to IRC to email, it’s all built on top of TCP/IP.

TCP is a connection-oriented protocol, which means a connection is established and maintained until the application programs at each end have finished exchanging messages.
TCP provides full reliable, ordered communication between two machines. The data you send is guaranteed to arrive and in order.
TCP is a connection-oriented protocol, which means a connection is established and maintained until the application programs at each end have finished exchanging messages.
TCP provides full reliable, ordered communication between two machines. The data you send is guaranteed to arrive and in order.
The TCP protocol will also split up and reassemble packets if those are too large.

**Characteristics**

- Reliable
- Ordered
- Automatic [fragmentation](fragmentation.md) of packets
- Stream based
- Control Flow ([Congestion Avoidance](congestion_avoidence/congestion_avoidance.md))

## UDP

UDP stands for “user datagram protocol” and it’s another protocol built on top of IP, but unlike TCP, instead of adding lots of features and complexity, UDP is a very thin layer over IP.

Like IP, UDP is an unreliable protocol. In practice however, most packets that are sent will get through, but you’ll usually have around 1-5% packet loss, and occasionally you’ll get periods where no packets get through at all (remember there are lots of computers between you and your destination where things can go wrong…)

**Characteristics**

- Not Reliable
- Not Ordered
- No [fragmentation](fragmentation.md) of packets
Expand All @@ -39,9 +44,10 @@ Like IP, UDP is an unreliable protocol. In practice however, most packets that a
- Message based

## Why UDP and not TCP | More

Those of you familiar with TCP know that it already has its own concept of connection, reliability-ordering and congestion avoidance, so why are we rewriting our own mini version of TCP on top of UDP?

The issue is that multiplayer action games rely on a steady stream of packets sent at rates of 10 to 30 packets per second, and for the most part, the data contained in these packets is so time sensitive that only the most recent data is useful.
The issue is that multiplayer action games rely on a steady stream of packets sent at rates of 10 to 30 packets per second, and for the most part, the data contained in these packets is so time sensitive that only the most recent data is useful.
This includes data such as player inputs, the position, orientation and velocity of each player character, and the state of physics objects in the world.

The problem with TCP is that it abstracts data delivery as a reliable ordered stream. Because of this, if a packet is lost, TCP has to stop and wait for that packet to be resent.
Expand All @@ -57,6 +63,7 @@ What TCP does is maintain a sliding window where the ACK sent is the sequence nu
It is not possible to implement a reliability system with these properties using TCP, so we have no choice but to roll our own reliability on top of UDP. TCP itself is built on UDP.

## When use TCP

Of course there could be use-cases for TCP like chat, asset streaming, etc. We can setup a TCP socket for this that is distinct from UDP.

We could also make our UDP channel reliable as described below so when we detect package lost on the client we could construct a new package
We could also make our UDP channel reliable as described below so when we detect package lost on the client we could construct a new package
5 changes: 5 additions & 0 deletions src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,10 @@ pub struct Config {
/// When we send a reliable packet, it is stored locally until an acknowledgement comes back to
/// us, if that store grows to a size.
pub max_packets_in_flight: u16,

/// The maximum number of unestablished connections that laminar will track internally. This is
/// used to prevent malicious packet flooding from consuming an unbounded amount of memory.
pub max_unestablished_connections: u16,
}

impl Default for Config {
Expand All @@ -73,6 +77,7 @@ impl Default for Config {
socket_event_buffer_size: 1024,
socket_polling_timeout: Some(Duration::from_millis(1)),
max_packets_in_flight: 512,
max_unestablished_connections: 50,
}
}
}
5 changes: 3 additions & 2 deletions src/net/connection.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,15 @@ pub trait Connection: Debug {
/// * messenger - allows to send packets and events, also provides a config.
/// * address - defines a address that connection is associated with.
/// * time - creation time, used by connection, so that it doesn't get dropped immediately or send heartbeat packet.
/// * initial_data - if initiated by remote host, this will hold that a packet data.
fn create_connection(
messenger: &mut impl ConnectionMessenger<Self::ReceiveEvent>,
address: SocketAddr,
time: Instant,
initial_data: Option<&[u8]>,
) -> Self;

/// Connections are considered established once they have both had both a send and a receive.
fn is_established(&self) -> bool;

/// Determines if the connection should be dropped due to its state.
fn should_drop(
&mut self,
Expand Down
48 changes: 34 additions & 14 deletions src/net/connection_impl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ impl ConnectionEventAddress for SocketEvent {
SocketEvent::Packet(packet) => packet.addr(),
SocketEvent::Connect(addr) => *addr,
SocketEvent::Timeout(addr) => *addr,
SocketEvent::Disconnect(addr) => *addr,
}
}
}
Expand All @@ -44,15 +45,15 @@ impl Connection for VirtualConnection {
messenger: &mut impl ConnectionMessenger<Self::ReceiveEvent>,
address: SocketAddr,
time: Instant,
initial_data: Option<&[u8]>,
) -> VirtualConnection {
// emit connect event if this is initiated by the remote host.
if initial_data.is_some() {
messenger.send_event(&address, SocketEvent::Connect(address));
}
VirtualConnection::new(address, messenger.config(), time)
}

/// Connections are considered established once they both have had a send and a receive.
fn is_established(&self) -> bool {
self.is_established()
}

/// Determines if the given `Connection` should be dropped due to its state.
fn should_drop(
&mut self,
Expand All @@ -66,6 +67,12 @@ impl Connection for VirtualConnection {
&self.remote_address,
SocketEvent::Timeout(self.remote_address),
);
if self.is_established() {
messenger.send_event(
&self.remote_address,
SocketEvent::Disconnect(self.remote_address),
);
}
}
should_drop
}
Expand All @@ -80,6 +87,13 @@ impl Connection for VirtualConnection {
if !payload.is_empty() {
match self.process_incoming(payload, time) {
Ok(packets) => {
if self.record_recv() {
messenger.send_event(
&self.remote_address,
SocketEvent::Connect(self.remote_address),
);
}

for incoming in packets {
messenger.send_event(&self.remote_address, SocketEvent::Packet(incoming.0));
}
Expand All @@ -102,6 +116,10 @@ impl Connection for VirtualConnection {
time: Instant,
) {
let addr = self.remote_address;
if self.record_send() {
messenger.send_event(&addr, SocketEvent::Connect(addr));
}

send_packets(
messenger,
&addr,
Expand Down Expand Up @@ -143,15 +161,17 @@ impl Connection for VirtualConnection {
}

// send heartbeat packets if required
if let Some(heartbeat_interval) = messenger.config().heartbeat_interval {
let addr = self.remote_address;
if self.last_sent(time) >= heartbeat_interval {
send_packets(
messenger,
&addr,
self.process_outgoing(PacketInfo::heartbeat_packet(&[]), None, time),
"heatbeat packet",
);
if self.is_established() {
if let Some(heartbeat_interval) = messenger.config().heartbeat_interval {
let addr = self.remote_address;
if self.last_sent(time) >= heartbeat_interval {
send_packets(
messenger,
&addr,
self.process_outgoing(PacketInfo::heartbeat_packet(&[]), None, time),
"heatbeat packet",
);
}
}
}
}
Expand Down
Loading

0 comments on commit 3411ee7

Please sign in to comment.