6.2. Network communication

In this chapter we will look at network communication to enable processes running on different computers to exchange information.

Communication between computers

Using the file system is an easy way to have multiple processes communicate with each other, under the requirement that the processes run on the same machine. Modern computing architectures are increasingly moving towards networks of computers. The most prominent concept here is cloud computing: The availability of compute resources and storage without knowing (or caring about) the actual topology of the machines that do the work.

To connect multiple machines that are physically disjoint, we use computer networks, whose study warrants a lecture series on its own. In its essence, computer networks work through a combination of physical components and a bunch of protocols that dictate how data is exchanged between multiple computers. The most important technologies include Ethernet for wired connections and the IEEE 802.11 standard for wireless connections, as well as the protocols TCP/IP and HTTP. Describing the way computer networks work is the OSI model, which defines seven abstraction layers that cover everything from the physical connection of computers to the interpretation of bytes by applications:

On the level of systems programming, we usually deal with everything from level 4 (the transport layer) upwards. Rust for example provides APIs in the standard library for the levels 4 and 5, to create connections using the TCP or UDP protocols. Higher level protocols are then covered by libraries. Some application programming languages, for example JavaScript, focus solely on level 7 protocols, mainly HTTP. To get lower level support, things like WebSockets can be used.

Connections - The basic building block of network communication

Network communication is based on connections between two machines or processes. All network communication is based on a client-server principle, where one process (the server) provides access to some resource through a service to other processes called the clients.

At this point you might think: 'Wait a minute! What about peer-to-peer networks? Aren't they like the opposite of the client-server model?' Which is a viable question, especially seeing that more peer-to-peer technologies such as blockchains have emerged in recent years. At its core, any peer-to-peer network still uses a client-server model internally, the only difference is that in a peer-to-peer network, every machine or process can be both server and client at the same time.

Servers are always systems software! This is because they are meant to provide their functionality to other software, not to users directly. Communication with a server always works through client software, since most server applications don't have any form of user interface. As a systems programming language, Rust is thus a good choice for writing (high-performance) server code!

So how do network connections actually work? There are different technologies for establishing network connections, we will look at connections in the internet using the internet protocol (IP). Every network connection over the internet has to fulfill a bunch of requirements (as per [Bryant03]):

• It is a point-to-point connection, meaning that it always connects a pair of processes with each other. This requires that each process can be uniquely identified. We will see in a moment how this works.
• It has to be a full-duplex connection, which is the technical term for a communications channel in which data can flow in both directions (client to server, and server to client).
• In general we want our network connections to be reliable, which means that the data sent from the source to the target eventually does reach the target. While Bryant et al. define this as a requirement for network connections, this does depend on the type of protocol used. If we want a reliable connection, TCP (transmission Control Protocol) is typically used nowadays, which guarantees that all data arrives in the same order it was sent and no data must be dropped. If this degree of reliability is not needed, for example in media-streaming services, Voice-over-IP applications or some multiplayer games, a simpler protocol called UDP (User Datagram Protocol) can be used, which makes no guarantees over order or reliability of the data transmission.

Identifying processes to establish connections using the Internet Protocol

Before we can establish a network connection, we need a way to identify processes so that we can connect them together. For this to work, we first need a way to uniquely identify machines connected to a network. We can use the Internet Protocol (IP) for this, which is a network layer protocol (layer 3 of the OSI model). With IP, data sent over a network is augmented with a small header containing the necessary routing information in order to deliver the data to the desired destination. Part of this header are unique addresses for the source and target machines called IP addresses.

There are two major versions of the IP protocol: Version 4 (IPv4) and version 6 (IPv6). Version 4 is currently (as of late 2021) the most widely used version in the internet, and it defines an IP address as an unsigned 32-bit integer. For us humans, IP addresses are usually written as a sequence of 4 bytes in decimal notation, separated by dots. As an example, 127.0.0.1 is a common IP address referring to the local machine, and it is the human-readable form of the IP address 0x7F000001. Since IPv4 uses 32-bit addresses, it can encode an address space of \(2^{32}=4,294,967,296\) unique addresses. Due to the large number of computers connected to the internet around the world, this address space is not sufficient to give a unique address to every machine, requiring techniques such as network address translation (NAT) which allows private networks of many machines to operate under one public IP address. Ultimately, IPv6 was devised to solve the problem of IP address shortage. IP addresses in IPv6 are 128-bit numbers, allowing a total of \(2^{128}=3.4*10^{38}\) unique addresses. Similar to IPv4 addresses, IPv6 addresses also have a human-readable form, consisting of up to eight groups of four hexadecimal digits each, separated by colons instead of dots: 2001:0db8:0000:0000:0000:8a2e:0370:7334 or 2001:db8::8a2e:370:7334 in a shortened form, is an example of an IPv6 address.

So with IP addresses, we can uniquely identify machines on a network. What about processes on a machine? Here, the operating system comes to help, and a trick is used. Instead of uniquely identifying processes, which would restrict each process to allow only one connection at most, we instead identify connection endpoints on a machine. These endpoints are called sockets and each socket has a corresponding address, which is a combination of the IP address of the machine and a 16-bit integer port number. If we know the IP address and port number of a service running on a server, we can establish a connection with this service from any other machine that is connected to the same network (disregarding network security aspects). Consequently, any service running on a server has to expose itself for incoming connections on a specific port so that clients can connect to this service. The concept of a socket is not part of the network layer, but instead of the transport layer (layer 4 of the OSI model), and as such is handled by protocols such as TCP and UDP. A socket address can look like this: 10.20.30.40:8000, which is a concatenation the IPv4 address 10.20.30.40 and the port number 8000 using a colon. This also works with IPv6 addresses, which have to be enclosed in square brackets however, because they already use a colon as separating character: [2001:db8::8a2e:370:7334]:8000 is the concatenation of the IPv6 address 2001:db8::8a2e:370:7334 and the port number 8000. A network connection can thus be fully identified by a pair of socket addresses.

When establishing a connection to a server, the client socket gets its port number assigned automatically from the operating system. For connections on the internet, we thus always use sockets. Many high level programming languages provide abstractions around sockets for higher-level protocols, for example to fetch data from the internet using the HTTP protocol, but ultimately, every connection is built around sockets.

Working with sockets in Rust

For the operating system, sockets are endpoints for connections and the operating system has to manage all low-level details for handling this connection, such as establishing the connection, sending and receiving data etc. Using the file abstraction, from the point of view of a program, a socket is nothing more than a file that data can be read from and written to. This is very convenient for development of network applications, because we can treat a network connection similar to any other file (illustrating the power of the file abstraction).

On Linux, the low-level details of working with sockets are handled by the socket API, but we will not cover it in great detail here. Instead, we will look at what Rust offers in terms of network connections. Sockets typically are covered in an introductory course on operating systems, so you should be somewhat familiar with them.

The Rust standard library provides a bunch of useful functions and types for networking in the std::net module. Let's look at the most important things:

First, there are a bunch of types for IP and socket addresses, with support for both IPv4 and IPv6: Ipv4Addr, Ipv6Addr, SocketAddrV4, and SocketAddrV6. These types are pretty straightforward, as the Rust documentation shows:

#![allow(unused)]
fn main() {
use std::net::{Ipv4Addr, SocketAddrV4};

let socket = SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), 8080);

assert_eq!("127.0.0.1:8080".parse(), Ok(socket));
assert_eq!(socket.ip(), &Ipv4Addr::new(127, 0, 0, 1));
assert_eq!(socket.port(), 8080);
}

Then there are the main types for managing network connections. Rust provides support for the two main layer-4 protocols UDP and TCP in the form of the types UDPSocket, TcpListener and TcpStream. Since UDP uses a connectionless communications model, there is no need to differentiate between the server and client side of the connection. For TCP, there is a difference, which is why Rust defines two different types for TCP connections. TcpListener is for the server-side of a connection: It exposes a specific port to clients and accepts incoming connections. The actual connections are handled by the TcpStream type.

Looking at the TcpStream type, we see that it implements the Read and Write traits, just like File does! This is the file abstraction at work in the Rust ecosystem. The neat thing about this is that we can write code that processes data without caring whether the data comes from a file or over the network. All the low-level details of how the data is transmitted over the network is handled by the Rust standard library and the operating system.

Writing network applications in Rust

Understanding network code is a bit harder than understanding regular code, because network code always requires multiple processes (at least one client and one server) that work together. So for our first venture into the world of network code, we will write only a server and use another piece of software to send data to the server: curl. curl is a command line tool for transferring data and can be used to easily connect to various servers to send and receive data.

Here is our first server application written in Rust, which accepts TCP connections at a specific port 9753, reads data from the connected client(s), prints the data to the console and sends the data back to the client(s) in reverse order:

use anyhow::{Context, Result};
use std::io::{Read, Write};
use std::net;

fn main() -> Result<()> {
    let listener = net::TcpListener::bind("127.0.0.1:9753")?;

    for connection in listener.incoming() {
        let mut connection = connection.context("Error while accepting TCP connection")?;

        let mut buf: [u8; 1024] = [0; 1024];
        let bytes_read = connection.read(&mut buf)?;

        let buf = &mut buf[..bytes_read];
        println!("Got data from {}: {:?}", connection.peer_addr()?, buf);
        // Reverse the bytes and send back to client
        buf.reverse();
        connection.write(buf)?;
    }

    Ok(())
}

This example uses the anyhow crate to deal with the various kinds of errors that can occur during network connections. As we can see, using the Rust network types is fairly easy. Calling TcpListener::bind("127.0.0.1:9753") creates a new TcpListener that listens for incoming connections to the port 9753 on the local machine (127.0.0.1). We get access to these connections by using the incoming method, which returns an iterator over all incoming connections, one at a time. Since there might be errors while establishing a connection, incoming actually iterates over Result<TcpStream, std::io::Error>. To get some more error information in case of failure, we use the context method that the anyhow crate provides. Once we have an established connection, we can call the usual methods from the Read and Write traits on it.

If we run this server, we can send data to it using curl. To send textdata, we can use the we can use the telnet protocol, like so: curl telnet://127.0.0.1:9753 <<< hello Running this from a terminal yields the following output:

curl telnet://127.0.0.1:9753 <<< hello

olleh%

If we inspect the server output, this is what we see:

Got data from 127.0.0.1:60635: [104, 101, 108, 108, 111, 10]

We see that our client connection ran on port 60635, which was a randomly assigned port by the operating system, and that the server received the following sequence of bytes: [104, 101, 108, 108, 111, 10]. If we translate these bytes into text using ASCII encoding, we see that they correspond to the string helloLF. The last character is a new-line character (LF), which explains why the response message starts with a new line.

Using our Rust server, we can write our first network client in Rust. Instead of using the TcpListener to listen for incoming connections, we directly create a TcpStream using TcpStream::connect:

use anyhow::Result;
use std::io::{Read, Write};
use std::net::TcpStream;

fn main() -> Result<()> {
    let mut connection = TcpStream::connect("127.0.0.1:9753")?;

    let bytes_written = connection.write(&[2, 3, 5, 7, 11, 13, 17, 23])?;
    let mut buf: [u8; 1024] = [0; 1024];
    let bytes_read = connection.read(&mut buf)?;
    if bytes_read != bytes_written {
        panic!("Invalid server response");
    }
    let buf = &buf[..bytes_read];
    println!("Server response: {:?}", buf);

    Ok(())
}

It simply sends a series of bytes containing some Fibonacci numbers through the network connection to the server and reads the server response. Running our client from a terminal gives the following output:

Server response: [23, 17, 13, 11, 7, 5, 3, 2]

As expected, the server returned the numbers in reverse order. With these, we have laid the foundation for writing network code in Rust!

Protocols

With the functionality from the std::net module, we can send raw bytes between different processes, however this is very hard to use, because neither server nor client know what bytes they will receive from the other end. To establish a reasonable communication, both sides have to agree on a protocol for their communication. This happens on the last layer of the OSI model, layer 7, the application layer. Here applications define what byte sequences they use to communicate with each other, and how these byte sequences are to be interpreted. If you write your own server, you can define your own protocol on top of the lower-level protocols (TCP, UDP), or you can use one of the established protocols used for network communication between applications in the internet, such as HTTP.

You could implement these protocols manually, they are fully standardized, but there are many good implementations in the form of libraries available. Working with these libraries will require a bit more knowledge of Rust, in particular about asynchronous code, and we won't cover this until chapter 7, so we won't cover using HTTP in Rust until we know about these other features.

Exercise - Writing a prime number server

Without knowing any protocols, the range of network applications that we can reasonably implement in Rust is limited. To get a better feel for why protocols are necessary, try to implement the following server application in Rust using the TCP protocol:

Write a server that can compute compute prime factors for numbers, and can also compute prime numbers. It should support the following operations:

• Compute the Nth prime number. By sending the positive integer number N to the server, the server should respond with the Nth prime number.
  • Example: Send 10 to the server, and the server responds with 29. Try to come up with a reasonable response if N is too large for the server to handle!
• Compute the prime factors of the positive integer number N. The server should respond with all prime factors of N in ascending order.
  • Example: Send 30 to the server, and the server responds with 2 3 5.
• Check whether the positive integer N is a prime number.
  • Example: Send 23 to the server, and the server responds with either 1 or yes

To implement these functionalities, you will need some kind of protocol so that the server knows which of the three functions (compute prime, compute prime factors, check prime) it should execute. For the responses, you also need a protocol, because they will vary in length and content. Think about the following questions:

• Calling read from the client on the connection to the server works with a fixed-size buffer. How can you ensure that you read the whole server response?
• Both the read and write functions send raw bytes, but your server should support numbers larger than a single byte. Decide whether you want to send numbers in binary format or as strings. What are the advantages and disadvantages of each solution?