6.3. Interprocess communication using signals and shared memory

In the previous section, we talked about communication between processes using a network connection. While this is a very general approach, there are situations where you might want more performance, in particular if you have multiple processes on the same machine that have to communicate. In this chapter, we will cover techniques for exchanging data between processes on a single machine. This process is called interprocess communication.

Besides using network connections, there are two main ways for interprocess communication in modern operating systems:

• Signals
• Shared memory

Signals

Signals are a mechanism provided by the operating system to exchange small messages between processes. These messages interrupt the regular control flow of a process and are useful to notify a process that some event has happended. Signals are used to notify a process about some low-level system event, for example:

• An illegal instruction was encountered by the CPU
• An invalid memory address was being accessed
• A program tried to divide by zero
• A program has received a stop signal from the terminal

Signals are not useful if you have a lot of data to send to a process. Instead, signals are often used to communicate fatal conditions to a program, causing program termination. Some signals, such as the one for stopping a process, halt the program until another signal is received that tells the program to continue executing.

So what exactly is a signal? The concept of signals is implemented by the operating system, and here signals are basically just numbers that have a special meaning. All valid signals are specified by the operating system and can be queried from a terminal using man 7 signal on Linux (or just man signal on MacOS). For each number, there is also a (semi) human-readable name, such as SIGILL for an illegal instruction or SIGSEGV for invalid memory accesses.

When using signals, you always have one process (or the OS kernel) that sends the signal, and another process that receives the signal. Many signals are sent automatically by the OS kernel when their respective conditions are encountered by the hardware, but processes can also explicitly send a signal to another process using the killThe name kill is a historical curiosity. The kill function can be used to send various signals to a process, not just for killing the process, as the name would imply. Historically, this was the main reason for the function, but over time, the ability to send different signals than the default SIGKILL signal for terminating a process was added, but the name stayed. function. Processes react to incoming signals using signal handlers, which are functions that can be registered with the operating system that will be called once the process receives a certain signal. All signals have default behaviour if no signal handler is in place, but this behaviour can be overwritten by adding a signal handler in your program. On Linux, registering a function as a signal handler is done using the signal() function.

Signals are most useful for process control, for example putting a process to sleep, waking it again, or terminating it. Signals themselves have no way to carry additional information besides the signal number, so their use for interprocess communication is limited. If for example you want to send some string message from one process to another, signals are not the way to do that. It is nonetheless good to know about signals as you will encounter them when doing systems programming. Especially in C++, the dreaded SIGSEGV comes up whenever your program tries to access invalid memory. We will not go into more detail on signals here, if you are interested, most operating systems courses typically cover signals and signal handling in more detail.

Shared memory

To exchange more information between processes, we can use shared memory. Shared memory is a concept closely related to virtual memory. Remember back to chapter 3.2 when we talked about virtual memory. Every process got its own virtual address space, and the operating system together with the hardware mapped this virtual address space to physical memory. This meant that virtual pages in one process are distinct from virtual pages of another process. To enable two processes to share memory, they have to share the same virtual page(s), which can be done through special operating system routines (for example mmap on Linux).

Shared memory depends heavily on the operating system, and the way it is realized is different between Unix-like operating systems and Windows, for example. Even within Linux, there are multiple different ways to realize shared memory. We will look at one specific way using the mmap function, but keep in mind that other ways are possible as well.

The main idea behind shared memory is that two processes share the same region of a virtual address space. If two processes share a single virtual page, no matter where in physical memory this page is mapped, both processes will access the same memory region through this virtual page. This works similar to using a file that both processes access, however with mmap we can guarantee that the memory is always mapped into working memory. This way, a fast shared address space can be established between two processes.

Let's take a look at the mmap function and its parameters:

void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset);

• The first argument void* addr acts as a hint to the operating system where in the process' address space the mapping should happen, i.e. at which address. Since mapping virtual pages has page size granularity, an adjacent page boundary is chosen on Linux
• The size_t length argument determines the number of bytes that the mapping shall span
• int prod refers to some flags for the kind of memory protection of the mapping. These are equal to the read and write protection flags that we saw in the chapter on virtual memory. On Linux, pages can also be protected for the execution of code.
• int flags defines flags for the mapping. We can use MAP_SHARED to create a mapping that can be shared with other processes.
• int fd is a file descriptor (i.e., the file handle) to a file that should be mapped into the process' virtual address space. mmap is useful for more than just establishing shared memory, it is also one of the fastest ways for reading files (by mapping their whole contents to a virtual address range)
• The off_t offset parameter is not relevant for this example, but can be used to map a specific region of an existing file into memory

If both processes use the same file descriptor when calling mmap, the same region of the physical address space gets mapped into the virtual address space of both processes and shared memory has been established. We can use an existing file for this mapping, or use the shm_open function to create a new shared memory object, which is simply a handle to be used for calling mmap to establish shared memory. When using shm_open, you can set the size of the shared memory region using ftruncate. Once the shared memory mapping is no longer needed, call munmap to unmap the shared memory object, and then close to close the file handle you obtained from shm_open.

The biggest caveat when using shared memory is that both reads and writes are unsynchronized. Think about what happens when one process writes a larger piece of data to shared memory, while the other process is reading from the same region of shared memory. Without any synchronization between the two processes, the process that reads might read first, reading only old data, or read somewhere inbetween while the data has been partially written. This is called a data race, we will learn more about it in the next chapter when we talk about concurrent programming. For now it is sufficient to understand that it is impossible to write a correct program without getting rid of data races, which we do by introducing some synchronization mechanism. These are also provided by the operating system and typically include ways to be used from multiple processes at the same time. Semaphores are one example.

Shared memory in Rust

The routines for creating and managing shared memory depend heavily on the used operating system. In contrast to memory management routines such as malloc, which are abstracted in the Rust standard library, shared memory has not made its way into the standard library. Instead, we have to use a crate such as shared_memory that provides a platform-independent abstraction over shared memory. This crate also provides some examples on how to use shared memory in Rust.