Sending a file over a network socket looks like the most trivial task in systems programming: read the bytes from disk, write them to the socket, done. Two system calls, a loop, a buffer. Yet that obvious approach makes your data cross the boundary between user space and the kernel four times and copies it four times end to end — and two of those copies are the CPU shuffling bytes that dedicated hardware could move for free. Understanding why is the difference between a file server that saturates the CPU and one that saturates the network card.
The hidden cost of read() + write()
Trace what actually happens when you read() a file into a buffer and write() (or send()) it to a socket. Per the classic IBM analysis of the problem, the traditional path incurs four copies and four context switches:
read()switches to kernel mode, and the DMA engine copies the file from disk into a kernel read buffer.- The CPU copies that data from the kernel buffer into your user-space buffer, and control returns to user mode.
send()switches back to kernel mode, and the CPU copies the data from your user buffer into a kernel socket buffer.- The call returns, and the DMA engine copies the socket buffer out to the network protocol engine — asynchronously.
Look at copies 2 and 3. Your data went from a kernel buffer, up into user space, and straight back down into another kernel buffer — untouched. Your program never looked at it or transformed it; it just acted as a very expensive relay. Those two CPU copies, plus the extra pair of context switches they require, are pure overhead. On a busy server moving gigabytes, that is a lot of memory bandwidth and CPU cycles spent doing nothing useful.
Why the kernel buffers in the first place
Before fixing it, it's worth knowing those intermediate kernel buffers aren't a mistake. They exist to improve performance in the common case: the kernel read buffer enables readahead (prefetching data you'll likely ask for next) and lets repeated reads of the same file hit the page cache instead of the disk. The problem is specific — it appears only when you are shuttling data straight through your process from one file descriptor to another without touching it. In that pattern, the trip through user space buys you nothing.
sendfile(): cut out the user-space round trip
The fix is to tell the kernel "move this file to this socket" and let it keep the data entirely in kernel space. That is exactly what sendfile() does. Introduced in Linux kernel 2.1, it collapses the read-then-write pair into a single system call.
The payoff is direct: the number of context switches drops from four to two, and the copies from four to three — and, crucially, only one of those three copies now involves the CPU. The data goes from disk to a kernel buffer by DMA, is copied once inside the kernel into the socket buffer, and is DMA'd out to the network. The pointless round trip up to user space and back is gone.
// Illustrative — the entire file-to-socket transfer in one syscall
#include <sys/sendfile.h>
off_t offset = 0;
ssize_t sent = sendfile(out_socket_fd, in_file_fd, &offset, count);
// The bytes never enter your process's address space.True zero-copy with scatter/gather DMA
One CPU copy still remains in that picture — the in-kernel copy from the read buffer to the socket buffer. Linux kernel 2.4 removed even that, given hardware that supports scatter/gather DMA. The socket buffer descriptor was modified so that instead of copying the actual bytes, the kernel merely appends descriptors — pointers recording the location and length of the data already sitting in the read buffer. The DMA engine then gathers the data directly from that buffer and passes it to the protocol engine.
The result is genuine zero-copy: two copies, both performed by DMA, and zero CPU copies, still with just two context switches. The CPU is entirely out of the data-movement business — it issues one system call and the hardware does the rest. This is why static-file web servers and streaming systems can push a network interface to its limit while the CPU stays nearly idle; the bytes travel from disk to NIC without the processor ever touching them.
Key Takeaways
- The naive
read()+write()path costs four copies and four context switches — two of the copies are the CPU needlessly relaying data through user space. - Kernel buffers are normally a feature (readahead, page cache); they only hurt when you pass data straight through untouched.
sendfile()(kernel 2.1) keeps data in kernel space: three copies, two context switches, only one CPU copy.- Scatter/gather DMA (kernel 2.4+) reaches true zero-copy — two DMA copies, zero CPU copies — by passing descriptors instead of bytes.
- Reach for zero-copy whenever you move data between file descriptors without transforming it; it trades CPU and memory bandwidth for near-free hardware transfers.