Zero copy

"Zero-copy" describes computer operations in which the CPU does not perform the task of copying data from one memory area to another or in which unnecessary data copies are avoided. This is frequently used to save CPU cycles and memory bandwidth in many time consuming tasks, such as when transmitting a file at high speed over a network, etc., thus improving the performance of programs (processes) executed by a computer.[1][2][3][4]

Principle

Zero-copy programming techniques can be used when exchanging data within a user space process (i.e. between two or more threads, etc.) and/or between two or more processes (see also producer–consumer problem) and/or when data has to be accessed / copied / moved inside kernel space or between a user space process and kernel space portions of operating systems (OS).

Usually when a user space process has to execute system operations like reading or writing data from/to a device (i.e. a disk, a NIC, etc.) through their high level software interfaces or like moving data from one device to another, etc., it has to perform one or more system calls that are then executed in kernel space by the operating system.

If data has to be copied or moved from source to destination and both are located inside kernel space (i.e. two files, a file and a network card, etc.) then unnecessary data copies, from kernel space to user space and from user space to kernel space, can be avoided by using special (zero-copy) system calls, usually available in most recent versions of popular operating systems.

Zero-copy versions of operating system elements, such as device drivers, file systems, network protocol stacks, etc., greatly increase the performance of certain application programs (that become processes when executed) and more efficiently utilize system resources. Performance is enhanced by allowing the CPU to move on to other tasks while data copies / processing proceed in parallel in another part of the machine. Also, zero-copy operations reduce the number of time-consuming context switches between user space and kernel space. System resources are utilized more efficiently since using a sophisticated CPU to perform extensive data copy operations, which is a relatively simple task, is wasteful if other simpler system components can do the copying.

As an example, reading a file and then sending it over a network the traditional way requires 2 extra data copies (1 to read from kernel to user space + 1 to write from user to kernel space) and 4 context switches per read/write cycle. Those extra data copies use the CPU. Sending that file by using mmap of file data and a cycle of write calls, reduces the context switches to 2 per write call and avoids those previous 2 extra user data copies. Sending the same file via zero copy reduces the context switches to 2 per sendfile call and eliminates all CPU extra data copies (both in user and in kernel space).[1][2][3][4]

Zero-copy protocols are especially important for very high-speed networks in which the capacity of a network link approaches or exceeds the CPU's processing capacity. In such a case the CPU may spend nearly all of its time copying transferred data, and thus becomes a bottleneck which limits the communication rate to below the link's capacity. A rule of thumb used in the industry is that roughly one CPU clock cycle is needed to process one bit of incoming data.

Hardware implementations

An early implementation was IBM OS/360 where a program can instruct the channel subsystem to read blocks of data from one file or device into a buffer and write to another from the same buffer without moving the data.

Techniques for creating zero-copy software include the use of direct memory access (DMA)-based copying and memory-mapping through a memory management unit (MMU). These features require specific hardware support and usually involve particular memory alignment requirements.

A newer approach used by the Heterogeneous System Architecture (HSA) facilitates the passing of pointers between the CPU and the GPU and also other processors. This requires a unified address space for the CPU and the GPU.[5][6]

Program interfaces

Several operating systems support zero-copying of user data and file contents through specific APIs.

Here are listed only a few well known system calls / APIs available in most popular OSs.

Novell NetWare supports a form of zero-copy through Event Control Blocks (ECBs), see NCOPY.

The internal COPY command in some versions of DR-DOS since 1992 initiates this as well when COMMAND.COM detects that the files to be copied are stored on a NetWare file server,[7] otherwise it falls back to normal file copying. The external MOVE command since DR DOS 6.0 (1991) and MS-DOS 6.0 (1993) internally performs a RENAME (causing just the directory entries to be modified in the file system instead of physically copying the file data) when the source and destination are located on the same logical volume.[8]

The Linux kernel supports zero-copy through various system calls, such as:

Some of them are specified in POSIX and thus also present in the BSD kernels or IBM AIX, some are unique to the Linux kernel API.

FreeBSD, NetBSD, OpenBSD, DragonFly BSD, etc. support zero-copy through at least these system calls:

  • sendfile;[17]
  • write,[18] writev [19] + mmap [20] when writing data to a network socket.

MacOS should support zero-copy through the FreeBSD portion of the kernel because it offers the same system calls (and its manual pages are still tagged BSD) such as:

Oracle Solaris supports zero-copy through at least these system calls:

Microsoft Windows supports zero-copy through at least this system call:

Java input streams can support zero-copy through the java.nio.channels.FileChannel's transferTo() method if the underlying operating system also supports zero copy.[28]

RDMA (Remote Direct Memory Access) protocols deeply rely on zero-copy techniques.

See also

References

  1. ^ a b Stancevic, Dragan (2003-01-01). "Zero Copy I: User-Mode Perspective". www.linuxjournal.com. Retrieved 2021-10-14.
  2. ^ a b Bröse, Eduard (2012-01-01). "ZeroCopy: Techniques, Benefits and Pitfalls". CiteSeerX 10.1.1.93.9589. {cite journal}: Cite journal requires |journal= (help)
  3. ^ a b Song, Jia; Alves-Foss, Jim (2012-01-01). "Performance Review of Zero Copy Techniques" (PDF). www.uidaho.edu. Retrieved 2021-10-14.
  4. ^ a b Baldwin, John (2020-05-01). "TLS offload in the kernel" (PDF). freebsdfoundation.org. Retrieved 2021-10-14.
  5. ^ "The programmer's guide to the APU galaxy" (PDF).
  6. ^ "AMD Outlines HSA Roadmap: Unified Memory for CPU/GPU". 2012-02-02.
  7. ^ "Caldera OpenDOS Machine Readable Source Kit (M.R.S) 7.01". Caldera, Inc. 1997-05-01. Archived from the original on 2021-08-07. Retrieved 2022-01-02. [1] (NB. Actually implemented since DR DOS "Panther" on 1992-06-22, see COMCPY.C/DOSIF.ASM in the COMMAND.COM sources of OpenDOS 7.01.)
  8. ^ Paul, Matthias R. (1997-07-30) [1994-05-01]. "II.4. Undokumentierte Eigenschaften externer Kommandos: MOVE.EXE". NWDOS-TIPs — Tips & Tricks rund um Novell DOS 7, mit Blick auf undokumentierte Details, Bugs und Workarounds. Release 157 (in German) (3 ed.). Archived from the original on 2017-09-10. Retrieved 2014-08-06. {cite book}: |work= ignored (help) (NB. NWDOSTIP.TXT is a comprehensive work on Novell DOS 7 and OpenDOS 7.01, including the description of many undocumented features and internals. It is part of the author's yet larger MPDOSTIP.ZIP collection maintained up to 2001 and distributed on many sites at the time. The provided link points to a HTML-converted older version of the NWDOSTIP.TXT file.) [2]
  9. ^ "sendfile(2) - Linux manual page". man7.org. 2021-03-22. Retrieved 2021-10-13.
  10. ^ "splice(2) - Linux manual page". man7.org. 2021-03-22. Retrieved 2021-10-13.
  11. ^ "tee(2) - Linux manual page". man7.org. 2021-03-22. Retrieved 2021-10-13.
  12. ^ "vmsplice(2) - Linux manual page". man7.org. 2021-03-22. Retrieved 2021-10-13.
  13. ^ "process_vm_readv(2) - Linux manual page". man7.org. 2021-03-22. Retrieved 2021-10-13.
  14. ^ "process_vm_writev(2) - Linux manual page". man7.org. 2021-03-22. Retrieved 2021-10-13.
  15. ^ "copy_file_range(2) - Linux manual page". man7.org. 2021-03-22. Retrieved 2021-10-13.
  16. ^ "Linux PACKET_MMAP documentation". kernel.org.
  17. ^ "sendfile(2) - FreeBSD manual pages". www.freebsd.org. 2020-04-30. Retrieved 2021-10-13.
  18. ^ "write(2) - FreeBSD manual pages". www.freebsd.org. 2020-04-30. Retrieved 2021-10-13.
  19. ^ "writev(2) - FreeBSD manual pages". www.freebsd.org. 2020-04-30. Retrieved 2021-10-13.
  20. ^ "mmap(2) - FreeBSD manual pages". www.freebsd.org. 2020-04-30. Retrieved 2021-10-13.
  21. ^ "sendfile(2) - Mac OS X Manual Page". developer.apple.com. 2006-03-31. Retrieved 2021-10-13.
  22. ^ "sendfile(3C) - Solaris manual pages". docs.oracle.com. 2021-08-13. Retrieved 2021-10-13.
  23. ^ "sendfilev(3C) - Solaris manual pages". docs.oracle.com. 2021-08-13. Retrieved 2021-10-13.
  24. ^ "write(2) - Solaris manual pages". docs.oracle.com. 2021-08-13. Retrieved 2021-10-13.
  25. ^ "writev(2) - Solaris manual pages". docs.oracle.com. 2021-08-13. Retrieved 2021-10-13.
  26. ^ "mmap(2) - Solaris manual pages". docs.oracle.com. 2021-08-13. Retrieved 2021-10-13.
  27. ^ "TransmitFile function (Win32)". docs.microsoft.com. 2021-05-10. Retrieved 2021-10-13.
  28. ^ Palaniappan, Sathish K.; Nagaraja, Pramod B. (2008-09-02). "Java zero-copy". developer.ibm.com. Retrieved 2021-10-13.