memory(5) UNIX System V (Concepts) memory(5) NAME memory - virtual, real and paging memory management DESCRIPTION malloc(3) and sbrk(2) return a pointer to a virtual address range validated by the virtual memory manager in the kernel for use by a process. Depending on the operating systems policies, real memory and paging resources may be reserved or allocated when the virtual address range is validated for a process. The virtual to real address memory address hardware maps the addresses generated by the instructions (the virtual addresses) to addresses used by the memory of the system (the real addresses). The translation lookaside buffer (TLB) expedites the translation. If a translation is not available in the TLB, a interrupt occurs. The slot in the TLB is freed on a least recently used (LRU) basis and the needed translation is installed from table managed by the virtual memory manager. The interrupted instruction is then retried when the process regains control of the CPU. The virtual storage manager tracks which virtual addresses are valid in each process and manages the virtual to real memory address translation hardware. The virtual memory manager is invoked by the sbrk(5) et al. system calls and by other kernel services which allocate virtual memory on behalf of the process such as memory mapped files and shared memory areas. If the requested amount of virtual address range can not be validated, the system calls immediately return failure (e.g., null pointer). When availsmem_accounting is false (zero), that is all these system calls do. Otherwise, the virtual storage manager also subtracts the number of virtual address page validated from the number of available real memory page frames and swap slots. If there is not sufficient resources, the virtual address range is invalidated and the system call returns failure. The resources are released when either the system calls are used to release them or the process terminates. The virtual storage manager is invoked by the kernel when a page fault occurs. If the concerned virtual address is invalid, then the process is sent a SIGSEGV. If the concerned virtual address valid, then the virtual memory manager determines whether the page is backed. If the page is not backed, the the virtual memory manager requests an available real memory page frame from the real memory manager and uses that to back the address. Most modern UNIX variants zero such pages for security reasona. If the page is backed by a real memory page frame, that page frame's Page 1 (printed 12/13/97) memory(5) UNIX System V (Concepts) memory(5) real address is used to set the TLB entry. If the page is backed by paging space, then the virtual memory manager requests that the paging supervisor obtain a real memory page frame and read the paging slot into that page frame and return it. The process sleeps while this I/O occurs. The virtual memory supervisor arranges that the page faulted instruction will be appropriately restarted. The virtual to real address memory address hardware maps the addresses generated by the instructions (the virtual addresses) to addresses used by the memory of the system (the real addresses). The translation lookaside buffer (TLB) expedites the translation. If a translation is not available in the TLB, a interrupt occurs. The slot in the TLB is freed on a least recently used (LRU) basis and the needed translation is installed from table managed by the virtual memory manager. If a virtual page is valid, not backed by a real memory page, but does have a backup in the swap area, a real memory page is made available, the data read into the page (the process sleeps while the I/O is done so that other processes can run). The real memory manager attempts to keep a pool of real memory page frames available by looking for pages that have not been referenced in a "long" time, assigning a swap slot, writing the pages contents to the swap slot. If a virtual page is valid and backed by a real memory page, that page is made available and its contents used. This can result from an LRU swap-out of a page frame that was not needed before it is referenced by the original owner. This is frequently called "page frame reclaim." The hardware maintains a reference bit and a change bit for each real memory page. The refernce bit is turned on if the page is either read or written. The change bit is turned on if the page is written, Periodically, the system reads and zeros the reference bit and tallys a counter for each referenced real memory page frame. These data are used to determine the least-recently-used (LRU) page frames. The change bit is used to assure proper writes and to determine if a real memory page frame that also has a back up in the swap space has a valid backup. The change bit is turned off when the write is scheduled. If it is still off when the write is done, the operation is successful; otherwise the swap slot is released for reassignment. The real memory page will also be removed from the pool to be swapped out because it had to have been recently referenced. When a page is read in because of a page fault, after the read is complete the change bit is reset but the swap slot is not Page 2 (printed 12/13/97) memory(5) UNIX System V (Concepts) memory(5) released. If the page re-enters the LRU pool and is still unchanged and still has its swaps slot, no write needs to be done. (This happens if the page was only read since the swap-in.) The swap space manager attempts to maintain a list of available swap slots. It periodically scans the change bits of real memory page frames with assigned swap slots, releasing the swap slots of those that have changed. If this does not find a sufficient number of available slots, a second scan of the same pages is done on a LRU basis until a suffient number is found. It is possible that neither a sufficient number nor any swap slot can be made available. If the real memory manager needs a swap slot to write a page to because of a page fault and there are no swap slots available, the system has to terminate a process (the famous `Killed') to make both its real memory page frames and its swap slots available. Every version of the operating system has its own policy on how to select the victim. Some sample policies are kill the process with the page fault or kill the with the most of some resource -- e.g., virtual address space, real memory or swap space. The combined behavior of the virtual memory manager, the real memory manager, and the swap manager causes a dense use of real memory of the system regardless of the users' pattern of usage of their virtual address spaces (each process has its own virtual address space). Because the granularity of the system facilities is a page and the granularity of the sparse array codes is usually smaller, less memory is lost to fragmentation, but more may be lost to the overhead needed to manage the small units, in the sparse array codes. To be fair, a similar trade-off occurs for page size. Small pages require more memory to manage it but there is less fragmentation. Deterministic behavior requires all virtual memory to have a reservation or allocation of either real memory, paging space or both. Program text pages which are read-only may be backed by the executable file; but, because of the difficulties with virtual file systems in System V Release 4 and after, many system do not do this. Dense usage of system resources requires that backing storage, either real memory or paging space not be allocated until it is needed. This means that a process may be terminated because it requires resources that are not available. fork system calls used to copy the entire memory image of Page 3 (printed 12/13/97) memory(5) UNIX System V (Concepts) memory(5) the process. After a fork system call frequently the child process executes a program via an exec system call releasing the copied memory. By using the hardware facilties to prevent writing to memory and copying only the virtual memory management data, it can be arranged that only those pages that the parent or child change before the child execute a program need to be copied. This is called `copy on write.' When a page is not longer be shared it can be allocated to the last process that needs it. AUTHOR Randolph J. Herber. Page 4 (printed 12/13/97)