Please consider a donation to the Higher Intellect project. See or the Donate to Higher Intellect page for more info.

PC Memory Management Overview

From Higher Intellect Vintage Wiki
Jump to navigation Jump to search

DOS users are faced with a confusing array of memory configurations today. There's conventional memory, upper memory, expanded memory, extended memory, and the high memory area to name a few. Plus there are acronyms like UMB (Upper Memory Block), EMS (Expanded Memory Specification), XMS (eXtended Memory Specification), HMA (High Memory Area), VCPI (Virtual Control Program Interface) and DPMI (DOS Protected Mode Interface), which confuse matters even further.

In this document, we'll try to explain the PC Memory Architecture, as well as popular memory management techniques and standards.

The reason for all of these different types of memory has to do with the evolution of the Intel 80x86 family of microprocessors over the years. Since the 8086 and 8088 microprocessors, used in the original IBM PC and "XT-class" machines, we've seen tremendous growth in the speed and capabilities of PC microprocessors, yet limitations of the original 8086 design still hold us back. The primary limitation is the ability of the 8086 and 8088 processors to only address a 1MB range of memory...which seemed like a lot for their time, when 64KB was the extent of the addressing capability of competing non-Intel microprocessors.

When IBM designed the original PC, they reserved the upper 384KB of this 1MB for the PC BIOS (Basic Input/Output System), video memory, and for adapter boards to install additional RAM, allowing applications to write directly to added RAM in order to communicate with the adapter. This left us with 640KB for DOS and application programs.

The 80286 microprocessor supports a full 16MB address space, however this additional memory is only accessible when the 80286 operates in what is termed as its protected mode. The default mode of operation for the 80286 is real mode, which is 8086 compatible. In real mode, the 80286 is little more than a fast 8086.

At the time that the 80286 was being designed by Intel, they had no idea that the PC and the 8086 would enjoy widespread popularity, and hence, the design of 80286 protected mode is very incompatible with real mode. Intel anticipated that since protected mode offered so many advantages over real mode, that real mode would just "go away".

While there is a processor instruction to put the processor into protected mode, there is no way to return! Applications like non-dedicated NetWare 2.x, OS/2 v1.x and the various DOS extenders, have to play some real feats of magic in order to switch between real and protected modes.

Mostly, the switch from protected mode back to real mode is done by performing a hard reset on the microprocessor (CTRL-ALT-DEL like). Before the processor is reset, a command is sent out to the keyboard controller. The response back from the keyboard controller is what wakes the processor back up from its reset, now in real mode. This is the reason why people often report sluggish keyboard problems with non-dedicated NetWare. Not all keyboard controllers are designed to support this operation.

The 80386 added some exciting new features. In addition to supporting 8086 real mode and 80286 protected mode, the 80386 supports even more than 16MB, 4GB of physical memory, and 1TB of virtual memory. Thanks to a flat 32-bit address space, all physical memory is accessible in one flat address space (programmers are often hampered by 64KB at a time addressing limits on the 8086 and 80286).

One of the real exciting enhancements of the 80386 is its virtual 8086 mode. Through the 80386's hardware features, it is able to emulate multiple 8086 microprocessors! This is how DesqView, Windows enhanced mode, and OS/2 v2 perform their magic multi- tasking DOS applications (with varying degrees of success).

The 80386 also supports memory paging in its hardware. Applications only see a logical view of memory. When an application makes a memory request, the processor redirects the access to the actual physical location. The 80286 also provided a limited subset of memory paging support in protected mode only (on the 80286, paging is on a per 64KB segment basis, the 80386 pages on 4KB boundaries), but the 80386 makes this available in virtual 8086 mode as well, so that memory management products like QEMM, 386-to-the-MAX and MS-DOS 5/DR-DOS 6 EMM386 can perform memory management magic for DOS applications.

Since we're talking about microprocessors for a background, I'll also mention the 80486. Effectively, the 80486 is a highly- tuned, highly-optimized 80386, which doesn't add many new capabilities, mostly just FASTER execution than past designs of this family. The 486 also includes an integrated math coprocessor (there is no need for an 80487) that really speeds up number crunching. The 486SX is essentially a crippled 486 without the built-in math coprocessor.

For the sake of completeness, I'll also mention the 386SX. The 386SX is software compatible with the 80386, except that all external accesses that it makes to the outside world are made 16 bits at a time instead of 32 bits, which makes it a tad slower than a full blown 386 when it comes to accessing memory and devices. Similarly, the 8088 is an 8086 compatible processor, except that it makes its external accesses 8 bits at a time instead of 16 bits at a time. While this type of design makes the system a little slower, it allows for lower priced components to be used in building the system, making enhanced processor capabilities available in machines at lower price points.

With a history of the Intel family of processors in mind, let's move on to the different types of memory.

Conventional Memory is the memory range between 0 and 640KB, which is directly accessible to DOS applications. DOS and TSRs begin loading at the bottom area of memory and work their way up.

Upper Memory (UMB for upper memory blocks) is memory between 640KB and 1MB. This memory begins at segment A000h, and is directly accessible from real mode, just "reserved" by the original PC design.

Video buffers comprise the first 128KB of memory. The first 64KB (A000-AFFF) is typically used by EGA/VGA graphics modes. The next 32KB is the monochrome video text buffer (B000-B7FF), followed by 32KB for the color text video buffer (B800-BFFF). (The different buffer addresses were intended to allow color and monochrome systems both reside in the same system.)

Other ranges of upper memory are used by other adapters (like network cards), and the PC's BIOS. The BIOS area is normally the top 64KB (F000-FFFF), although some systems use a 128KB BIOS (like PS/2s) (E000-FFFF).

The left-over memory can be used for various purposes. On a 386 system, the 386 processor's paging techniques are used to map extended memory (memory beyond the 1MB limit, we'll get to it shortly) into the upper memory area, so that it can be addressed by the processor in real mode (or really virtual 8086 mode). On a 286 system, an expanded memory board can be installed, which installs physically addressable memory into this range of upper memory.

Upper Memory blocks, or UMBs, are used for loading TSR programs "high", such as with the MS-DOS 5 LOADHIGH and DR-DOS 6 HILOAD commands. Memory managers for the 80386 can map memory into the upper memory area so that TSRs can be loaded outside of the standard 640KB. It is interesting to note that LOADHIGH and HILOAD load programs into upper memory, and not the actual area of memory known as the High Memory Area (HMA), which we will cover shortly.

The memory manager remaps exTENded memory for use as upper memory. On a 286 or 8086 with exPANded memory hardware supporting the LIM EMS 4.0 specification, Quarterdeck's (the QEMM developers) QRAM product can convert exPANded memory into upper memory.

ExPANded memory itself is actually addressed out of upper memory. In most implementations, 64KB of the upper memory area is set aside as the expanded memory page frame. Applications can then map in 64KB of expanded memory at a time, requesting different "pages" of expanded memory. (The LIM EMS 4.0 specification can get a little more flexible than this.) Usually, the expanded memory (EMS) page frame is located at segment D000h.

ExTENded memory is memory that is not addressable by an 80x86 processor in real mode. It is memory above the 1MB boundary...accessible only from protected mode (or made accessible to other applications in virtual 8086 mode on a 386 through paging), which means it is only of real use to protected mode applications like those built with a DOS extender (Lotus Release 3.x), non-dedicated NetWare, OS/2, and others.

A memory manager like QEMM or 386-to-the-MAX or MS-DOS 5/DR-DOS 6 EMM386 can convert extended memory into expanded memory and/or upper memory through 80836 memory paging techniques.

Ironically, the VCPI (Virtual Control Program Interface) is used by memory managers and DOS Extenders to convert this exPANded memory back to exTENded memory, for programs like Lotus 3.x, which require exTENded memory instead of exPANded memory. Essentially, VCPI (supported by QEMM and 386-to-the-MAX, and available only on 80386 and above processors), is intended to give the application the type of memory that it wants...exPANded memory or exTENded memory. Rather than setting aside a pre- allocated amount of each type of memory, both exPANded and exTENded memory are allocated from the same pool.

DPMI (DOS Protected Mode Interface), is similar to VCPI in concept, but different in implementation. DPMI, supported by Windows 3.x, updated versions of QEMM and 386-to-the-MAX, and several popular DOS extenders, is designed to allow applications running in protected mode access to DOS and BIOS services. As DOS is a real mode operating system, it requires extensions to allow protected mode applications access to DOS services.

Real mode DOS applications can also access extended memory by transferring data in and out of extended memory (which involves toggling the processor between real and protected modes with a little help from the BIOS). These days, most applications that do this use the XMS specification. QEMM and 386-to-the-MAX provide this support, as does Microsoft HIMEM.SYS. The XMS specification is a specification to provide some sort of memory management over extended memory.

There is a special area of exTENded memory, called the High Memory Area (HMA), which is the first 64KB of extended memory, which can be directly accessed from a DOS without the processor being in protected mode.

The HMA is used by MS-DOS 5 when DOS=HIGH is specified in the CONFIG.SYS file to load part of DOS into the HMA, making more conventional memory available to applications. Similar support is provided in DR-DOS by HIDOS.SYS and/or the EMM386.SYS /B=FFFF command-line option and HIDOS=ON in the CONFIG.SYS.

For versions of DOS prior to MS-DOS 5, Novell's XMSNETX shell also makes use of the XMS HMA to load part of the NetWare shell outside of conventional memory. With MS-DOS 5 or DR-DOS 6, it is typically better to let DOS use the HMA as it will give you better performance and more memory.

Here's a memory chart to help you out...

       +------------------+ 16MB and beyond
       |                  |
       |   EXTENDED       |   Addressable in protected mode, not real mode
       |   MEMORY         |
       |                  |
       |------------------| 1MB + 64KB (- 16 bytes if you're picky)
       | EXTENDED MEMORY  |   first 64KB of exTENded memory
       |    HMA           |   used by MS-DOS DOS=HIGH or XMSNETX shell
       |------------------| 1MB
       | UPPER MEMORY     | (LOADHIGH into unused blocks)
       | PC BIOS          | (usually F000h-FFFFh, sometimes E000h-FFFFh)
       | Expanded Memory  | (usually D000h-DFFFh)
       |  Page Frame      |
       | Network Cards    | (Varies)
       |  & other RAM/ROM |
       | Video Buffer     | (A000h-BFFFh)
       |------------------| 640KB (segment A000h)
       |                  |
       |  CONVENTIONAL    |
       |  MEMORY          |
       |                  |
       |  Applications    |
       |                  |
       |  TSRs            |
       |  DOS             |
       |  BIOS data area  |
       +------------------+ 0KB (bottom of memory)


Well...the PC memory architecture is confusing.

In real terms, what all of this means is that 80386 memory management is your best bet, and why the 80286 and below machines are dead-end products for the future...they lack the extendibility and flexibility of the 386 architecture. 386 and 486 computers can turn exTENded memory into exPANded memory and Upper memory for applications that use those types of memory, and still support applications that use exTENded memory...making memory configuration issues far simpler.

- Brett Warthen, Infinite Technologies (Last Revision: April 15, 1992)