From Open Watcom
It is not usual that one of the many address lines of a CPU becomes famous, yet IBM managed to do just that with the A20 address line of the IBM AT and compatible computers.
As usual, the reasons for the A20 mess are historical. The 8088 and 8086 CPUs had only 20 address lines and were thus able to address 1MB of memory (with byte granularity). But as everyone knows, those CPUs actually use two 16-bit registers (segment and offset) to form the 20-bit address. The segment value is shifted left four bits and added to the offset.
This has two implications. First, there are multiple ways to address a single physical memory location. For instance addresses 0:400, 40:0, and 10:300 all address physical memory at offset 400h.
Second, it is possible to use "impossible" combinations of segment and offsets, such as FFFF:FFFF. This translates to address 10FFEFh, only the 8088/8086 silently chops off the top bit (those CPUs were remarkably bad at error diagnosis and handling) and accesses memory at address FFEFh. Now, there is no overwhelmingly good reason to access the first 64KB of memory in such a convoluted way, but some programs did.
The designers of the IBM AT decided that by default, the AT should emulate the earlier 8088-based PC and XT computers and should provide address wraparound. To do that, it was enough to have the single address line A20 tied to zero; all of the addresses in the 1MB to 1MB+64KB range then effectively have 1MB subtracted and address the first 64KB of memory.
This approach provided compatibility with existing strangely written PC/XT code. One of the problematic pieces of code was reportedly the executable decompression routine for executables built using the EXEPACK option of older versions of Microsoft's linker. If address wraparound wasn't enforced and if it was being loaded at an address under 64KB (that was why this bug was hidden for so long), a 'Packed File corrupt' message could result when attempting to run a compressed executable. The linker shipped with Microsoft C 6.0 fixed this bug.
The problem is obvious. The fact that the 80286 had a larger address space (24 bits, or 16MB) was one of its major new features. But to access that memory, address line A20 had to work just like all the others.
Every software wishing to switch to protected mode must ensure that the A20 gate is on, ie. the A20 address line functions normally. In addition, some memory managers and newer versions of DOS support so-called High Memory Area or HMA. This is the first 64KB above 1MB. Clever people have noticed that if the 8088-compatible address wraparound isn't emulated, it's actually possible to address (very nearly) 1MB+64KB memory on 286 and later CPUs without ever leaving real mode. This additional 64KB was quite valuable when 640KB was the usual limit. But for the HMA to be accessible, the A20 address line again had to be enabled.
The Solution and More Problems
To solve this dilemma, IBM added a new mechanism to control the behaviour of the A20 line in the AT, a so-called A20 gate. This gate either allows the A20 address line to function normally or forces the A20 line to be always zero. For reasons of expediency, this gate was accessed through the keyboard controller, just because the keyboard controller (Intel 8042, a generic programmable micro-controller) had a few spare output pins left.
There were two problems with this solution. Unless one used the BIOS (and many couldn't or wouldn't), operating systems and DOS extenders had to know how to control the A20 gate and had to be able to turn it on and off. On top of that, the A20 gate control through the keyboard controller wasn't especially fast, as the keyboard controller was really meant to control, well, a keyboard -- which in computer terms is a fascinatingly slow and quiet input device. The designers of the AT didn't expect this to be an issue, as they (very naively, as it turned out) assumed that people switching to protected mode would generally stay in protected mode -- especially when the 286 couldn't even directly switch from protected to real mode, another long story -- and that the speed of switching to protected mode would be largely unimportant.
No one expected the tenacity of DOS and that it would remain to be a major force well into the 1990s. A DOS extender may switch between real and protected mode hundreds or thousands of times per second, and clearly these switches have to be fast. This in turn led to other, faster, incompatible ways of controlling the A20 gate.
In hindsight, it was a mistake not to provide a BIOS service to control the A20 gate in the AT BIOS. That would have avoided most of the problems. The BIOS had to program the gate when switching to protected mode and when copying extended memory, so a routine to control the A20 gate existed in the AT BIOS but wasn't exposed as a separate service (that only happened much later when it was too late).
There are at least three common ways to control the A20 gate, and several other more exotic ways.
The original IBM AT used, as noted above, the keyboard controller. The A20 control was one of the pins on the 8042 keyboard controller's output port P2. To cleanly change the A20 state, one had to send the D0h command to the keyboard controller, read the current output port state, set or clear the A20 control bit (bit 1), send the D1h command, and send the new output port value.
HP Vectra machines introduced an improved version which was perhaps implemented simply by updating the keyboard controller firmware. Keyboard controller commands DDh and DFh could be used to disable or enable the A20 address line without affecting anything else. This avoided most of the slow communication with the keyboard controller. This method works on many modern PCs.
IBM itself used a different approach in its PS/2 line of systems (introduced in 1987). The PS/2s included a new System Control Port B accessible at port 92h. Bit 1 of this port controlled the A20 line. Accessing this port was considerably faster than communicating with the keyboard controller. Again, this method is available on many modern PCs and is sometimes referred to as 'Fast A20 Gate'.
On 286 and 386 systems, the A20 line was handled solely by circuitry external to the CPU. The motherboard chipset could manipulate the address lines and the CPU didn't need to know.
On the Intel 486 CPU, this was no longer possible; because the CPU has internal cache, it must know whether the A20 line is masked or not. For this purpose, the 486 and later CPUs have an #A20M (A20 Mask) pin. Modern chipset also usually provide two methods of controlling the A20 gate: the original keyboard controller method and the PS/2 style port 92h method (see above). If fast A20 gate control via port 92h is enabled, it usually overrides the keyboard controller; that is, if the A20 line is enabled via port 92h, the keyboard controller state is irrelevant.
Programs naturally tried to use the fastest A20 control method available. It is possible to safely determine the current A20 gate setting (by checking whether eg. addresses 0:0 and FFFF:10 map the same memory or not) but not every program was coded defensively enough, and attempting to use unsupported A20 control methods may have undesirable side effects. Clearly, failing to enable the A20 line before switching to protected mode is likely to be fatal, although not disabling it when switching to real mode may not have any negative impact.
For instance HIMEM.SYS in MS-DOS 6.22 provides 17 methods (many only differing by timing) for controlling the A20 line. While the driver is almost always able to automatically detect the correct method to use, this isn't possible in 100% cases. Similarly DOS extenders (such as DOS/4GW) tend to provide ways to override the A20 gate control method selection.
For 'real' operating systems such as Windows or various UNIX systems and clones, A20 switching is usually not much of an issue. Since they only switch to protected mode once during startup, they can use the most compatible AT A20 control method, or they could call the BIOS to perform the switch (and enable the A20 line). Unfortunately, some of the newer PCs have eliminated the 8042 keyboard controller (such as the Intel Macs), and so could not control the A20 line using the keyboard controller. One of the newer methods have to be used. GRUB for example needed a patch before it worked on the Intel Macs.
A memory manager or a DOS extender is in a completely different situation. It potentially needs to switch the A20 line on and off many times a second, hence the method has to be reliable and fast. Newer extenders usually need not worry about A20 control too much as the machines with exotic A20 control mechanisms are long gone, but older ones tend to provide a wider variety of A20 control types.
The quest for backwards compatibility sometimes creates more problems than it solves.