Exploring Windows 3.x

From Open Watcom

Jump to: navigation, search

Windows 3.x is, thankfully, obsolete. Support for Win16 applications still exists in Windows XP, and of course in all versions of Win9x, but no one develops for this platform anymore, in part because most today's programmers wouldn't know how. Nevertheless, Windows 3.x deserves scrutiny, if for no other reason but to serve as a warning and practical demonstration of what horrors the computing industry is willing to inflict upon the unsuspecting public in the pursuit of a quick buck, though Windows 3.0 was not done with the intent to do so (Microsoft were focusing mostly on OS/2), and the success of Windows 3.0 was completely unexpected.

This article is intended primarily for programmers with good knowledge of the x86 platform but little or no familiarity with Windows 3.x. Some knowledge of x86 CPU architecture is required.

Contents

History

The history of Windows goes back all the way to 1983. Microsoft was one of the first ISVs to develop applications for the upcoming Apple Macintosh (yes, Microsoft wrote Mac apps before they wrote Windows apps). Microsoft saw the potential of the Mac and decided to develop a similar platform for the IBM PC. The Mac group at Microsoft was under a NDA and was not supposed to give out any information about the Mac, but a piece of paper never stopped Microsoft. [1]

After much fanfare and lengthy delays (sound familiar?), Windows 1.0 was unveiled in 1985. It was a flop. Windows 1.0 looked butt ugly with an atrocious color scheme and graphics apparenty drawn by the developers' pre-school children. Windows 1.0 did not support overlapped windows, only tiled (in the early days of GUIs, one school of thought claimed that overlapping windows were a bad thing). Windows 1.0 was purely 16-bit real mode DOS application and thus constrained by the infamous 640K limit (though in 1985, this was not yet too much of a problem). This early version of Windows, as well as Multitasking DOS 4.0, however already introduced the New Executable (NE) format and dynamic link libraries (DLLs). Multitasking DOS 4.0 still used Int 21h calls, but Windows introduced the well-known KERNEL, USER and GDI components with device independent architecture.

One of the early Windows users was Mitchell Waite, the father of Visual Basic. [2]

Windows 2.0 was released in 1987. This release came at a time when the future (according to Microsoft and IBM) was OS/2. Windows was touted as a stepping stone to OS/2. According to some Microsoft sources, version 2.0 was to be the last version of Windows and the major reason for its release was to serve as a vehicle for the Excel spreadsheet (remember that there was a character-mode DOS version of MS Word, but Excel only ever existed as a GUI application).

Windows 2.0 was not significantly different from version 1.0. It was still a real mode application, still ugly, although it did support overlapped windows. The only technically interesting development in Windows 2.x days was Windows/386. This was a version of Windows requiring a 386 (obviously) that supported virtual DOS machines via the V86 mode of 386 processors. In other words, multiple DOS applications could be run at the same time. However, Windows still ran in V86 mode inside one of the virtual DOS machines, called the "System VM".

However, thanks to several Windows apps like Adobe PageMaker and Microsoft Excel, Windows was beginning to catch on with a few users and developers. One of the utilities that was made in this era was FixDS, to patch the Windows prolog so that the SS was used as the DS.

Around 1988, work commenced on two other projects. One of them was OS/2 version 2 (also known as OS/2-386), a 32-bit version of OS/2 for 386 systems; this project was eventually released by IBM as OS/2 2.0 in 1992. The other project was OS/2 NT, a portable 32-bit version of OS/2 supporting multiple CPU architectures and sporting many advanced features; this project was eventually released by Microsoft in 1993 as Windows NT 3.1.

By 1990, the rift between IBM and Microsoft was growing and neither company was very happy with the other. A small group of programmers at Microsoft decided to create a protected mode version of Windows, just to see how it worked. It kind of did and the management was excited. The project got a green light and in 1990, Windows 3.0 was released, which were the time Windows got really popular. Soon all major ISVs were busy creating Windows versions of their flagship applications. The rest, as they say, is history.

Windows 3.0 Overview

Windows 3.0 supported PCs from lowly XT up to the latest 386s and 486s. Windows 3.0 could be run in three different modes. Real mode worked on 8088 and 8086 CPUs which did not support protected mode. This mode was very similar to Windows 2.x (and compatible with Windows 2.x applications). There was a provision to swap to EMS (if available) but most old-timers agree that Windows 3.0 was essentially unusable on pre-286 PCs due to utter lack of processing power.

Where things got interesting was the Standard mode, intended for 286 or newer processors. Standard mode ran in protected mode and had one major advantage: It wasn't limited to using 640KB, like MS-DOS applications at the time were. A maximum of 16MB RAM could be directly addressed (no one had that much memory back then) and the CPU provided support for swapping of memory segments to disk. The other advantage was somewhat hypothetical. Certain protection violations could be detected and recovered from -- in theory. In practice, Windows 3.0 was the Age of the UAE (Unrecoverable Application Error). Errors could indeed be detected but recovery was mostly wishful thinking of the marketing department, though Windows itself could -- in some cases -- recover by terminating the application.

The most interesting was the Enhanced mode, requiring a 386. For native Windows applications, this mode was not very different from Standard mode, although rudimentary support for 32-bit memory access was provided (more on this later). The big improvement was support for multiple DOS boxes and virtual device drivers (VxDs, usually files with a .386 extension). Virtual device drivers were 32-bit modules, notoriously difficult to write and correspondingly prone to crashes, used to extend the functionality of Windows and DOS in many ways. As well as being the shape of things to come with Chicago (i.e. Windows 95).

DOS Extenders

Windows 3.x was, among other things, a DOS extender. One key development related to Windows 3.0 (and OS/2 2.0) was DPMI (the DOS Protected Mode Interface). Ever since PCs equipped with 286 and especially 386 CPUs started appearing, there was enormous interest in utilizing the Protected Mode capabilities of those processors but without giving up the popular DOS platform.

DOS extenders solved this problem. A DOS extender was a relatively small piece of code which could switch the processor to protected mode, load a 32-bit (or, in some cases, 16-bit) user application, and provide a number of translation services between protected and real mode. These services included memory management, interrupt handling, DOS and BIOS calls, etc. Some popular early extenders were PharLap 386|DOS and 286|DOS, Rational DOS/16M and DOS/4G, Ergo OS/286 and OS/386.

The problem with DOS extenders was that they were totally incompatible with each other, as well as with advanced operating systems. Early extenders took full control of the CPU and ran at ring 0 (most privileged protection level). Such DOS extenders were also incompatible with EMM386 and other memory managers (which had to use V86 mode). Those who remember Ultima VII know this only too well (though it switched to unreal mode, not protected mode, it had to switch to unreal mode through protected mode and so posed similar issues).

The first stab at a solution was VCPI, a standard for switching between real and protected mode. VCPI allowed DOS extenders to coexist with 386 memory managers. However, VCPI did not solve interoperability with platforms such as the upcoming Windows 3.0 and OS/2 2.0.

DPMI provided a far more comprehensive solution. A DPMI server was the controlling piece of software which handled mode switches, memory allocation, interrupt management, virtual memory, etc., etc. A DPMI client called DPMI services (through interrupt 0x31) and did not directly manipulate the CPU. A key property of DPMI clients was that they could run at ring 3 (least privileged protection level) and hence could be executed under Windows 3.0, OS/2 2.0, or Windows NT.

It is important to realize that a typical DOS extender was both DPMI server and DPMI client. If there was a DPMI server already present, the DOS extender would simply use the existing DPMI services. If no DPMI server was provided, however, the DOS extender would act as a DPMI server and manage the processor and memory on its own.

Windows 3.x was a DPMI server and provided DPMI services for both DOS and Windows client applications.

Windows 3.x Architecture

The architecture of Windows 3.x was bizarre to say the least. This was largely a consequence of the requirement for compatibility with Windows 2.x applications.

From the CPU's point of view, all Windows applications (tasks in Windows 3.x parlance) ran inside a single process. There was only one LDT shared by all Windows tasks and consequently there was absolutely no protection from having the memory owned by one task overwritten by any other task! While this made memory sharing trivial, it also greatly contributed to the instability of Windows.

Multitasking was cooperative, just like in old versions of MacOS (though unlike Multitasking DOS 4.x, which sported preemptive multitasking). A task had to yield to the OS in order to schedule a different task. The yields were built into certain API calls, notably message processing. As long as a task processed messages in a timely manner, everything was great. If a task stopped processing messages and was busy executing some processing loop, multitasking was no more.

Memory management was rather interesting. Windows had a relatively sophisticated heap manager, just like Multitasking DOS 4.0. There were two heaps, local and global. The local heap was a part of the default data segment (DGROUP) and hence usually small, rarely over 4-16KB or so. Only near pointers were needed to access the local heap. Global heaps used far pointers and could thus be much larger. Huge memory objects were supported, ie. allocation of several consecutive 64KB segments.

An important feature of Windows 3.x programming were callbacks. Callbacks were functions called by Windows, typically windows or dialog procedures. Callback functions needed special handling by the compiler, which will be discussed later.

It is important to keep in mind at all times that Windows 3.x was not a full operating system. Windows ran on top of DOS and while DOS was circumvented in some cases (eg. input and output), Windows relied on DOS for file management. The only exception was Enhanced mode where disk access and later (in Windows 3.11 for Workgroups) also the filesystem could optionally be handled by 32-bit VxDs. There was an incestuous relationship between Windows and DOS where Windows relied on numerous internal DOS structures and was able to provide the illusion of multiple instances of DOS for concurrently executing tasks.

Consequences of Bizarre Architecture

The memory layout of Windows 3.x seriously complicated programmers' lives. There was an understandable desire to conserve memory and share code segments among multiple instances of tasks. However, because there was only one LDT (and obviously only one GDT, too), different instances of a task had to have different selectors for their respective default data segments (DGROUP). This meant that the code segment could not contain relocations to data segment.

Microsoft's solution was special function prologs and epilogs. This obviously required special compiler support. If Microsoft hadn't had their own compiler then, they probably would have had to come up with some less crazy method. The Windows prologs and epilogs looked like this:

push    ds      ; Could be 'mov ax,ds' instead of
pop     ax      ; these two instructions.
nop
inc     bp
push    bp
mov     bp,sp
push    ds
mov     ds,ax
.
.
.
pop     ds
pop     bp
dec     bp
retf    n

The inc bp was not strictly required in protected mode. It was needed for Real mode; Windows (or Multitasking DOS 4.x) could move around segments and tweaking the bp value on stack served as a marker to help Windows keep track of things.

The rest were just hoops that code needed to jump through in order to get the right data segment value. The nop instruction served as a placeholder so that Windows could diddle with the prolog and replace it with a single mov instruction.

In the Windows 2.x era, some thinking programmer realized that hey, we know what the data segment is -- it's the same as the stack segment (for tasks -- not true for DLLs!). The first three instructions of the prolog could be replaced with mov ax,ss. FixDS was a utility that did this. Note that these prologs and epilogs were not required for every function, only for functions that Windows could call directly.

For DLLs, the situation was slightly simpler, or maybe more complicated. The prolog for DLL exported function was different and used mov ax,DGROUP to get the data segment value. Because the DLL code segment was shared among all tasks using the DLL, a Windows 3.x DLL could only have one data segment (DGROUP) shared by all instances. Needless to say, this made DLL writers' lives somewhat difficult.

Heap management in Windows 3.x (and Multitasking DOS 4.0) was likewise full of intrigue and adventure. As noted above, Windows could move segments around, which in Real mode meant that their address changed (in protected mode, the segment could physically move without changing the selector value). Thus global heap allocations did not return pointers but rather handles. When an application wanted to access the memory, the heap had to be locked; this converted the handle to a real pointer. After use, the heap needed to be unlocked so that Windows (or Multitasking DOS 4.0) could move it someplace else.

It is not surprising that soon after Windows 3.0 was released, most programmers gave up on Windows Real mode. It was far too much pain for the meager gain of supporting systems where Windows 3.0 was almost completely unusable to begin with. Many applications even required the Enhanced mode and a 386.

Differences Between Windows 3.x Versions

Windows 3.1 was released in 1992 and although it was not architecturally noticeably different, it was an improvement. Real mode was no longer supported. Multimedia Extensions were built in, where they had to be added to Windows 3.0 (and Windows 3.0 with Multimedia Extensions was called just that). Windows 3.1 also supported TrueType fonts, which were a significant improvement over fixed-size bitmap and simple scalable vector fonts of earlier Windows versions.

Windows for Workgroups 3.1 added built-in networking and notably supported networking VxDs to replace real-mode components and reduce mode switches when communicating over a network. In WfW 3.11, standard mode support was dropped and therefore 386 or later was required. The system architecture was moving towards Chicago (also known as Windows 95) and VxDs became commonplace. WfW 3.11 completely removed real-mode dependencies for networking. This (together with the release of Windows NT 3.1) required a new version of NDIS, called NDIS 3.0, which led to NDIS 3.1, which was used in Windows 95 and Windows NT 3.5x.

It is a little known fact that there were four versions of Windows 3.1: The original Windows 3.1, Windows for Workgroups 3.1, Windows for Workgroups 3.11, and Windows 3.11. Windows 3.1 and WfW 3.11 were far more widespread than the other two releases, though Windows 3.11 was also released as a patch to Windows 3.1.

The Windows 3.1 family timeline was as follows:

  • Windows 3.1 - released April 6th, 1992. Initial release.
  • Windows for Workgroups 3.1 - released October 1st, 1992. Windows 3.1 + NDIS 2.0 networking.
  • Windows for Workgroups 3.11 - released November 1st, 1993. Added NDIS 3.0 VxD-based networking.
  • Windows 3.11 - released December 31st, 1993. A few fixes, updated drivers and additional NetWare support.

Windows Extenders

While Windows 3.0 alleviated the 640KB memory limitation, it was still a 16-bit platform and did nothing about the 64KB segment size restriction. There was demand for a 32-bit programming environment.

Microsoft's eventual solution was Win32s, intended as an entry into the promised land of Win32. There were several other solutions released much earlier, around 1990 or 1991. One of them was Watcom's Win386 extender.

Windows extenders took the idea of building a 32-bit environment around a 16-bit kernel from DOS extenders and applied it to Windows 3.x. Watcom Win386 (still available in Open Watcom) provided the ability to write 32-bit flat model applications which run on top of Windows 3.x. Unlike Win32s, Win386 did not require any separately installed runtime and was completely contained in the application's executable. Some programmers claim that Win386 was significantly easier to work with than Win32s.

Win386 provided thunks and "cover functions" for the entire Windows 3.x API. Where Windows used 16:16 far pointers in function arguments and structures, Win386 used 32-bit near pointers. Code could typically be written so that it could be built as both 16-bit or 32-bit. Callback functions needed special attention because they required thunks. For any callback function, Win386 had to build a thunk which would be called from 16-bit Windows code, translate any parameters, and call the 32-bit user code. For API calls, cover functions provided the inverse functionality in translating 32-bit addresses to 16-bit and calling 16-bit code.

Win386 is also compatible with Win9x and Windows NT, as well as with Win-OS/2. The only tricky part is debugging, which requires a VxD and hence a real Windows 3.x or 9x installation.

Comparing Windows 3.x and OS/2 1.x

It is interesting to compare and contrast Windows 3.x and OS/2 1.x. Microsoft was heavily involved in OS/2 1.x development and there are many similarities, as well as striking differences, between the two platforms.

OS/2 was intended to be a DOS replacement and was initially named DOS 5 or CP-DOS (Control Program DOS, a pun on PC-DOS). OS/2 however was certainly not DOS. While version 1.0 had no GUI and the command line was very reminiscent of DOS (most commands were the same), the OS was nothing like DOS. OS/2 was a protected mode operating system and hence required a 286 or later CPU. OS/2 provided virtual memory management (segment swapping), preemptive multitasking, and multithreading. In 1987, it was a highly advanced operating system. The major weakness of OS/2 was poor compatibility with DOS. A 286 had no V86 mode, therefore only one DOS application was supported and switching to real mode was required. Many DOS applications did not run in the OS/2 "penalty box" and those that did could easily compromise system stability.

Microsoft promoted "Family API" (FAPI), a way to write dual-mode DOS and OS/2 applications, but this technology never really caught on outside Microsoft. FAPI applications were NE modules with a DOS stub which contained a miniature OS/2 API emulator. On OS/2, only the protected mode portion was used, on DOS the stub provided OS/2 API functionality.

Starting with version 1.1 (1988), OS/2 included a GUI (Presentation Manager or PM). The PM API was essentially Windows API version 2.0, with identical concepts but more refined abstraction and significantly more capable graphics subsystem.

While OS/2 used the exact same NE format for its executables like Windows, the programming model was considerably cleaner because its design was not constrained by backwards compatibility considerations. OS/2 had real processes with separate LDTs, which meant that multiple instances of an application could use data selectors which were numerically identical but pointed to different memory. Thus there was no need for funky function prologs. Only exported DLL functions needed special prolog to load the DLL's DGROUP selector (instead of the caller's), usually requested by _loadds keyword.

Thanks to multiple LDTs, DLLs on OS/2 1.x weren't crippled and the programmer could choose whether DLL default data segment should be shared among processes or different for each process. DLLs on OS/2 could be used much like static libraries without worrying about multiple processes using them.

Resources

The Watcom Heapwalk applet (wheapwlk) provides excellent insight into Windows memory management. While it is probably overwhelming for Win16 neophytes, it should prove very handy for users with some background in Windows 3.x internals. It supports viewing of any block of memory with the ability to disassemble code segments or display resources in appropriate format.

Dr. Watcom for Windows 3.x is a debugger-like utility which gains control in the event of application fault. It can provide significant help in crash diagnosis and also offers the option to manipulate registers and restart instructions for the adventurous. It is very useful in situations where a full-fledged debugger can not be easily used.

The book Windows Internals: The Implementation of the Windows Operating Environment by Matt Pietrek provides excellent in-depth description of Windows 3.x architecture -- tasks, modules, message queues, windows, memory management, the works.

The book Undocumented Windows: A Programmer's Guide to Reserved Microsoft Windows Api Functions by Andrew Schulman, David Maxey, Matt Pietrek contains a wealth of information that cannot be found in the Windows SDK documentation and explains much of the Windows functionality. A must for Windows 3.x debugger writers.

Larry Osterman's blog contains some information on Multitasking DOS 4.0

Personal tools