Writing DLLs

From Open Watcom

Jump to: navigation, search

Contents

Introduction

This article was written to help programmers design and implement Dynamic Link Libraries (or DLLs for short), a feature of modern programming that is often used but relatively poorly understood. It is aimed at programmers who already know what DLLs are, but their experience with implementing DLLs is limited.

It should be noted that DLLs in the Windows world and shared libraries in the UNIX world are conceptually very similar and often interchangeable. This article will focus on DLLs, but some differences between DLLs and shared libraries will be pointed out.

History

Dynamic linking is a relatively old technique. On the PC platform, OS/2 1.0 (released 1987) supported a DLL mechanism not substantially different from modern implementations. The Multics operating system introduced a shared library facility back in the 1960s. UNIX style shared libraries are a direct extension of static libraries, while DLLs are somewhat different and more specialized.

Why Write DLLs

This question will not be discussed. It is presumed that the reader already knows why he or she is writing DLLs.

Overview of DLLs

Perhaps the most crucial point to understand is that DLLs are not fundamentally different from static libraries. Code within a DLL will become part of the process image of a program that loaded it, just like the code that is part of the program's main executable.

This has some important implications. A DLL does not own any resources. Any resources that a DLL acquires are owned by the process in whose context the DLL executes. If a DLL opens a file, the file handle is owned by the process. If the DLL allocates a block of memory, the memory is owned by the process. If the DLL is unloaded but it does not close the file handle or free the memory, the file will be still open and memory will be still allocated. This is especially important to keep in mind when writing dynamically loaded DLLs in order to avoid resource leaks.

Runtime Environments

There is, however, a catch. Imagine a program that has a statically linked C runtime library. This program uses a DLL (either statically or dynamically loaded) which also has a statically linked C runtime library. Even though the main executable and the DLL are part of the same process, the two C runtime environments are separate and the resources managed by them may not be used by the "other" module, not even when both are built by the same version of the same compiler.

For example, if a DLL allocates memory on the heap (perhaps through malloc(), the pointer can not be freed (eg. passed to free() by the DLLs. In the better case, an attempt to do this will immediately cause a crash. In the worse case, the memory will be leaked or the heap will be quietly corrupted. There are similar issues with stream handles, signal handlers, exit lists, etc.

If an executable (or DLL) allocates a block of memory, it can of course pass the pointer to the DLL (or executable), but the other module cannot free the memory.

How to solve this problem? There are three fundamentally different approaches:

  • Use a single runtime environment. This is achieved by linking both the main executable and the DLL(s) against the DLL version of the C runtime. Since there is now only one runtime environment, the problem disappears. However, this is only a viable solution if both the main executable and the DLL(s) are under the developer's control.
  • Make sure resources are only managed by the runtime which allocated them. That is, if you open a stream in a DLL, never pass the FILE * pointer to the main executable or another DLL. Instead, provide an externally callable interface to read/write the file if that is necessary. Same goes for memory etc. This is the approach typically used by well designed static libraries as well.
  • Use the OS API directly. If you (on Win32) CreateFile in a DLL, you can CloseFile the same file in the main executable or in another DLL. You are, in essence, using the OS's runtime environment, which exists in a single instance for the entire process. This is the least useful solution as it requires non-portable platform specific code, but may be useful in situations where the application is already tied to a specific platform.

DLL and shared library differences

On UNIX platforms, shared libraries always use Position Independent Code (PIC), while executables often don't. PIC designates code that is independent of its load address. This enables shared libraries to be loaded at different addresses in different processes while still sharing the same code pages. The cost is slightly slower and more complex code. Note that PIC does not affect how source code is written, only how it is built.

On Windows and OS/2, DLL code is not position independent and is no different from code in an executable. Instead, the OS tries to map a DLL at the same address in each process. This leads to simpler and faster code at the cost of potentially needing multiple physical copies of DLL code if the address space is too tight and a DLL must be loaded at different addresses in two processes.

The other difference does affect DLL/shared library design. UNIX shared libraries typically use a global name space. If an executable is dynamically linked against symbol foo, the dynamic linker will search for a shared library which exports foo to satisfy the reference. The problem with this scheme is obvious: if two or more shared libraries, export symbol foo the programmer has limited control over which one will be used and results may be unpredictable.

In contrast, DLLs use two-level namespaces. That is, each module (main executable or DLL) has it own separate namespace. It is perfectly possible for the main executable and each DLL it loads to define a public symbol called foo and there will be no conflicts.

In addition, DLL exports may be imported by ordinal and not by name. Importing by ordinal is slightly more efficient but somewhat more difficult to manage. It is typically only used for DLLs with stable interfaces.

Calling DLLs

The above point has implications for using DLLs. When building an executable, the programmer cannot say 'symbol foo is going to dynamically linked' and be done with it. At link time, it must be known which module (DLL) each symbol will be coming from. This information is stored in the resultant module and will be used by the OS loader.

Linking with DLLs

There are two common ways to supply this information to a linker. One is linker definition files, the other is import libraries. Most compilers support both ways, although there are implementation differences, The following text gives examples for Open Watcom.

The IMPORT Directive

The Open Watcom linker (wlink) supports the import directive; the directive can be supplied directly on the command line, or in a separate command file (which will effectively become part of the command line). The syntax is as follows:

IMPORT imp{,imp}
imp ::= internal_name module_name[.entry_name | ordinal]

The internal_name is the name the application used to refer to the symbol; module_name, predictably, is the name of the module (DLL) which exports this symbol; entry_name is the name under which the module exports the symbol; ordinal offers alternative way to refer to the symbol by number instead of by name. It should be apparent that the name under which a symbol is exported need not be the same as the name under which the symbol is imported. However, except in special cases, it makes no sense to make the names different.

If a program calls function foo() which is implemented in DLL bar.dll, the linker directive will usually be:

import foo_ bar

Why the trailing underscore? Because the compiler usually mangles symbol names, and appended underscore is the default for Open Watcom.

Using Import Libraries

For this and other reasons, the import directive is somewhat difficult to use and manage. There is a another way which is much easier to use: import libraries.

An import library is a file with .lib extension which contains special records that store exactly the same information supplied to the import directive. Import libraries are used in the same way as traditional static libraries, but instead of the actual symbols they only contain "pointers" to their implementation in a separate DLL.

Import libraries are created by the Open Watcom library manager, wlib. There are several ways in which wlib can create import libraries; see the Open Watcom C/C++ Tools User's Guide for details. The simplest and most common way to create import libraries is the implib option of wlink (the linker will call wlib behind the scenes). The full syntax is:

OPTION IMPLIB[=imp_lib]

Simply using option implib when linking bar.dll will create bar.lib with import records for all symbols exported by the DLL. This import library is then used whenever a module wants to import a symbol from bar.dll. The user need not worry about name mangling or adding new and removing old exports, the entire process is automatic.

Writing DLLs

Code that ends up in a DLL is, for the most part, no different from code that is built into an executable. In fact in some cases the same source code can be built either as DLL or as EXE image. An important task when building a DLL is marking symbols that will be exported. These constitute the public interface of the DLL.

Again there are several ways to mark symbols for export (note that exported symbols are nearly always functions, but might also be data items). The most generic method is supplying the export information to the linker when the DLL is being created. For Open Watcom, this is the export directive. The basic syntax is as follows:

EXPORT export{,export}
export ::= entry_name[.ordinal][=internal_name]

This is analogous to the import directive. The entry_name is the name that will be exported from the DLL and will be visible to DLL callers. The ordinal is supplied when specific ordinal number is to be used for the export (otherwise, an ordinal will be assigned by the linker). The internal_name is used when the exported name is not the same as the symbol's actual name.

As with DLL imports, this method provides maximum control but also requires maintenance work. It is much more convenient (and usually sufficient) to mark the symbols for export in source code. The compiler will insert special records into the object file that the linker will subsequently use to select symbols for export.

Open Watcom C/C++ supports two ways to mark symbols for export. The two methods have different syntax but the same semantics. They are the __export keyword and the __declspec(dllexport) specifier. They are used as follows:

int __export foo( void );
__declspec(dllexport) int foo( void );

Once the code is written, the other important task is building it as a DLL. Besides using the appropriate linker options, the compiler usually needs to be informed that it is building a DLL as well. For Open Watcom, the compiler option is -bd. This option must be used with at least one module and is required for proper DLL startup code to be linked in.

Unlike executables, DLLs have no main() function. A DLL in general need not have any startup code at all. However, DLLs which use runtime library functions usually do require startup code. In some cases, a DLL may need to perform some tasks right after it is loaded and just before it is unloaded. There are several ways to do this; the easiest is writing a LibMain function if the target is Win32 or 32-bit OS/2.

16-bit Windows and OS/2 DLLs are slightly different and will not be discussed here. For Win32 and 32-bit OS/2, a LibMain function will be called during DLL loading, immediately after the C runtime has been initialized (so that library functions may be used). The function will be called again during DLL unloading just before the runtime is shut down. On Win32, LibMain will also be called when threads are starting and exiting. See the Open Watcom C/C++ Programmer's Guide for details. Note that if no user-written LibMain is provided, an empty implementation from the runtime library will be used.

When linking a DLL with Open Watcom, it is easiest to use the predefined nt_dll and os2v2_dll systems when building Win32 and 32-bit OS/2 DLLs, respectively.

To summarize, when creating a DLL it is necessary to:

  • Mark symbols for export
  • Compile with -bd switch
  • Link as DLL

Shared vs. Nonshared DLL Data

The code segments (or sections) of a DLL are always shareable, and in fact that's one of the major reasons for writing DLLs. But what about data segments? In almost all cases, data segments will be nonshared, ie. every process using a given DLL will have its own copy of the DLL's data segment. That way, the DLL instances are completely independent of each other -- which means no worries for you, the programmer.

In certain atypical situations, it may be necessary for all or portion of DLL's data to be global, ie. shared between all instances. This could be required for example if a DLL was managing some hardware resource and might have to arbitrate access from several processes.

If all of DLL's data is shared (global), the C statically linked runtime cannot be used in that DLL (not unless you really, really know what you're doing) -- because sharing a single runtime environment between several processes is the last thing you want. Sharing only specific data items in a separate data segment(s) is possible, however.

The MANYAUTODATA (for many instances, ie. nonshared data) and SINGLEAUTODATA (for single instance, ie. shared data) linker options can be used to control sharing of automatic data segments in Windows and OS/2 DLLs and executables. It is not recommended to change the defaults unless programmers are aware of all implications. Changing the defaults is very, very rarely needed.

The SEGMENT linker directive can be used to control sharing for individual segments/sections, and #pragma data_seg may be used to control placement of data items in C/C++ source code. Please see the relevant linker and compiler documentation for details -- but you will probably never need these features.

Using DLLs

For the most part, a function in a DLL is used exactly like any other function. On Win32, there is but one fly in the ointment: Functions need to be declared as coming from a DLL. Win32 optimizes DLL imports through the import address table (IAT). Normally this would be (and should be!) irrelevant to application programmers, except the compiler needs to know about this so that it could generate the proper (indirect) calls to functions imported from DLLs. To convey this information to the compiler, the __declspec(dllimport) specifier must be used used (a mirror image of __declspec(dllexport)).

A function imported from a DLL should be declared as follows:

__declspec(dllimport) int foo( void );

The __declspec(dllimport) specifier will cause the compiler to generate indirect calls to a function, with an __imp_ prefix added, where a direct call would normally be used. This is why it is not enough to specify imports to the linker -- the generated code needs to be different. Again, this is only needed on Win32, not on OS/2 (either 32-bit or 16-bit) or Win16. In fact attempts to use __declspec(dllimport) for targets other than Win32 will cause link failures.

Although conceptually __declspec(dllimport) and __declspec(dllexport) are two sides of the same coin, their usage is different. While __declspec(dllexport) is equivalent to the export linker directive and can be replaced by it, __declspec(dllimport) must always be used (on Win32) in addition to the import linker directive or import libraries.

Linking with DLLs has already been discussed in the Calling DLLs section above. To summarize, when writing programs that use DLLs, it is necessary to:

  • On Win32, mark functions as imported with __declspec(dllimport)
  • Link with DLL import library or use import directive to resolve imports

An Example

When writing both a DLL and its calling code (which could be an executable or another DLL), it is very advisable to share header files between the two. That way, the compiler will guarantee that the declarations the calling code uses match the actual definitions. Due to the vagaries of Win32 DLL programming noted above, this is slightly (but only very slightly) tricky.

One possible -- certainly not the only -- method is prefixing all exported function declarations with a macro, for example DLLENTRY, and defining it like this:

#ifdef __SW_BD
  #define DLLENTRY __declspec(dllexport)
#else
  #ifdef __NT__
    #define DLLENTRY __declspec(dllimport)
  #else
    #define DLLENTRY
  #endif
#endif

The __SW_BD macro is defined by Open Watcom C and C++ compilers when the -bd is switch is used, and __NT__ macro corresponds to Win32 target. The above construct will ensure that the right functions will be marked for export from DLLs and that on Win32, imported functions will be properly handled. For platforms that don't use DLLs, the DLLENTRY macro can readily be gotten out of the way by defining it to be empty.

A small example of DLL construction and usage is available for download. It demonstrates how to write a DLL and its caller, how to share a header file between the two, and as a bonus also shows LibMain usage. Trivial scripts are provided to build a Win32 and 32-bit OS/2 version of the example. The scripts deliberately use the simplest way to do this, using the wcl386 utility. IDE and makefile usage is beyond the scope of this article.

Note that the OS/2 version of the example can be built on Win32 and vice versa, as long as the appropriate runtime libraries are installed. This is achieved through the explicit -bt and -l options, as well as -I switch to point to the platform specific header files (which are only needed for LibMain).

Personal tools