Mac OS X 10.1 Prebinding Notes

This document describes an optimization called prebinding which enables fast launching of applications on Mac OS X.

Normally, when you build an application or dynamic library, the static linker (ld) records the names of the symbols the executable links against, and marks references to these symbols as "undefined."

When an application is launched, the dynamic linker (dyld) must bind the needed undefined references from the executable and dynamic libraries to their respective definitions. The binding process can take time, since the linker must map the library to an unoccupied address range and calculate the address of each referenced symbol in the library.

Building a dynamic library with prebinding enabled eliminates the normal binding overhead by predefining the library at a specified address range. When an executable or other dynamic library is built against a prebound library, the linker can directly reference symbols in the prebound library by address, instead of leaving the addresses undefined.

The static linker also records the time stamp of libraries dependent libraries. When the program is executed, the dynamic linker checks to see that all the build time stamps match and that the prebound address ranges of all code does not overlap. If both of these conditions are met, the binding of undefined references is already done, which saves a considerable amount of time. If the time stamps don't match, or prebound executable addresses overlap, the prebinding is undone and the program is bound normally.

Note that prebinding is only applicable to Mach-O executables. CFM PEF binaries do not support prebinding.

Requirements for Prebinding

To work properly, libraries and executables must meet several important requirements:

Libraries must not have overlapping preferred addresses. For each release of Mac OS X the Apple-supplied libraries are all prebound and do not have overlapping addresses. Your libraries must not overlap with any of your own libraries, and also must not overlap with any of the Apple-supplied libraries that are installed with Mac OS X. There is currently no way to automatically select an address for a library to ensure that it does not overlap the addresses of other libraries. Non-Apple libraries can use any address in the range 0x00000000 to 0x3FFFFFFF. For the Mac OS X 10.0 release, all addresses above 0x90000000 are also available. To set the address of a library, pass either the -seg1addr flag or the -seg_addr_table flag to ld(1). All executables should be at the default address, 0x00000000. The default address is 0x00000000. When selecting addresses for libraries, the address range of the largest executable using the libraries should be avoided.
There can be no undefined symbols. In most cases, this means that you must link against your dependent libraries. It also means that there can be no circular dependencies (cases where library A calls a function in library B, and a function in library B also calls a function in library A). Circular dependencies can be removed by changing the code to dynamically look up the function instead of directly referencing it.
You cannot override symbols that are referenced in flat namespace images used by the dependent libraries. For example, you can't define your own malloc and then prebind using flat namespace libraries.

Producing a Prebound Build

All libraries must be built in dependent order and built prebound. That is, libraries must be built before the libraries (and executables) that link against them are built. To build prebound, either pass the -prebind flag to ld(1), or define the LD_PREBIND environment variable.

You do not need to relink every time the prebinding of a library might change. You can use the redo_prebinding(1) tool and the update_prebinding(1) tool to update prebindings. However, in order to use these tools, you need to link against libraries that were initially prebound. If a library was not prebound when it was originally linked against, redo_prebinding(1) cannot prebind it. It must be initially built as a prebound library, as specified above.

Building Your Project Prebound

The easy way to do this is to set the environment variable LD_PREBIND before building your projects. For example, if you are using tcsh(1):

% setenv LD_PREBIND

This has the same effect as passing -prebind to the ld(1). For Project Builder framework and library projects you can add -prebind to OTHER_LDFLAGS. For application projects, Project Builder adds -prebind to the linker flags by default.

If your project's prebinding is disabled when built, here's how to fix it:

If prebinding fails you will see a warning message in the build log. Prebinding has a number of requirements to allow it to work (see above). If any of these requirements aren't met prebinding will disabled and a warning message will be printed by the static linker when building.

The most common message you might see is:

/usr/bin/ld: warning prebinding disabled because of symbols overridden in dependent dynamic shared libraries: /BinCache1/objc4/Objects/objc4-133.obj~2/objects-optimized/objcopt.tproj/objcopt.o definition of _swap_mach_header in section (__TEXT,__text) /System/Library/Frameworks/System.framework/System(swap.o) definition of _swap_mach_header

In this case, this happened for the objc4 project because it had a copy of System framework's swap.c in the project. It did this long ago as a workaround when the System framework's swap.c was out of date. The fix for this was to remove the objc4 project's copy of swap.c, thus eliminating the symbol override.

If you need to track down what object is referencing a specified symbol then ld(1)'s -ysymbol flag can be used. If you need to determine why a specific module is being linked in from a library ld(1)'s -whyload flag can be used.

Another example:

ld: warning prebinding disabled because of symbols overridden in dependent dynamic shared libraries: /BinCache1/PBDevKit/Objects/PBDevKit-378.1.obj~2/objects-optimized/DevKit/Lowlevel.subproj_subproj.o definition of _regcomp in section (__TEXT,__text) /System/Library/Frameworks/System.framework/System(regcomp.o) definition of _regcomp

In this case PBDevKit was using a different version of regcomp, regexec and regfree than the System framework's version. In this case, PBDevKit needs this different version but does not need the code in System to use it's version. The best fix for this problem is to hide the names of these three functions in PBDevKit and their uses. This can be done in a number of ways:

PBDevKit could declare these symbols as __private_extern__
PBDevKit could remove these symbols from the list of exported symbols using the nmedit(1) tool
If the System framework was built as a two-level namespace library and PBDevKit was built as a two-level namespace library, this would not be a problem, because these symbols would be declared in different namespaces and therefore not overlap.

Another message you might see is:

/usr/bin/ld: warning prebinding disabled because dependent library: /System/Library/Frameworks/System.framework/Versions/B/System is not prebound

This happened because the project was built against a version of System.framework that was not prebound. This might be a build order problem. If you have programs or frameworks that depend on other frameworks you must build them in the right order, and you must ensure that they are prebound.

Another example is:

/usr/bin/ld: warning prebinding disabled because dependent library: /System/Library/PrivateFrameworks/PBDevKit.framework/Versions/C/PBDevKit is not prebound

In this case the prebinding of PBDevKit was disabled so that caused the prebinding of the program linking against it to be disabled (see the above example).

Checking If A Binary Is Prebound

To check if a binary was built prebound you can use otool(1) to look for the PREBOUND flag in the mach header. For example:

% otool -hv /bin/cat /bin/cat: Mach header magic cputype cpusubtype filetype ncmds sizeofcmds flags MH_MAGIC PPC ALL EXECUTE 10 1456 NOUNDEFS DYLDLINK PREBOUND

A prebound binary has the build time stamps of its dependent libraries recorded in them. To see the build time stamps again:

% otool -Lv /bin/cat /bin/cat: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 50.0.0) time stamp 982996740 Fri Feb 23 22:39:00 2001

And viewing the dependent libraries build time stamp can also be done with otool(1):

% otool -Lv /usr/lib/libSystem.B.dylib /usr/lib/libSystem.B.dylib: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 50.0.0) time stamp 982996740 Fri Feb 23 22:39:00 2001

Checking If Prebinding Is Successful

Set the environment variable DYLD_PREBIND_DEBUG and run the program. For example using tcsh(1) and checking the program sync(1):

% % setenv DYLD_PREBIND_DEBUG % sync dyld: sync: prebinding enabled

If a program is not built prebound but all the libraries are built prebound then dyld may try to use the prebound libraries. For example:

% setenv DYLD_PREBIND_DEBUG % cc -v dyld: cc: prebinding enabled using only prebound libraries

This happened because when cc(1) was built the System framework it was built against was not prebound. dyld(1) can fail trying to use prebound libraries if the program overrides symbols defined in it's dependent libraries that are used by a dependent library. For example:

% /usr/bin/objcunique dyld: /usr/bin/objcunique: trying to use prebound libraries failed due to overridden symbols

Currently dyld(1) only tries to use the prebound libraries if the program uses only one prebound library. This is because with flat namespace libraries the checking needed to make sure all the libraries don't override any of each other symbols is more time consuming than the optimization would generally save.

Updating the Prebinding

Another way prebinding may fail is that the build time stamps of the libraries a program was built against with do not match the libraries the program is run against. In the following example, sync1 was built using using a different version of the System.framework:

% setenv DYLD_PREBIND_DEBUG % /tmp/sync1 % dyld: /tmp/sync1: prebinding disabled because time stamp of library: /System/Library/Frameworks/System.framework/Versions/B/System did not match %

The program will run correctly but the prebinding optimization will be undone and the program dynamically bound as usual. Releases from Apple should never be in this state. However, if you change a framework on your system you can update the dependent programs (or frameworks) without rebuilding them by running redo_prebinding(1) on them. For example, using the case above:

% redo_prebinding /tmp/sync1 % setenv DYLD_PREBIND_DEBUG % /tmp/sync1 dyld: /tmp/sync1: prebinding enabled %

The program update_prebinding(1) can also be used to update the prebinding and is used as part of the Installer's processing during installation.

Invalidating the Prebinding

To force the prebinding to be out of date one can run strip(1) or nmedit(1) on one of the dependent libraries to cause the built time stamp to be changed. A common way to do this and not effect the symbols contained in the library is to run strip(1) with the -X option as all the local symbols starting with 'L' are already stripped by the assembler by default.

Prebinding and Library Initialization

Since prebinding requires that the entire set of libraries be initialized when loaded, libraries may be initialized in a different order than they would without prebinding. This can uncover latent problems in the order of library initializations that went undetected without prebinding, particularly if there were library initialization routines which did not explicitly call the initialization routines for libraries on which they depend. As a result, unexpected problems can occur when prebinding is enabled.