SCons - the backbone of the cctbx build system

Conceptually it is a trivial task to compile and link portable source code. However, in real life this is one of the most time-consuming nuisances, in particular if multiple, diverse platforms have to be supported. In the version 1.0 release of the cctbx we made an attempt to address this with the home-made fast track build system. Of course home-made is often good enough, but a professional solution is almost always better, especially if it comes with no strings attached.

Fortunately such a system is now available: SCons, short for Software Construction tool. This is a perfect fit for the cctbx because the SCons developers have apparently adopted a similar “professional is better than home-made” philosophy: SCons is implemented in pure Python, and SCons configuration files (the equivalent of Makefiles) are pure Python scripts. SCons has many advantages compared to a traditional make-based build system. To quote some points from the SCons documentation:

  • Global view of all dependencies – no more multiple build passes or reordering targets to build everything.
  • Reliable detection of build changes using MD5 signatures – no more “clock skew detected, build may be incomplete”.
  • Built-in support for C, C++, Fortran, Yacc and Lex.
  • Improved support for parallel builds – like make -j but keeps N jobs running simultaneously regardless of directory hierarchy.
  • Building from central repositories of source code and/or pre-built targets.
  • Designed from the ground up for cross-platform builds, and known to work on Linux, POSIX, Windows NT, Mac OS X, Tru64 Unix, and OS/2.

When we moved from our home-grown build system to SCons we found all these points to be perfectly true. It only took very little effort to write a small configure script for setting up a master SConstruct file based on the user’s choice of which cctbx modules to use and which to ignore. New modules can easily be tied into this system simply by providing a SConstruct file in the module’s top-level directory. The author of the new module has complete control over the build process. The existing settings can simply be reused, customized, or totally replaced, all within one uniform and 100% platform-independent framework, the Python language.