Reducing Compilation Time: Unity Builds

I recently stumbled upon a method to reduce compilation times in C/C++ projects by so called Unity Builds (UB). The technique behind UB  is quite trivial: compilation time is reduced by reducing the amount of disk access during compilation.

During compilation included files are walked multiple times. This is done for each translation unit the compiler encounters. Unless you own a solid state disk, your hard drive uses moving parts to read, write and seek positions that will slow down compilation.

In the demo project I’ve created, see links at the bottom of the post, there are actually two dummy translation units, t1.cpp and t2.cpp, that both include the same file ub/b.h which in turn includes ub/a.h. When compiled and instrumented, in MSVC you can use the switch /showIncludes to list the files parsed during compilation, you’ll see an output similar to

Compiling...
t2.cpp
Note: including file: b.h
Note: including file:  a.h
t1.cpp
Note: including file: b.h
Note: including file:  a.h

As you can see, for each translation unit the same include files are opened. The idea behind UB is to reduce the amount of redundant work (i.e. including files), by generating a unity translation unit that simply includes all of the other translation units.

So we add a file ub.cpp with the following content

#include <t1.cpp>
#include <t2.cpp>

remove the translation units t1.cpp and t2.cpp from the project (or we will hit multiple defined symbols) and compile. The compiler output this time is

Compiling...
ub.cpp
Note: including file: t1.cpp
Note: including file:  b.h
Note: including file:   a.h
Note: including file: t2.cpp

The careful reader will spot four includes now compared to four includes in the original example. So, did we win anything? Well yes, the first time b.h is encountered it is opened and parsed which leads to the inclusion of a.h. The next time we hit b.h it is not parsed anymore and so isn’t a.h.

For major software projects with deep nested include hierarchies and a plenty of translation units, we’d spare the compiler a lot of work by using unity builds.

Unfortunately, the manual procedure taken above is suitable only for the smallest software projects. So, we need a way of automating unit build generation. I’ll provide a solution for those who use CMake in the next section.

Automating Unity Builds

Unity Builds can easily be automated with CMake. Here’s a function that, when called, enables unity builds for a bunch of provided translation units.

function(enable_unity_build UB_SUFFIX SOURCE_VARIABLE_NAME)
  set(files ${${SOURCE_VARIABLE_NAME}})
  # Generate a unique filename for the unity build translation unit
  set(unit_build_file ${CMAKE_CURRENT_BINARY_DIR}/ub_${UB_SUFFIX}.cpp)
  # Exclude all translation units from compilation
  set_source_files_properties(${files} PROPERTIES HEADER_FILE_ONLY true)
  # Open the ub file
  FILE(WRITE ${unit_build_file} "// Unity Build generated by CMake\n")
  # Add include statement for each translation unit
  foreach(source_file ${files} )
    FILE( APPEND ${unit_build_file} "#include <${CMAKE_CURRENT_SOURCE_DIR}/${source_file}>\n")
  endforeach(source_file)
  # Complement list of translation units with the name of ub
  set(${SOURCE_VARIABLE_NAME} ${${SOURCE_VARIABLE_NAME}} ${unit_build_file} PARENT_SCOPE)  
endfunction(enable_unity_build)

To use it simply invoke enable_unity_build

set(INCLUDES inc/ub/a.h inc/ub/b.h)
set(SRCS src/t1.cpp src/t2.cpp src/main.cpp)

# Comment to disable unit build
enable_unity_build(UnityBuildDemo SRCS)

add_executable(
  UnityBuildDemo
  ${INCLUDES}
  ${SRCS}
)

Performance Improvements

Here is a diagram showing the performance improvement a set of random libraries developed at company-site. Each column corresponds to a library or unit test project for a library. The performance was measured on a Quad-Core PC having parallel project builds enabled.

Unity Builds

Best Practices

Continuous Integration

In contrast to the opinion of The Magic of Unity Builds, we do not use UB for our daily developer work, as minor changes quite frequently cause a rebuild of the entire unity. Instead, Unity Builds are used as the compilation method of choice for our continuous integration servers. Cut down compilation time helps in getting feedback more quickly from automated test suites.

Split Unities

Having to many compilation units collected in a single unity build file can lead to compilation problems such as out of memory issues. When such an issue occurs, you should split the monolithic unity into separate smaller unities. Here’s how this can be done using enable_unity_build

set(INCLUDES inc/ub/a.h inc/ub/b.h)

set(SRCS_A src/t1.cpp)
set(SRCS_B src/t2.cpp src/main.cpp)

# Comment to disable unit build
enable_unity_build(A SRCS_A)
enable_unity_build(B SRCS_B)

add_executable(
  UnityBuildDemo
  ${INCLUDES}
  ${SRCS_A} ${SRCS_B}
)

Most of the times, however, a single Unity Build file per project works without any problems.

Code Quality

Introducing unity builds can reveal code quality issues such as duplicating code in translation units. Since ‘all’ translation units are collected in a single unity, multiple, previously equally named but separate type declarations/definitions will become a single type and thus lead to ‘multiple defined symbol’ errors.

Links

About these ads

5 thoughts on “Reducing Compilation Time: Unity Builds

  1. Pingback: Reducing Compilation Time: Precompiled Header « Christoph Heindl

  2. Pingback: Unity Build « Nberserk's Blog

  3. It has nothing to do with disk I/O but what matters is avoiding full parsing of header files and compiling of all the code there. Another possible benefit is faster program execution because the optimizer can see all the code and optimize past function boundaries. However, it isn’t a magic bullet as there are also many disadvantages.

    • I know your reply is a bit old. But I’m wondering about what’re the ‘many’ disadvantages with Unity Builds? I’ve been searching this for a while. Would be good to know in detail.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s