Reducing Compilation Time: Precompiled Header

Update 2010/09/25 The source code listing has been updated to incorparate a suggestion by DO to ensure that the header unity is the first unity in the list of source files.

This post is the second installment in a series of articles on reducing compilation times for larger C++ software projects.

In the first installment, ‘Unity Builds’ where introduced as a technique to lower compilation times. ‘Unity Builds’, by definition, collect multiple translation units into a single larger translation unit. For automated build systems that need to rebuild the entire code basis frequently, the overall compilation time is reduced. But, IMHO, for developers who change code rather locally, ‘Unity Builds’ result in larger compilation times.

With precompiled headers we seek for the same goals as with ‘Unity Builds’ – reducing compilation times by reducing the amount of preprocessor work. Precompiled headers instruct the compiler to preprocess a set of header files (and their recursive inclusions) into a form that is faster to process later on. This technique is available at least for MSVC and GCC toolsets.

We will reuse the demo-project created in the first installment. This time, however, we will add a few Boost include statements to a.h. You can download the demo project at the bottom of this post. Instrumented with /showIncludes a compilation pass (without precompiled headers enabled) results in something along these lines

Compiling...
t2.cpp
Note: including file: b.h
Note: including file:  a.h
<- hundreds of boost includes here ->
t1.cpp
Note: including file: b.h
Note: including file:  a.h
<- hundreds of boost includes here ->

The project compiled in about 8 seconds. Next, we’ll add the precompiled header technique to instruct the compiler to preprocess Boost includes and re-use them at later compilation passes.

Note The remaining article focuses on MSVC toolset specifics and provides an automated solution using CMake. I’d greatly appreciate any solutions for other toolsets in the same manner.

Setting Up Precompiled Headers

For the precompiled header technique to work, we need at least one designated header file that contains the required include directives to preprocess. Additionally, a designated translation unit is required that will function as a container for the preprocessed headers. Next, we need to enable precompiled headers for translation units and assign them a precompiled header file. Finally, we must adopt the translation units to include the precompiled header before any other include directive.

This sounds like a lot of work, doesn’t it? As in the first installment, I will provide a convenient CMake solution that will handle most of the steps transparently. What’s left for the user is to provide the name of an ordinary header file that is to be turned into a precompiled header, plus a set of translation units that should benefit from these precompiled headers.

Automating Precompiled Headers

Without further discussions, here is a CMake function that enables precompiled headers for a set of translation units.

# Instructs the MSVC toolset to use the precompiled header PRECOMPILED_HEADER
# for each source file given in the collection named by SOURCE_VARIABLE_NAME.
function(enable_precompiled_headers PRECOMPILED_HEADER SOURCE_VARIABLE_NAME)
  if(MSVC)
    set(files ${${SOURCE_VARIABLE_NAME}})
    
    # Generate precompiled header translation unit
    get_filename_component(pch_basename ${PRECOMPILED_HEADER} NAME_WE)
    set(pch_abs ${CMAKE_CURRENT_SOURCE_DIR}/${PRECOMPILED_HEADER})
    set(pch_unity ${CMAKE_CURRENT_BINARY_DIR}/${pch_basename}.cpp)
    FILE(WRITE ${pch_unity} "// Precompiled header unity generated by CMake\n")
    FILE(APPEND ${pch_unity} "#include <${pch_abs}>\n")
    set_source_files_properties(${pch_unity}  PROPERTIES COMPILE_FLAGS "/Yc\"${pch_abs}\"")
    
    # Update properties of source files to use the precompiled header.
    # Additionally, force the inclusion of the precompiled header at beginning of each source file.
    foreach(source_file ${files} )
      set_source_files_properties(
        ${source_file} 
        PROPERTIES COMPILE_FLAGS
        "/Yu\"${pch_abs}\" /FI\"${pch_abs}\""
      )
    endforeach(source_file)
    
    # Finally, update the source file collection to contain the precompiled header translation unit
    set(${SOURCE_VARIABLE_NAME} ${pch_unity} ${${SOURCE_VARIABLE_NAME}} PARENT_SCOPE)
  endif(MSVC)
endfunction(enable_precompiled_headers)

The function can be decomposed into these steps:

  1. Mark the ordinary header file as a precompiled header file
  2. Generate a translation unit that will host the precompiled header results
  3. Assign the precompiled header to all translation units given
  4. Force the prefix inclusion of the precompiled header at all translation units

To apply it, simply add something like along these lines

set(INCLUDES inc/pch/a.h inc/pch/b.h)
set(SRCS src/t1.cpp src/t2.cpp src/main.cpp)

# Turn on precompiled headers
enable_precompiled_headers(inc/my_pch.h SRCS)

add_executable(
  PrecompiledHeadersDemo
  ${INCLUDES}
  ${SRCS}
)

Turning on precompiled headers lowered the compilation time to 6 seconds on the first build (where the pre-compiled header is generated). Subsequent modifications to any translation unit or headers (except modifcation to the precompiled header) reduced the compilation times to zero seconds.

Best Practices

Header Selection
It should be obvious by now that headers which rarely change should be placed in the precompiled header file. Including headers that change frequently will defeat the purpose of precompilation. Some headers that perform tricky initialization upon inclusion cannot be added to the set of precompiled headers (check boost/serialization/class_export.hpp for a notable example).

Header Placement
I would not recomment putting your precompiled headers inside your project’s include directory for the following two reasons:

  • Lazy developers will start to directly include your precompiled header, using it as a facility to group include statement. This can easily result in a single header that includes the world.
  • Shipping your product with precompiled header files is probably not the best idea.

No Precompiled Headers
Make sure your software compiles without precompiled headers enabled. Create, for example, an automated build without precompiled headers to ensure compatibility.

Split Precompiled Headers
Create multiple precompiled headers suited for groups of translation units.

set(INCLUDES inc/pch/a.h inc/pch/b.h)

set(SRCS_A src/t1.cpp)
set(SRCS_B src/t2.cpp src/main.cpp)

enable_precompiled_headers(inc/my_pch_a.h SRCS_A)
enable_precompiled_headers(inc/my_pch_b.h SRCS_B)

add_executable(
  PrecompiledHeadersDemo
  ${INCLUDES}
  ${SRCS_A} ${SRCS_B}
)

Rebuild Precompiled Headers
Note the risk that your toolset may not recognize that the selected headers to be precompiled have changed (for example when you upgrade to a new library version). Make sure to have your precompiled headers recreated manually.

Links

About these ads

13 thoughts on “Reducing Compilation Time: Precompiled Header

  1. We go with unity builds of 5..30 files. It is a fair balance between whole builds and building after small changes. This possibility was not mentioned.

    And also we use precompiled headers too. :-)

    • You are absolutely right. What’s the size of the project you are dealing with? What compile-time reduction did you achieve using the proposed combination of unity builds and precompiled headers?

      — Christoph

  2. 10k files, 5M lines, 80M bytes. We had PCH from the beginning of time so I cannot tell the difference. When we introduced the partial unity builds the time went down from 30mins to 10mins. Building after small changes did not increase significantly.

  3. I’m trying to use your macro, but it seems to be trying to compile another source file first, and panics when it can’t find the .pch file, which of course hasn’t been generated yet.

    • really? do you have an simple example you want to share? You can drop me a mail so I can have a look at it. What version of CMake are you using? What toolchain are you generating for (Visual Studio)?

      — Christoph

      • When looking at your implementation I would guess that the problem Michael Daum is describing might be related to the the following line:
        set(${SOURCE_VARIABLE_NAME} ${${SOURCE_VARIABLE_NAME}} ${pch_unity} PARENT_SCOPE)

        I might rather have used:
        set(${SOURCE_VARIABLE_NAME} ${pch_unity} ${${SOURCE_VARIABLE_NAME}} PARENT_SCOPE)
        to make sure that the newly created program unit will be compiled before any other file and therefore guarantee that the pre-compiled header will always be available.

        • DO, thanks for pointing this out. I have updated the post to incorporate your suggestion.

          Frankly, I haven’t received any feedback from Micheal Daum yet, so it is hard to tell what the problem is/was.

          Anyways, thanks for reading my article so carefully and pointing this out.

          Best regards,
          Christoph

  4. Thanks for an enlightening post. How much of this would apply to GCC? Would it just be a simple matter of changing the flags in the calls to SET_SOURCE_FILE_PROPERTIES?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s