CMake FetchContent

In my CMake Fundamentals series, I’ve been using the relatively recently added FetchContent module, without providing much explanation of how it works. I’ll attempt to change that with this post – I’ll demonstrate a few basic uses for the module and will try to convince you why it’s a worthwhile effort to use over non-cmake alternatives. Let’s get started with the why, and gradually move on to how.

Why FetchContent

Projects of reasonable complexity will need to manage dependencies – be it third-party, or developed internally. The broad options one has when choosing the method of dependency management is to consume dependencies in either a binary form – already compiled libraries, be it static or shared – or as source files by incorporating them and compiling as part of the project. The former is often the only option, especially if we don’t have access to the source code. It has the advantage of reducing the overall compile times and the disadvantage of introducing concerns such as ABI compatibility. The latter on the other hand introduces the obvious cost of compiling the code (which takes time) but has a number of advantages, some of them being potentially less link-time or run-time issues, no more ABI compatibility issues, and perhaps better optimization. Which one is right for your project is a different discussion.

The FetchContent module inherently isn’t limited to any of these use cases – it can support both, and more, to some extent. As its name might imply – at its core it just fetches (downloads) some arbitrary content – source code, scripts, binaries, or whatever else your project might need. And that’s all it does – i.e. it’s not a full dependency-management system. You probably could build one on top of it though. Even though the module isn’t completely limited, its main purpose is to incorporate other (sub)projects and scripts into a project. And these are the use cases I’ll be focusing on here.

Why would you choose FetchContent over alternatives though? First of all, there aren’t many. A popular choice is to use git submodules. This requires you to manually sync the project and its dependencies – always ensuring that the submodules are up-to-date. This may seem like a minor inconvenience, but it adds up over time. FetchContent on the other hand is part of the build system – simply configuring the project will ensure that the dependencies are up-to-date. If you’re managing an open-source project you’re not forcing contributors to follow a multi-step workspace setup. Everything is encapsulated. That’s a big plus to me.

Basic use

The most basic use of the module is extremely simple. An example that I like to use, and one that I think should be applicable to anyone, is including the googletest testing framework into your project. The googletest documentation recommends to compile it alongside your project, rather than using pre-built distributions. Older googletest documentation provided a read-to-use solution based on CMake ExternalProject module (still available here). This works, however, it’s quite a bit of CMake code, that requires some explanation. I’m glad to see that the updated googletest docs actually suggest the use of the FetchContent module instead. This is exactly the example I will reiterate here.

cmake_minimum_required(VERSION 3.14)
project(FetchExample)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_REQUIRED ON)

include(FetchContent)

FetchContent_Declare(googletest
    GIT_REPOSITORY https://github.com/google/googletest
    GIT_TAG release-1.10.0
)

FetchContent_MakeAvailable(googletest)

Using the FetchContent module is a two-step process – first the dependency is declared using the FetchContent_Declare command. This command takes the name of the dependency – here googletest – and some form of a reference where to download it from. In the example shown above the dependency will be cloned from a git repository using the specified version – GIT_TAG – this could also be a commit hash, branch name, etc. Another alternative would be to specify a URL pointing to an archive (tar.gz, zip) – this is the option shown in the official googletest docs.

The second step is to actually process the dependency – download it and add it to the project – FetchContent documentation calls this populating the content. In the example shown this is done using the FetchContent_MakeAvailable command – this is a shorthand for the most common use case.

And most of the time that’s all you need. This will make the targets declared by the dependency available to your project – here these will be gtest, gtest_main and gmock. Sometimes, however, you’d like more control over how the downloaded dependency is included into your project. To do so you will need to forego the FetchContent_MakeAvailable command and use the more verbose alternative.

Populating the Content

Like I already mentioned the FetchContent_MakeAvailable is a shorthand for the most common use case demonstrated above. It encapsulates the following pattern:

# FetchContent_Declare same as before

FetchContent_GetProperties(googletest)
if (NOT googletest_POPULATED)
    FetchContent_Populate(googletest)

    # Perform arbitrary actions on the googletest project

    add_subdirectory(${googletest_SOURCE_DIR} ${googletest_BINARY_DIR})
endif()

The FetchContent_GetProperties(<dependency>) command exposes information about the given <dependency> (the one specified in FetchContent_Declare) in form of global properties. These properties in case of the above example will be named

• googletest_SOURCE_DIR – root directory of the downloaded dependency
• googletest_BINARY_DIR – build directory of the dependency
• googletest_POPULATED – boolean informing if the is ready for use

The first two should be obvious. The last property is set to TRUE if the given dependency has already been populated (downloaded) and to FALSE otherwise. The SOURCE_DIR and BINARY_DIR properties are valid only if the POPULATED property is TRUE. It is then checked before calling the FetchContent_Populate command which does all of the work of actually downloading the content. It also defines the exact same set of properties described above.

The advantage this pattern has over using FetchContent_MakeAvailable is that once the content is populated – i.e. the dependency is downloaded – we can use arbitrary commands to consume the dependency. The most common is to simply call add_subdirectory, thus processing the CMakeLists.txt found in the SOURCE_DIR of the dependency. But what if the dependency doesn’t have a CMakeLists.txt? We could then instead use the include command or even define all the necessary targets manually, based on the defined SOURCE_DIR property.

A good example would be reuse of custom cmake scripts. Assume you have the following scripts you’d like to reuse in your projects:

doxygen.cmake

gcov.cmake

sanitizers.cmake

Providing support for the tools they’re named after. Instead of copying them into the project you could have a dedicated repository and use FetchContent to include them:

FetchContent_Declare(ProjectHelpers
    # ...
)

FetchContent_GetProperties(ProjectHelpers)
if (NOT projecthelpers_POPULATED)
    FetchContent_Populate(ProjectHelpers)

    include(${projecthelpers_SOURCE_DIR}/doxygen.cmake)
    include(${projecthelpers_SOURCE_DIR}/gcov.cmake)
    include(${projecthelpers_SOURCE_DIR}/sanitizers.cmake)
endif()

Note that the global properties defined by FetchContent_GetProperties and FetchContent_Populate use lowercase prefix of the dependency name.

This is just an example. You could do anything really – the FetchContent_Populate gives you full access to the downloaded dependency.

FetchContent configuration

If you start using the FetchContent module heavily you will undoubtedly notice that your CMake configure times get really long. This is because by default FetchContent always checks if the dependencies are up-to-date. This can take a noticeable amount of time, no matter how fast your connection is. To reduce this overhead you can add the following line somewhere in your CMakeLists.txt:

set(FETCHCONTENT_UPDATES_DISCONNECTED ON CACHE BOOL "Do not update everytime I run cmake, please")

This will cause FetchContent to download the dependency only if it hasn’t been populated yet, without making further attempts to update it every time, thus reducing the configuration overhead. Please also note that the comment at the end of the set command is mandatory, even if it’s just an empty string.

Summary

FetchContent is a versatile and convenient way of managing project dependencies. It wouldn’t be my first choice for managing binary dependencies – a job better suited for tools like Conan – but it’s been my go-to solution for all other types of dependencies lately – mostly the ones exemplified here.

Know that FetchContent has one more major use case – composition of arbitrary CMake projects into a single workspace, mono-repo, super-build or whatever you want to call it. This requires, however, some more explanation, so I’m saving it for a dedicated post.

 

References

• FetchContent module
• Old googletest readme
• New googletest docs
• CMake set cache entry docs