CMake Fundamentals Part 7

In the previous part of the series, we looked at how to add tests to a project. The next step is providing support for deploying, or installing, the built binaries to our users. No matter if you’re working on a library or an executable application, users need to be able to deploy the binaries to a location of their choosing in a reliable way. In this part of the series, we take a closer look at installing libraries.

Defining installation requirements

What does it mean to “install” a library? Let’s first consider how a library is used. A library will generally be a dependency of another project – either a larger library or an application. It could be a development (build) dependency, or it could be runtime dependency. In the former case, the library could be either static or shared. In the latter case, it will always be a shared library.

A practical definition for “installing a library” could then be something along the lines of

“Deployment of the binaries, be it static or shared, and possibly header files – for development purposes, to a directory of user’s choosing, in a way that exposes the binaries, headers and potential information about transitive dependencies, for convenient consumption by depending projects.”

This may be somewhat vague, but the key point, I believe, is the convenience. In the context of CMake this would mean supporting the standard ways of incorporating dependencies into a project. In part 6 I introduced the FetchContent module. It’s a great solution for including dependencies that need to be built alongside the project. However, it’s not quite sufficient for binaries – it could be used to download the package, but what then? We still need the information about include and link paths, possibly defines, flags and transitive dependencies. In CMake all this is handled by Find modules, or the more recent Config files. Both of these are a standard way of exposing targets that encapsulate all the necessary information to the user. Find modules and Config files can be consumed with the help of the find_package command – it will find the Find module or Config file associated with the given package and process them, thus making targets they define available. But first things first. Let’s forget about Config files and find_package for a moment and see how to add the most basic installation support.

One more thing before I get into it. I first need to mention that deploying libraries, especially if multiplatform support is it be considered, is a huge topic that could be approached in many different ways. The number of variables to consider is simply too large, especially for an introductory post. Instead of discussing all the possibilities I will present and describe an approach that supports a decent number of use cases on Linux as well as on Windows. I have no idea about Apple platforms, sorry not sorry. Unfortunately for the same reasons I also do not consider how to handle shared libraries. I wouldn’t be able to cover this topic well here, so I’d rather postpone it to a future, dedicated post.

The approach I’m about to describe provides support for relocateable installs, that are find_package‘able via generated Config files. It’s a basic, but more or less standard setup that could be extended to accomodate specific project’s needs.

Install directory layout

One of the first things to consider when preparing for project installation is the install directory layout. Where the static and shared libraries, executables, and headers go. This is the bare minimum, one might also want to consider any possibly generated documentation, scripts, etc. Unless a project has some specific requirements regarding this layout, there is a standard approach that can be used. Anyone vaguely familiar with Linux would expect the following:

.
├── bin/        # executable binaries
├── include/    # header files
└── lib/        # static or shared libraries

And this is exactly the layout I’m going for. The specified paths are relative to a root install directory, which makes the installation relocateable – the user isn’t forced to install the package to /usr/, /usr/local, /opt/, or any other path one might think to hardcode. Instead only the relative layout below the root install directory is preserved – this is sufficient to relyably use the package.

This standard layout is provided by the GNUInstallDirs module. Despite the name, the module handles the layout in a cross-platform way, e.g. on Windows the DLLs will go into bin/ rather than lib/, as expected. Once included this module exposes the following variables (plus a few more):

CMAKE_INSTALL_BINDIR        # bin/
CMAKE_INSTALL_LIBDIR        # lib/
CMAKE_INSTALL_INCLUDEDIR    # include/

These variables can be used when specifying install paths, instead of hardcoding them. This handles the considerations of the target platform for us.

Installing targets

Now that we know what we’d like the install directory layout to look like we can proceed to actually installing the targets defined in our project and the resulting binaries. This is actually relatively simple and is handled by the aptly named install command. All that’s required is addition of the following lines to the top-level CMakeLists.txt:

include(GNUInstallDirs)

# (...)

install(TARGETS Add Subtract Maths MathsDemo
    RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
    LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
    ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
)

Seems too easy, right? Because that’s not the whole story of course. First, since in part 4 of the series I introduced the concept of modularization and insisted on applying it across the project it would now also make sense to handle installation on a per-module basis – each module should be responsible for correctly installing itself. So instead of calling install once on all targets it might be better to call it separately for each module. It’s a little more work, but it leads to more maintainable project setup. Specifying the DESTINATION for each artifact type seems like some boilerplate that could be done away with by wrapping the call in a function. That might be a reasonable thing to do, but since CMake 3.14 it’s unnecessary – the values given above are the default, if the GNUInstallDirs module is included. Let’s rely on that, and install each module separately:

CMakeLists.txt:

include(GNUInstallDirs)

# (...)

install(TARGETS MathsDemo)

add/CMakeLists.txt:

install(TARGETS Add)

subtract/CMakeLists.txt:

install(TARGETS Subtract)

Maths/CMakeLists.txt:

install(TARGETS Maths)

As a whole this is exquivalent to the previous single install command call. This approach however, makes each module self-contained. Anyway, let’s configure, build and install the project, to see what happens. The first thing to do when testing project installation is to set the CMAKE_INSTALL_PREFIX variable to a local install directory. Either specify it on the command line or hardcode it temporarily in the CMakeLists.txt. I’ve chosen the first approach:

$ cmake -DCMAKE_INSTALL_PREFIX:PATH=./install -S . -B build
$ cmake --build build -j7
$ cmake --build build --target install

-- Install configuration: "Debug"
-- Installing: /home/user/cmake_fundamentals/install/include
-- Installing: /home/user/cmake_fundamentals/install/include/gmock
-- Installing: /home/user/cmake_fundamentals/install/include/gmock/gmock-cardinalities.h
-- Installing: /home/user/cmake_fundamentals/install/include/gmock/gmock-generated-function-mockers.h.pump
(...)
-- Installing: /home/user/cmake_fundamentals/install/lib/libAdd.a
-- Installing: /home/user/cmake_fundamentals/install/lib/libSubtract.a
-- Installing: /home/user/cmake_fundamentals/install/lib/libMaths.a
-- Installing: /home/user/cmake_fundamentals/install/bin/MathsDemo

Wait, what? Why are gmock and gtest being installed? Since we’ve chosen to use FetchContent to embed googletest within our project, and gtest calls install for its own targets, it is being installed alongside our project. This can be easily dealt with by setting INSTALL_GTEST to OFF just before FetchContent_MakeAvailable(googletest). The relevant lines in tests/CMakeLists.txt now look as follows:

set(INSTALL_GTEST OFF)
FetchContent_MakeAvailable(googletest)

That was easy only because the googletest guys were nice enough to think about supporting this use case and made the installation conditional. The same should be done for our project if we wanted to support the use case of embeding our project via FetchContent as well. Let’s not get too ahead of ourselves though. Back to basic target installation.

If the project is intalled now, we get the following output:

$ cmake --build build --target install
Install the project...
-- Install configuration: "Debug"
-- Installing: /home/user/cmake_fundamentals/install/lib/libAdd.a
-- Installing: /home/user/cmake_fundamentals/install/lib/libSubtract.a
-- Installing: /home/user/cmake_fundamentals/install/lib/libMaths.a
-- Installing: /home/user/cmake_fundamentals/install/bin/MathsDemo

As you can see the libraries are now installed into the expected directories. You may have noticed that the header files are not installed though. This is because headers need to be handled explictly, to accomodate the potential layout differences – it’s unlikely that one would want all headers of a library to be dumped directly into the include directory. It’s much more likely to have some layout that logically groups the header files. We’ll deal with that next. Oh, by the way. I have chosen to rename the Main target to an equaly unimaginative MathsDemo.

Since the project is already modularized this will be straightfoward. Each module will of course be responsible for installing its headers. The layout should correspond to the module layout in the source tree. To achieve this we’ll need to use a different form of the install command – install(DIRECTORY …) as one might expect it install an entire directory. We might also choose to install on a per-file basis, but let’s go with the DIRECTORY approach, since it’s more succinct.

add/CMakeLists.txt:

install(DIRECTORY add
    DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}
)

subtract/CMakeLists.txt:

install(DIRECTORY subtract
    DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}
)

maths/CMakeLists.txt:

install(DIRECTORY maths
    DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}
)

Again, nothing too difficult to understand. This could be wrapped along with the previous install call into a single function or macro to make the whole process even more painless. Re-installing the project now results in the following output:

-- Installing: /home/user/cmake_fundamentals/install/lib/libAdd.a
-- Installing: /home/user/cmake_fundamentals/install/include/add
-- Installing: /home/user/cmake_fundamentals/install/include/add/calc_status.h
-- Installing: /home/user/cmake_fundamentals/install/include/add/add.h
-- Installing: /home/user/cmake_fundamentals/install/lib/libSubtract.a
-- Installing: /home/user/cmake_fundamentals/install/include/subtract
-- Installing: /home/user/cmake_fundamentals/install/include/subtract/subtract.h
-- Installing: /home/user/cmake_fundamentals/install/lib/libMaths.a
-- Installing: /home/user/cmake_fundamentals/install/include/maths
-- Installing: /home/user/cmake_fundamentals/install/include/maths/maths.h
-- Installing: /home/user/cmake_fundamentals/install/bin/MathsDemo

Right, we now have the install layout we have defined in the first paragraph. So far so good. The next step is to make this installation consumable by other projects, that is, exposing all the necessary information – include directories, paths to the actual libraries, etc. in the form of targets. This is done using export targets.

Export target installation

Every install(TARGETS …) command may optionally specify an EXPORT set to which the target belongs to, thus adding the target to that EXPORT set. An EXPORT set (target) carries full information about properties of each target included into the EXPORT, and all the inter-target dependencies. We will have a single EXPORT set for the entire project (library), let’s name it Maths, since that’s the library we’d like to expose to the end user. Each install(TARGETS …) call needs to be extended as follows:

CMakeLists.txt:

include(GNUInstallDirs)

# (...)

install(TARGETS MathsDemo
    EXPORT Maths
)

add/CMakeLists.txt:

install(TARGETS Add
    EXPORT Maths
)

subtract/CMakeLists.txt:

install(TARGETS Subtract
    EXPORT Maths
)

Maths/CMakeLists.txt:

install(TARGETS Maths
    EXPORT Maths
)

Doing so loads the Maths export target with all the necessary information, it doesn’t expose it to the user however. That’s done by installing the export target itself, like so:

install(EXPORT Maths
    DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/Maths
    NAMESPACE Maths::
)

Let’s unpack what’s going on here. We’re saying that the EXPORT target Maths should be installed to a directory lib/cmake/Maths – this is a CMake convention – I explain the why later. The NAMESPACE argument means that all targets exposed by the installed EXPORT target will have the given prefix – here Maths::. This means that the end user of our library will need to include the Add library by linking Maths::Add, rather than just Add. The namespace is introduced in order to avoid name clashes with other packages. Things are starting to get a little complicated. Fret not, we’re nearing the end. Though trying to install the project now would result in a number of error messages:

CMake Error in add/CMakeLists.txt:
  Target "Add" INTERFACE_INCLUDE_DIRECTORIES property contains path:

    "/home/user/cmake_fundamentals/add"

  which is prefixed in the source directory.


CMake Error: install(EXPORT "Maths" ...) includes target "Add" which requires target "ProjectConfiguration" that is not in any export set.

(...)

It all boils down to two issues:

1. The targets in the export set contain include directories pointing to an absolute path in the source tree – there is zero chance that these paths will be correct on the end-users machine. We need a way to swap these paths out for correct paths, relative to the install directory.
2. The EXPORT target needs to be able to provide full inter-target dependency context, but the ProjectConfiguration target is not installed. We need to install it, even though it’s only an interface target, and doing so will have no effect other than adding information about its dependencies to other targets.

The second issue is simpler, so let’s fix it first. It’s enough to add ProjectConfiguration to the install(TARGETS …) call in the top-level CMakeLists.txt:

CMakeLists.txt:

include(GNUInstallDirs)

# (...)

install(TARGETS MathsDemo ProjectConfiguration
    EXPORT Maths
)

The first issue requires more explenation. Each target defined in the project also adds include directories to its INCLUDE_DIRECTORIES and INTERFACE_INCLUDE_DIRECTORIES properties. For example, the Add target:

target_include_directories(Add
    PUBLIC
        ${CMAKE_CURRENT_SOURCE_DIR}
)

The above call appends an absolute path expansion of CMAKE_CURRENT_SOURCE_DIR to both properties. This path is absolute and valid only when building the project. Since we want installed packages to be relocateable this path needs to be replaced with a path relative to the install root directory upon installation. This is done using a set of boilerplate generator expressions, the above target_include_directories command needs to be replaced with the following:

target_include_directories(Add
    PUBLIC
        $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
        $<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
)

Without going into too much detail about how generator expressions work (see my post about generator expressions if you’d like to know more) this call will do exactly what we want – it will expand to the path in the source tree when building the project and to a path relative to CMAKE_INSTALL_INCLUDEDIR when the target is installed. After applying the same modification to all targets, the project can finally be installed.

$ cmake --build build -t install
(...)
-- Installing: /home/user/cmake_fundamentals/install/lib/cmake/Maths/Maths.cmake
-- Installing: /home/user/cmake_fundamentals/install/lib/cmake/Maths/Maths-debug.cmake

Two new files are installed with this call – Maths.cmake and Maths-debug.cmake. These are generated by installing the EXPORT target and contain all the information necessary to consume our library. Inspecting them is an excercise for the interested reader ( 😉 ). These files are the basis for adding the find_package support. The rest will be handled by yet another CMake module – CMakePackageConfigHelpers.

Package Config

The CMakePackageConfigHelpers module exposes a set of functions for generating package Config file – one of the ways to support the find_package command. The other is writing a Find module – this is the old way of doing things and should generally be avoided for new projects. Generating the Config file is relatively straightforward. It involves three steps:

1. Write a <project-name>Config.cmake.in template file, which will be used to generate the Config.
2. Call the configure_package_config_file function to actually generate the Config file.
3. Install the generated Config file, the same way you would any other file.

The template file needs to be provided, because the decision of what it needs to contain is up to the us. In the most basic cases however, the same, very simple configuration file can be used:

cmake/MathsConfig.cmake.in:

@PACKAGE_INIT@

include(${CMAKE_CURRENT_LIST_DIR}/Maths.cmake)
set_and_check(Maths_INCLUDE_DIR "@PACKAGE_CMAKE_INSTALL_INCLUDEDIR@")
set(Maths_LIBRARIES Maths::Maths)
check_required_components(Maths)

This correctly supports relocateable packages on all platforms. Notice that the Maths.cmake generated by installing the EXPORT target is being included here – that’s where all of the real work is done. Next, variables exposing the include path and the libraries are set. This is not strictly required, but can be expected by some users. Notice that a new set_and_check command is used here. This is a function provided by the

CMakePackageConfigHelpers

module, all it does is sets the variable and ensures the pats are valid. The

check_required_components

function should be self explenatory. All that’s left to do is configuring and installing the file:

include(CMakePackageConfigHelpers)

configure_package_config_file(
    cmake/MathsConfig.cmake.in                      # the template file
    ${CMAKE_CURRENT_BINARY_DIR}/MathsConfig.cmake   # destination
    PATH_VARS CMAKE_INSTALL_INCLUDEDIR
    INSTALL_DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/Maths
)

install(FILES
    ${CMAKE_CURRENT_BINARY_DIR}/MathsConfig.cmake
    DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/Maths
)

What happens here is the Config template is used to generate the MathsConfig.cmake file, we also need to provide the information about where this Config file will be installed, so that the @PACKAGE_INFO@ tag is expanded correctly. We then actually install the generated Config into the same directory into which the Maths.cmake and Maths-debug.cmake were installed. Also note the PATH_VARS argument – variables passed in here, need to be prefixed with PACKAGE_ in the template. This ensures that the path is correctly expanded to a path prefixed in the package installation directory.

Place all of the above commands at the bottom of the top-level CMakeLists.txt for now. Aside for the template file – it should be located in the cmake directory. Let’s run the install target and inspect the result.

$ cmake --build build --target install
Install the project...
(...)
-- Installing: /home/user/cmake_fundamentals/install/lib/cmake/Maths/MathsConfig.cmake

As you can see the MathsConfig.cmake file has been installed alongside the library. And that’s it really. The package installation is now well behaved, relocateable and supports the find_package command. Before I prove to you that all this works, there’s a few more improvements that we could apply.

Supporting multiple configurations

If you look closely at the installed scripts, one of them is named Maths-debug.cmake – it’s configuration-specific. This implies that there might also need to exist Maths-release.cmake, and possibly other. The current setup could be considered fine if we required that each configuration is installed into a different install-prefix directory. This requirement is limiting. A common convention is to ensure that both debug and release build of the library can coexist by adding a suffix to the debug configuration. This is done by specifying an appropriate property on the target:

set_target_properties(Add
    PROPERTIES
        DEBUG_POSTFIX _d
)

This will cause _d to be appended to the binary resulting from debug builds of the Add target. Of course the same needs to be done for all targets in the project.

If you install the project now you’ll see that all the libraries have the newly specified suffix.

$ cmake --build build --target install
Install the project...
-- Install configuration: "Debug"
-- Installing: /home/user/cmake_fundamentals/install/lib/libAdd_d.a
-- Installing: /home/user/cmake_fundamentals/install/lib/libSubtract_d.a
-- Installing: /home/user/cmake_fundamentals/install/lib/libMaths_d.a
(...)

Some further considerations would be to add versioning and support for shared libraries – as is, if the project was built as a shared library it could be installed just fine, maybe even linked to, but you wouldn’t have much luck running the final binary. But like I already mentioned at the start I’d rather cover this topic in a dedicated post. For now, let’s briefly address how to add versioning to packages.

Versioning

Setting a project version number is rather simple – all it takes to do is is to pass an additional argument to the very first project command call:

project(Fundamentals VERSION 0.1.0)

This will have the result of the following cache variables being defined:

Variable	Value
PROJECT_VERSION	0.1.0
PROJECT_VERSION_MAJOR	0
PROJECT_VERSION_MINOR	1
PROJECT_VERSION_PATCH	0

For completeness sake, I’ll just mention that a PROJECT_VERSION_TWEAK variable is defined as well, but this isn’t compliant with semver, so I’m choosing to ignore it (it’s debatable if semver makes sense in the context of C++, but that’s a different post). In addition to the above, there’s also a project-call-specific variable defined, corresponding to each of the above. This follows the pattern of <project-name>_VERSION, etc. This might be relevant in a larger project where you might want to package and version certain components of the project separately.

So the project has a version, great. How do we actually propagate that to the installed package? CMakePackageConfigHelpers has you covered. In addition to the configure_package_config_file function it also exposes write_basic_package_version_file, which does exactly that. It generates a version file, which then also needs to be installed, same as the Config file. An exaple follows.

write_basic_package_version_file(
  ${CMAKE_CURRENT_BINARY_DIR}/MathsConfigVersion.cmake
  VERSION ${PROJECT_VERSION}
  COMPATIBILITY SameMajorVersion
)

install(FILES
    # ...
    ${CMAKE_CURRENT_BINARY_DIR}/MathsConfigVersion.cmake
    DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/Maths
)

Note the specified COMPATIBILITY – this let’s find_package determine if the package found is compatible with the one requested by the consuming project. SameMajorVersion is compatible with the semver concept. The following alternatives are available: AnyNewerVersion, SameMinorVersion, ExactVersion. You must decide for yourself which one suits your project at its current stage the best, taking into account API and ABI stability.

Installing the project now will result in the MathsConfigVersion.cmake file being installed, as expected:

Install the project...
(...)
-- Installing: /home/user/cmake_fundamentals/install/lib/cmake/Maths/MathsConfigVersion.cmake

And that’s it. Shared library support aside, this is a complete basic library packaging setup. The only thing to do is testing if it all works as expected.

Finding packages

The most direct way to test this solution would be to define a minimal project which uses it. Instead of defining something completely new I’ll just move the code associated to the MathsDemo executable to a separate project. I’ll spare you the details of what needs to be removed from the Fundamentals project, but here’s the complete setup of the test project:

directory tree:
.
├── CMakeLists.txt
└── main.cpp

CMakeLists.txt:

cmake_minimum_required(VERSION 3.15)
project(HelloInstalling)

find_package(Maths 0.1.0)

add_executable(MathsDemo)
target_sources(MathsDemo
    main.cpp
)

target_link_libraries(MathsDemo
    PRIVATE
        Maths::Maths
)

main.cpp (Same as before, moved from the Fundamentals project):
#include <iostream>
#include <limits>

#include <maths/maths.h>

void use_add(int a, int b)
{
    Maths::calc_status cs{};
    auto const result = Maths::add(a, b, cs);
    if (cs == Maths::calc_status::success)
    {
        std::cout << a << " + " << b << " = " << result << "\n";
    }
    else
    {
        std::cout << "Error calculating " << a << " + " << b << "\n";
    }
}

int main()
{
    const int a{42};
    const int b{102};

    use_add(a, b);
    use_add(a, std::numeric_limits<int>::max());
}

The interesting parts are in the CMakeLists.txt in bold. The find_package command call searches for the Maths package of the specified version (enforcing the compatibility specified in the package). Once the package is found, all of the targets defined in the MathsConfig.cmake file are available. In this case we’re linking the Maths::Maths target to the MathsDemo executable.

How does CMake know where to search for the package? By default it searches some platform-specific default directories, on a Unix-y system this is something like:

/usr/lib/cmake/<name>
/usr/lib/share/cmake/<name>
/usr/lib/<name>/cmake
(...)

This is just an example, see the documentation for the full story. What does this mean in practice? System-wide installed packages will be found without any additional actions needing to be done. What if we don’t want to install a package system-wide just for testing? Before the system paths are searched, find_package first searches all paths listed in the CMAKE_PREFIX_PATH variable. This is the best way of ensuring that a package is found without hard-coding any paths.

Let’s see an exapmle. In my case the cmake_fundamentals and the test_project are sybling directories:

.
├── cmake_fundamentals
│   └── install
└── test_project

The CMAKE_PREFIX_PATH needs to be set to the relative or absolute path of the cmake_fundamentals/install directory.

$ cmake -DCMAKE_PREFIX_PATH:PATH=cmake_fundamentals/install -B test_project/build -S test_project
-- Configuring done
-- Generating done
-- Build files have been written to: /home/user/test_project/build

$ cmake --build test_project/build/
Scanning dependencies of target MathsDemo
[ 50%] Building CXX object CMakeFiles/MathsDemo.dir/main.cpp.o
[100%] Linking CXX executable MathsDemo
[100%] Built target MathsDemo

$ test_project/build/MathsDemo
42 + 102 = 144
Error calculating 42 + 2147483647

Summary

In this post I demonstrated how to set up simple packaging for a library. This is a basic but solid skeleton, suitable for extension to accomodate project needs. A first good step would be to encapsulate all of the installation logic scattered across the project into a module. In the next post I’ll follow up on the packaging and demonstrate how to set up projects to support the FetchContent use case – thus making it possible to embed a dependency the same way we did googletest.

One more thing. No more posts this long. I promise.

 

References

• Git gist
• GNUInstallDirs
• CMakePackageConfigHelpers
• find_package
• FetchContent