Make vs Ninja Performance Comparison

Ever since I started using CMake to handle generating my build files I have relied on Makefiles. Most linux distributions come with the make command so getting up and running doesn’t require too much effort. Make and its derivatives been around for almost 40 years and it’s an extremely powerful tool that can do many things beyond simply compiling code. There are cases where the flexibility and power of make are overkill in terms of compiling code and if you are willing to trade them with improved performance Ninja might be what you are looking for.

Ninja, written by Evan Martin is a build system that is focused on performance. It was designed for fast incremental builds and large projects in general. To quote the chromium project “Ninja is a build system written with the specific goal of improving the edit-compile cycle time”.

Ninja does not output information about the current progress of the build on more than one line. Warnings and Errors are output like normal. Make on the other hand will output a line for every single cpp file that was compiled and linked. So beyond the performance improvements Ninja has a higher signal to noise ratio than Make.

I wanted to measure the performance of Make and Ninja for several projects, namely chrono and ogre.

• Chrono 383 files
• Ogre 495 files

Compilation will be performed on an Intel Haswell i7-4770K CPU with 32GB of ram. The host OS is Arch Linux and the compiler used is GCC 4.9.2.

To generate files for Ninja use the -G flag in cmake to specify the generator type.

cmake -G Ninja

Commands used for timing

Timing was performed using the linux “time” command and all builds were done in memory and all output was redirected to /dev/null. Redirecting to dev null was performed so that only the timing output would be written to standard out. In practice > /dev/null 2>&1 should not be used.

for i in {1..8}; do make clean && time make -j $i > /dev/null 2>&1; done
for i in {1..8}; do ninja clean && time ninja -j $i > /dev/null 2>&1; done

Results:

Performance for increasing threads, all times are in seconds

Chrono

Compiling an already compiled project, compiling after touching a single file. (picked at random)

Ogre

Compiling an already compiled project, compiling after touching a single file. (picked at random)

Ninja and Make are very close in terms of performance. It has less overhead when performing incremental builds but for full builds any performance gains would be negligible.