C++ incrementals builds for continuous integration

We have a fairly large C++ project built in VS2005 that can take up to 40 minutes to compile and build from scratch, and a further 10 minutes for the installers as the software is being built in both 32-bit and 64-bit configurations. I would like to reduce this time to somewhere around 10 minutes at least as I feel it's important to get fast build feedback when using continuous integration.

When using incremental builds by deleting the final linked files but not the .obj files the build process seems to go much faster, however errors seem to pop up here and there, such as .dlls not being able to be loaded. From a clean build everythin开发者_如何转开发g works fine. I am using TeamCity as the CI system of choice.

Maybe the incremental build behavior is better in later versions of Visual Studio and it might be a good motivation to upgrade ? Has anyone encountered similar issues?

I can't actually answer your question, but since you're looking for speeding up your builds, this blog post about fast builds with Visual Studio 2010 and an SSD might be helpful. http://qualapps.blogspot.com/2010/09/lightning-fast-builds-with-visual.html

Good question.

When I was putting together a CI system for a large C++ project for Microsoft Visual Studio 2003/2005/2008 I enounteded problems as well with incremental builds. Especially when using pre-compiled headers it seems to not work under all circumstances to do an incremental build. I would be interested hear if someone has a detailed explanation of this, i.e. what works and what doesn't.

In my case, the project took more than one hour to build from scratch, so in order to have some reasonable speed of feedback intraday, I ended up doing a clean nightly build which was the release build and intraday I used incremental builds based on the nightly. This worked fairly well, except for cases when the incremental building wasn't able to pick up changes correctly and re-compile all necessary things. I tried this approach because I consider getting feedback fast very important and if the incremental build fails once a month or less, I was ready to live with the comprimise.

In general, I like to have things better than what I described above, so in other projects where it has been possible to get more hardware and re-organize the components to be built in parallel, I usually do complete re-builds. If you can do builds in parallel you can speed up building a lot.

Other things to consider are:

• include vs. forward declaration
• template usage
• general dependencies between things
• take out parts of projects as independent libraries, which can even be pre-built.

There are a lot of things which can be done to get building faster and I would in most cases consider these to incremental builds.

My experience is that often, especially for debug builds, the bottleneck is disk IO. It helps to compress the output and intermediate files directories. Also, I have seen some builds being faster when not using incremental linking feature (Enable Incremental Linking - No). Another setting to consider is to turn off browse information generation (Enable Browse Information - None). Also, do not forget to use the parallel compilation feature of VS 2005. Since often it is the IO that is the bottleneck, it helps to use more build threads than there are CPUs.

Never noticed the behavior myself, but I would first suspect Link Time Code Generation. For continuous integration, you could skip that.

I should also mention IncrediBuild. Throwing hardware at the problem, especially hardware that you already have, is a quick win.