Why does Delphi's compilation speed degrade the longer it's open, and what can I do about it?

My company has been running a large project on Delphi for more than a decade. Our codebase has been growing over the years and now stands at around 4 million lines of code. Compilation speed is becoming an issue. We've spent time weeding out unit circular referencing (a known cause of slow compilation) and examined every aspect of the setup. It gets to a point we can't improve it any further materially with what we can control.

At the moment, on a state of the art PC with 4 cores running Windows XP SP3 and Delphi 2006, start Delphi fresh and do a full build, it takes ~40 secs. Then, if we do another full build in the same Delphi se开发者_Go百科ssion immediately, it will take 1m 40s. Do another full build again, it will get worse. So on and so forth.

(We are well aware Windows itself caches files and this has a big impact on the compilation speed. The above figures are based on that the files are cached. We set up such scenario by getting Delphi to compile the project once, terminating it and then starting a new Delphi session. So while 40 secs doesn't see to be slow, it's only because the files are cached by Windows. And we do this in order to have an apple-to-apple comparison.)

What puzzles us is why the compilation speed gets worse. (We have observed in the past the slow down was worse if the project had a lot of unit circular referencing.) If we terminate Delphi and start a new session, the compilation time will get back to 40 secs. An even more interesting thing we've observed is, we can achieve the same speed "improvement" by clicking the "Cancel" button to abort the compilation and then do the full build right away. Compilation time will get back to 40secs too.

It appears to us Delphi's own cache of unit dependency isn't as efficient as building it from scratch and it is getting worse over time. And it also appears the Cancel button somehow clears this cache. What we are thinking is, if we can tap into the Delphi IDE subsystem that does this clearing, we can always maintain the compilation speed at its peak performance. But we don't know how.

Does anyone know what we can do?

We are still using Delphi 2006 as we haven't yet found a feasible way to port our large project to Unicode. I read on forums that the latest Delphi XE exhibits similar compilation speed issue with unit circular referencing. Anyone knows if Delphi XE has addressed the problem?

p.s. We are also aware that splitting the project into runtime packages can reduce compilation time. But for deployment and administrative reasons, we try to avoid using runtime packages.

If you build your application, here are some tricks to speed up the process:

• Erase all *.dcu before the build (del *.dcu /s);
• Run a good defragmenter on your corresponding hard drive;
• Put most of your source files in the same directory, and try to leave the IDE and Project library paths as short as possible, with the most used entries at first;
• Install DelphiSpeedUp.

Delphi 2007 should compile faster than Delphi 2006.

Delphi 2009/2010/XE would probably be slower: from user experiment, the implementation of generics and new RTTI made the compilation process more complex, and the actual implementation was found out to be slower e.g. than with Delphi 2007.

Update:

Did you try enabling the ProjectClearUnitCacheItem hidden menu entry?

I've this entry enabled either by the CnPack, either by DDevExtension (I don't know which one do this, probably the later). This could be used to clear the internal unit cache.

The gradual performance degradation could be due to some sort of memory leak or other bug in the compiler. Heaven knows D2005 and D2006 had enough of them! If you can't upgrade to a Unicode-enabled version of Delphi, you ought to at least update to D2007 (which I believe is still available from Embarcadero) for better stability.

Also, as Robert Frank mentioned in a comment, check out Andreas Hausladen's tools. Just a few days ago he released a patch that improves compilation speed quite a bit. Unfortunately, that specific feature is apparently only for D2009 and later, but a lot of his fixes help speed various things up, including the compiler.

It's well worth trying DelphiSpeedUp from Andreas Hausladen but that will only help IDE performance rather than compilation as I understand it.

The other idea that nobody has suggested yet is to use high spec solid state disks.

I recommend using 64 bit Windows 7 with a large amount of RAM for the best file caching performance.

Just be thankful your project isn't written in C++!

Consider building with runtime packages in-house, then building monolithic executables when sending code to a QA department or distributing your application.

This requires extra maintenance, but the dramatic increases in build times are worth it IMO.

We have a 2.4 MLOC project with about 40-50 smaller, supporting applications. When compiled against a group of runtime packages the project builds in about 500K lines and builds about 6 times faster (15 seconds vs. 90 seconds). Many of the smaller applications compile in one second or less because so much of the packaged code is shared.

You need to be sure to test the monolithic executable, not the packaged executable. But generally you shouldn't see too many behavior differences if you follow good coding practices.

This question have some more advice to get better compilation speed. Avoiding circular references and detect unused units (with CnWizards) are of greater effect.

Did you try to compile the code using a script command line?

Did recompiling from command line made the process stand on 40 seconds?

run from cmd "dcc32.exe" to see usage.

Update: I can't check it now, however you should try compiling from command line and see if you try to run from the ide, the ide should not recompile, and lets you run with debug.