Mon Feb 5 11:18:01 CET 2007

New toy for testing purposes (building in 30 minutes)

We are about to buy new system for the community build server I'm using to build We have some money and as a pure coincidence I have had access to a machine that would be close to our "dream machine" over the weekend. But it is a bit over our budget ;-)

The machine is a dual Xeon 5160 3GHz (two dual core Xeons) with 4GB RAM and fast disks (15k RPM). The system was Debian with gcc 4.1, gcj.

I collected some data from my build system (no system stuff, with binfilter, with mozilla from source, with SDK, build --all in transex3 and then build --all in instsetoo_native, all on environment, building DEB packages only).

en-US standard build without ccache: 3 hours 51 minutes.
The same build with hot ccache cache: 1 hour 41 minutes.

At this time we started to be a bit nervous because it was actually slower than our current Opteron based system which builds one language with hot ccache cache in 1 hour and 35 minutes even with slower disk subsystem and slower memory. But the cold ccache build is still twice as slow as on the Xeon based machine.

As the testing machine has four cores, we started to experiment with multi process builds. We first raised MAXPROCESS to 2 to allow two processes to run in parallel. This resulted in 1 hour and 26 minutes. Not that interesting gain...

So I decided to test module parallelism instead of in-module parallelism. This was more interesting :-)

With up to 2 modules built in parallel, the time was 55 minutes.
With up to 4 modules built in parallel, the time was 38 minutes.
With up to 8 modules built in parallel, the time was 34 minutes.
With up to 12 modules built in parallel, the time was 30 minutes.

The fastest build I was able to do was 29 minutes 59 seconds.

The effect of in-module parallelism was not that interesting, so "optimal" configuration was up-to-12 modules built in parallel. To confirm this, I started the build with ccache cache removed: 1 hour 13 minutes. The next build with hot ccache cache was again 30 minutes long.

To sum up: you *can* build in half an hour!.

Posted by Pavel | Permanent link | File under: