[Scons-dev] Default number of jobs

[William Deegan]

David,

Firstly, this is a discussion better fitted to the users mailing list.

Secondly as you've posted before it's easy to make the change from the SConstruct level to change the default per project.

In general SCons aims to be a make replacement, and this includes some of the behavior.
Changing the default to more than -j 1 would break this compatibility.

I'd be against any such change.

Gary - Opinions?

-Bill

On Jul 28, 2012, at 9:30 AM, David Stone wrote:


> SCons currently defaults to only 1 job. On multi-processor systems,

> this is sub-optimal. I recommend that we change the default behavior

> to be equal to the number of processors on their system. This can be

> done now by adding the following lines:

> 

> import multiprocessing

> SetOption('num_jobs', multiprocessing.cpu_count())

> 

> There is some debate about the exact amount of jobs to run, but most

> people agree that you want N * number of processors, with the debate

> being about what value of N to use. The most commonly cited values are

> 1, 1.5, and 2. I was in the 1.5 camp, but after doing some profiling,

> I found that 1 gives the best performance. On a 4 core Intel i5

> machine with 4 GiB of memory, these are my results:

> http://doublewise.net/misc/scons-build-jobs-table.png

> 

> zyp in the SCons chat ran similar tests on an Intel i7 machine with 32

> GiB of memory, 4 cores and 8 hardware threads (so the number returned

> by multiprocessing.cpu_count() is 8), and these are their results:

> http://paste.jvnv.net/view/izIQC

> 

> Both of our tests show pretty significant decreases in time up to the

> number of cores. In my case, cores == hardware threads, but it looks

> like when they are different, you still get small increases in speed

> as you increase to the number of hardware threads. Adding jobs above

> the number of cores gives a steady increase in time (and at some point

> the time taken would grow significantly as the computer runs out of

> memory). This was not found to be a significant risk for any realistic

> distribution of cores and memory.

> 

> In summary, virtually every user will benefit from having the number

> of jobs default to being exactly their number of hardware threads

> (which is what is returned by multiprocessing.cpu_count()), and there

> is almost no risk of losing performance to this change.

> _______________________________________________

> Scons-dev mailing list

> Scons-dev at scons.org

> http://two.pairlist.net/mailman/listinfo/scons-dev