[Scons-dev] SCons-like build system
Kenny, Jason L
jason.l.kenny at intel.com
Wed Jan 21 09:47:38 EST 2015
Yes posix_spawn will make a big difference on posix based systems. More so as the build get larger and the build system needs more memory. Windows default spawn logic is more like posix_spawn so it does not suffer in this case.
The core issue is that a build system spawns lots of processes to do work ( compiler, linkers, etc..). the common spawn logic in posix system is to fork, which requires the system to setup a copy on write of the build system memory to "child" process. As it turns out this is awful for a build system as the child process never touches the "shared" memory from the process, however the parent process is busy touching it as it setting states, etc. This requires system to copy all the memory we are writing to in the build system. Given that the build system will in general being spawning lot of process. This leads to a worse cases memory access pattern on any modern chip architecture, more so as the build gets larger and the build system needs more memory. Given the posix spawn avoids the cloning the memory, the memory access pattern goes to a much better case ( ie no copy of memory to new page tables, which lead to lower tlb misses and then being more likely we are in cache)
Jason
-----Original Message-----
From: Scons-dev [mailto:scons-dev-bounces at scons.org] On Behalf Of Constantine
Sent: Wednesday, January 21, 2015 6:19 AM
To: dl9obn at darc.de; SCons developer list
Subject: Re: [Scons-dev] SCons-like build system
Hello Dirk,
I tested this SCons version with memory optimizations on 10000 C++ files.
Updated graphs can be found here:
https://github.com/aqualid/aqualid/wiki/Benchmark-results-for-v0.5
I see about 20% improvement.
As I understand using posix_spawn function should improve performance on Linux and OSX.
Best regards,
Constantine.
On 01/13/15 21:55, Dirk Bächle wrote:
> As a follow-up of these investigations I have prepared a patch that
> switches SCons (especially the Node class) to using slots, in order to
> reduce the overall memory consumption. If you're interested in trying
> it out with your benchmark, you can get a copy with
>
> hg clone http://bitbucket.org/dirkbaechle/scons_experimental -r
> switch_node_to_slots
>
> _______________________________________________
> Scons-dev mailing list
> Scons-dev at scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org
https://pairlist2.pair.net/mailman/listinfo/scons-dev
More information about the Scons-dev
mailing list