[Scons-dev] Subprocess issue on Linux?

[Dirk Bächle]

On 02.04.2014 23:38, Gary Oberbrunner wrote:

>

> On Wed, Apr 2, 2014 at 4:51 PM, Dirk Bächle <tshortik at gmx.de 

> <mailto:tshortik at gmx.de>> wrote:

>

>     This idea may be feasible, but I'd rather try to get the actual

>     shell spawning to be as fast as possible. We have some valid

>     approaches for this, so let's try them out...maybe one of them is

>     fast enough, such that we don't have to care about the "extra"

>     work mentioned above anymore. Speeding up the spawn/fork stuff

>     would be more transparent to the user than trying to "detect"

>     which commands need a full shell and which don't.

>

>

> They're orthogonal.  Both useful, but either can be pursued 

> independently of the other.  Avoiding shells will be most valuable in 

> builds with lots of tiny commands (could halve the build time). 

>  Avoiding fork is most valuable when SCons is using lots of memory 

> (which it often is).  My sense, based largely on Dirk's research, is 

> fixing spawn has a bigger payoff right now.

>


Since we now know that the "fork" problem is something to care about, 
I'd really like to use the current momentum. I don't want to push 
anybody, I want us to have a clear vision about how to take steps in the 
right direction.

Is someone working on this right now? If not, who volunteers? Let's just 
communicate...and be on the same page about knowing *who* does *what*, 
and *when*.

What's a little behind this is, that I think this point has some 
strategic importance. It's a single issue that keeps a lot of users from 
switching to SCons. By getting it out of the way, we can present (and 
kind of "sell") our project much better in any media out there.
I have submitted a proposal for the poster session at the EuroPython 
2014 in Berlin. It's title is "How spawning many processes sequentially 
can kill performance", and if I could not only talk about our problem, 
but also present a solution that has some benefit for the Python 
community, that definitely carries some potential in my opinion.

So let's really get this crackin'! ;)

Best regards,

Dirk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://two.pairlist.net/pipermail/scons-dev/attachments/20140404/ab2313a1/attachment.htm>