[Scons-dev] SCons performance investigations

[Dirk Bächle]

Hi there,


On 21.07.2017 17:39, Andrew C. Morrow wrote:
>
> Hi scons-dev -
>
> The following is a revised draft of an email that I had originally 
> intended to send as a follow up to 
> https://pairlist4.pair.net/pipermail/scons-users/2017-June/006018.html. 
> Instead, Bill Deegan and I took some time to expand on my first draft 
> and add some ideas about how to address some of th e issues. We hope 
> to migrate this to the wiki, but wanted to share it here first for 
> feedback.
>
>

and thanks to everyone who contributed to this interesting discussion so 
far. I'd like to chime in on the topic, but not by delving into 
"optimization"...you seem to have this working area covered nicely.

Instead I'm very much after somehow connecting all the important points 
that you brought up, with my own plans about the future of SCons' 
design. I've started to write down how I'd like the core sources to be 
more flexible, more lightweight and more eligible to things like 
subclassing and adding new classes or strategies.
What I'd like to propose as the overall strategy is, to not try and make 
the current "scons" executable to be more versatile. Here I mean 
"versatile" in the sense that it can be adapted to several different 
"build project scenarios", such that it will give the maximum of speed 
or safety in each case.
Rather I'd like to put this "versatility" into the SCons core and its 
classes. Strengthening the framework character of SCons again by 
allowing new frontends to be developed more easily.
One of these frontends could be a "scons-lite" (*) for "flat" and large 
C/C++ projects, where the main task is "compile classes into libs". By 
simply not supporting "override environments"

     env.Library('foo', Glob('*.cpp'), CPPFLAGS=['-wall','-g'])

we wouldn't have to "subst" so much and build/update times would go down.

This approach would allow us to leave the current "scons" frontend as it 
is, instead of adding more and more features and command-line arguments 
to it. This feels just right to me...I probably can't exactly explain why.

Finally, one more comment regarding performance. It's tempting to try 
and reach the build and update times of other tools like "ninja" and 
"make", but it's also okay to not fully reach them. We don't have to 
hide with our project, especially once the "stubprocess" wrapper has 
been integrated.
After all, there is a very important (in my opinion) distinction to be 
made between SCons and other build tools like the autotools or CMake: 
the latter two, and a bunch of others, are "Makefile" generators. They 
allow you to (re)create a build description for your project, that will 
ensure correct builds as long as the dependencies don't change. But the 
decision about when you should rerun this generator step is left to the 
user, which is a disadvantage in my eyes.
Since SCons is reconsidering all dependencies for every build, it's 
natural that we spend some more time than a "brute-force make". I guess 
what I'm trying to say here is: let's not over-optimize and by that 
destroy the "beauty of SCons". ;)

So, once again, I'd like to join the party with my (crazy?) "design 
ideas". Does this sound interesting, and if yes, how do we organize 
things to further discuss this?


Best regards,

Dirk


(*) : Naming is hard... :P