[Scons-dev] Scanners

[Chris BeHanna]

On Aug 17, 2012, at 04:11 , TOM TANNER (BLOOMBERG/ LONDON) <ttanner2 at bloomberg.net> wrote:


> I was having a look at why my build was potentially slow, and I noticed that my perl scanner was being called multiple times for the same file.

> 

> The only reason I can think of for this is that the environments are different (they have a different ID), but they are only different for reasons that aren't going to affect the perl script.

> 

> One of my problems with performance is possibly due to the number of environment clones I have, partly due to the port from an existing build system, and partly because from (painful) experience in other somewhat similar situations, it's safest to clone the environment before calling an SConscript.

> 

> I'm in somewhat of an uncomfy position here. I could rewrite substantial portions of my script to avoid cloning the environment for individual objects, and rely instead on passing lists of keywords to individual builders, but it'd make the script more difficult to understand and maybe less stable. And I don't know how much effect it would have.


	Passing keywords results in the construction of an OverrideEnvironment instance that inherits most of its settings and dictionary from its parent, and then overrides dictionary key/value pairs from the keywords that you specified when you invoked the builder.  Thus, you're not necessarily saving that much.

	From what I understand, there is only one DAG in a build; however, you may have uncovered a real place for optimization:  if scons just blindly calls a scanner each time a file turns up as a dependency of another file, it's going to end up scanning these files multiple times.  If it were, instead, to lookup the file's MD5 hash in the DAG first, to see if it's already present, and skip the scanning step, lots of time would be saved.

	If I have described scons's behavior correctly, then this is not necessarily a pathology of your build; rather, it's a thing in scons itself that would need to be fixed (and that's a chore, as "the DAG" is split into many different areas in the code).

-- 
Chris BeHanna
chris at behanna.org