[Scons-dev] Why are Builders excluded from env.Clone()?

[Jonathon Reinhart]

Hello everyone,

Let me start out with a quick experiment:

    env = Environment()
    env2 = env.Clone()
    print env['BUILDERS']['StaticObject'] is env2['BUILDERS']['StaticObject']

This surprisingly prints "True". The takeaway here is that builders are not
copied when environments are Clone()-ed. This seems to disagree with the
docstring for Environment.Clone() [1] which says:

    """Return a copy of a construction Environment.  The
    copy is like a Python "deep copy"--that is, independent
    copies are made recursively of each objects--except that
    a reference is copied when an object is not deep-copyable
    (like a function).  There are no references to any mutable
    objects in the original Environment.
    """

This behavior does appear to be intentional, however:

    builders = self._dict.get('BUILDERS', {})

    clone = copy.copy(self)
    # BUILDERS is not safe to do a simple copy
    clone._dict = semi_deepcopy_dict(self._dict, ['BUILDERS'])
    clone._dict['BUILDERS'] = BuilderDict(builders, clone)

BUILDERS is explicitly excluded in the semi_deepcopy_dict() call.

My questions:
 - Why are Builders explicitly excluded from the env.Clone()?
 - Would it be reasonable to add an optional argument to Clone()
   (e.g. really_deep) which causes Builders to not be excluded?


Some background:

Several times I've noticed that changes made by Tools can "leak" out into other
environments (than the one upon which the tool was called). For example,
consider the Cython tool [2] which does the following:

    def generate(env):
        env["CYTHON"] = "cython"
        env["CYTHONCOM"] = "$CYTHON $CYTHONFLAGS -o $TARGET $SOURCE"
        env["CYTHONCFILESUFFIX"] = ".c"

        c_file, cxx_file = SCons.Tool.createCFileBuilders(env)

        c_file.suffix['.pyx'] = cython_suffix_emitter
        c_file.add_action('.pyx', cythonAction)

        c_file.suffix['.py'] = cython_suffix_emitter
        c_file.add_action('.py', cythonAction)

        create_builder(env)

This code is consistent with the Tools included with SCons (e.g. gcc).

The big problem here is that c_file, cxx_file are *not* unique to the passed-in
environment. As my experiment above showed, builders are common to all Clone()s
of that environment.

This causes issues (that are incredibly difficult to track down!)
where Tools can
interact poorly, even when applied to different environments. Here's a scenario
using a hypothetical Zython tool that converts .py files to .c files:

    base_env = Environment(
        tool_path = ['somewhere'],
    )

    cython_env = base_env.Clone()
    cython_env.Tool('cython')

    zython_env = base_env.Clone()
    zython_env.Tool('zython')

Because add_action() is a misnomer and should be called set_action(), this
would result in the zython tool being *the* .py -> .c builder for all
environments cloned from base_env.

Of course a workaround for this is to create a brand new Environment(), and the
builders will be created new as well. This is inconvenient though, and
according to the documentation, shouldn't be necessary.

What can we do about this?


Best regards,

Jonathon Reinhart



[1]: Environment.Clone()
     https://bitbucket.org/scons/scons/src/3763c12a/src/engine/SCons/Environment.py#Environment.py-1377

[2]: cython Tool
     https://github.com/cython/cython/blob/master/Tools/site_scons/site_tools/cython.py