[Scons-dev] Toolchain requirements

[Dirk Bächle]

Hi Gary,

here are my comments:

On 25.05.2014 19:14, Gary Oberbrunner wrote:
> I'd like to kick off a round of discussion about toolchains, so can 
> make some progress toward a design.  I have some preliminary thoughts. 
>  Please comment.  Apologies for the HTML mail, let me know if this 
> isn't readable for you.
>
>   * Allow installing external tools (pip install or ...)
>  *
>       o scons --version (or similar) should list installed tools and
>         toolchains
>       o missing external tools should give sensible errors
>
I'm not sure I understand what you mean by "missing" here. An external 
Tool that isn't able to find the executables or packages it depends on 
in the current system? Like an overview, where you get an 
Available/NotFound for each Tool?

What I wouldn't like is, if we'd define a set of external Tools that the 
user would *have* to install. We can provide a package of external tools 
in a bundled fashion, something like our "best of". But each user should 
be free to install it, or only the single Tools he needs, without 
getting confusing messages about a Tool not being found, just because we 
think he needs to have it.

>   * Tool setup must happen before reading SConstruct somehow
>
I don't know if this is really required, see my comment below about 
"lazy initialization" of Tools. It would be good to have a pool of 
Platform-related info (which OS/Python/Kernel are we running on?) at 
this early stage, that all environments/tools can access directly.
>
>  *
>       o DefaultEnvironment and all new Environments should know about
>         all tools
>       o alternative: lazy-construct DefaultEnvironment
>       o user-specified tools and toolchains need to be specifiable at
>         beginning of build
>   * User should be able to set default tools and toolchains
>
...and the order in which they are probed, if possible this 
specification should be per OS.

>  *
>       o unused tools shouldn't take any startup time
>   * Lazy init of tools and chains
>  *
>       o This is faster because unused tools don't matter
>       o It allows missing unused tools to not give errors, but missing
>         used tools can (and should)
>       o But it makes configuring environments much harder for users,
>         because they can't override or append to tool-provided
>         variables until those exist.  This would break a lot of
>         existing SConstructs.
>       o We need to find some kind of compromise here:
>      o
>           + Explicitly list tools required by build (where?): this
>             should work well because only the needed tools will be
>             initialized
>           + if nothing explicitly specified, fall back to current method
>

We should also take the case into account where a user might set up a 
"dummy" environment. Depending on which Toolchain seems to be available, 
he then wants to setup his actual environments, or install additional 
packages first, or...

I, personally, think that we don't need lazy initialization here. If we 
give a user more option to control which Tools/Toolchains are probed in 
which order, he can size down the effort for detecting stuff nicely. At 
least, the latter (=toolchain control) should have higher priority over 
the certainly nice and cool "lazy init" feature.
Doing things "lazy", also makes things like debugging, writing test 
cases or creating issue reports a little harder...

>   * Within a tool:
>  *
>       o specify dependencies on other tools
>
What kind of dependencies are you thinking about? Module-wise, 
method-wise or both? Is it more about data dependencies, or order 
dependencies for the probing phase?

>  *
>       o detect existence on system reliably, and without modifying env
>      o
>           + need better error messaging: ability to probe silently,
>             but also give sensible errors when needed
>       o constructor needs to allow args: version, path, ABI, etc.
>         (this is important)
>       o allow for common setup (all C compilers, etc.) as now
>       o tools should be versioned so user can check if up to date, etc.
>   * Tool chains:
>  *
>       o either-or
>       o and
>       o collections
>   * Platform
>  *
>       o How much do we need to know about the platform, for tools to
>         initialize themselves?
>       o Cross-compilation comes into this, but may be too much to
>         include as a general part of this project.
>
It's really hard to draw the line here. For my personal taste, we should 
have a PlatformInfo with only the basic data about OS/CPU/RAM/Kernel and 
the Python version we're running under.
Everything else, like "latest Java version" or "available 
cross-compilers", should be handled by the rest of the toolchain/tools 
stuff somehow.

In general, I just don't want to have things getting all muddy and 
confused to fast. This will happen anyway down the road of 
implementation, faster than we can think. ;)
So let's try to be as clear and concise as possible right now at the 
beginning, and establish clear visions and borders about what we want to 
do, and what not.

The PlatformInfo is a good example for this, where an apparent crippling 
of functionality can save us some work, regarding the implementation 
*and* explanation. Let's not forget that we also have to document all 
this, in a clear and consistent manner!

>  *
>       o It may be useful to define toolchains and enable/disable them
>         by platform
>       o Of course the default toolchains need to be different by platform
>
+1!
>
>  *
>       o It may be possible for a default toolchain to just search for
>         all tools in a particular order and pick the first, as long as
>         the tool-dependency system is robust enough.
>   * Usability
>  *
>       o $CC etc. must never be left blank (without a prior
>         tool-missing error message at least) - this is a common problem
>
Do you want to introduce two classes of environment variables here, like 
"never-empty vs. may-be-empty"?

>  *
>       o Must be backward compatible, at least for all common cases.
>
But improved functionality and greater ease of use has the higher 
priority over backwards compatibility, right?

>  *
>       o Must not require any new user files (e.g. something in
>         site_scons) for normal operation
>       o Need a clear guide on requirements for new tools
>      o
>           + how to make a a tool
>           + how to include tests
>   * Considerations
>  *
>       o "batteries included?"
>      o
>           + Each tool should do its best to set itself up, find
>             executables, etc.
>           + What about SCons policy of not relying on $PATH?  Maybe we
>             should relax that or have an option?
>
My idea here would be a command-line option that automatically adds the 
current shell's $PATH to each created Environment, unless the user 
explicitly states a different setting for ENV['PATH'] in the 
Environment's constructor.
This would make it easier for a user to just "get things running", no 
matter what kind of security/stability risks this would involve.

SCons itself should continue to stick to the principle of "clean 
environments" as the default.

>  *
>       o minimum magic, maximum flexibility
>       o what about single tools?  Should every tool be required to be
>         part of a toolchain (even if it's just one tool)?  Maybe this
>         doesn't matter much.
>
I don't think this should be required. A user should be able to pick 
freely from a list of Toolchains and single Tools what he wants to use 
for his environment.


Finally, I'd like to add the two following points to the discussion, 
even if they might be a little off-topic:

- We still have problems with tools/programs like "LaTeX" and the 
"Docbook XSL stylesheets" that rely on relative paths, so the program 
has to be executed in the same folder as the source/target file. Maybe 
we can take up on the former work about getting SCons freed of all 
"os.chdir"s, such that we could then implement a proper "chdir" feature 
for Tools/Builders.

- The issue #1406 ( scons.tigris.org/issues/show_bug.cgi?id=1406 ) is 
still open, raising the question how to handle paths with spaces when 
they get passed in from the surrounding OS shell to a Tool's command or 
include paths.


So much for today.

Best regards,

Dirk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://two.pairlist.net/pipermail/scons-dev/attachments/20140529/9004c8ea/attachment-0001.html>