[Scons-dev] Extensions to the Tool subsystem...

[Dirk Bächle]

Hello developers,

based on my proposed changes to the current tests in src/test there has 
been some discussion about how a Tool should work. Especially in 
connection with the LaTeX Tool, questions like:

   - Do we want to have one "latex" Tool for all, or separate ones for 
"miktex", "texlive"...?
   - Should a Tool try to find a "Miktex" installation in the current 
system, or simply search for
     the "latex" executable while relying on a correct setup of the PATH 
variable?

showed up.

Before simply repeating my pull request of the changes mentioned above, 
I'd like to reach a consensus about some basic guidelines and design 
decisions for the Tool subsystem. I reread the PlatformToolConfig page 
in the Wiki, describing the design of the long-planned IAPAT extension. 
Its requirements are partly taken into account, but I'd like
to keep the "GnuBS-like configure context" out of this discussion. My 
idea is to come up with a plan for our Tools, that hopefully makes it 
easy to add all the "configure" stuff later.

I also want to make a distinction between:

   - defining the Tool (and its supporting classes) as basic class in 
the framework and
   - how we use this Tool module to organize the build/configure/test 
workflows in our
     default implementation of SCons.

On the PlatformToolConfig page, the following remark can be found:

<quote>
(Comment: As I was writing this page, I found myself flipping back and
forth as to whether a Tool module configured a tool (that is, a single
command) or a toolchain (that is, a series of commands). The current Tool
modules actually implement toolchains (e.g., the gcc.py module provides
the environment variables for the compiler, the linker, the static archiver,
the shared archiver, and the bundle archiver). This isn't good
modularization, which suggests that there should be a higher-level
module explicitly for toolchains that can invoke one or more tool
modules as building blocks. That isn't in this proposal (should it be?),
but it's something that should be kept in mind for the future.)
</quote>

Since we try to provide a framework for build systems, our Tool shouldn't
care. Both should be possible ways for a user to extend the build engine.

With this being said...

What does a Tool do?
=============

A Tool has the task of changing an existing Environment. It can alter 
construction variables like CC or the
ENV['PATH'] and is most often used to add Builders or modify existing ones.



What can the Tools subsystem do?
=====================
(My christmas wishlist ;) for the SCons Tools)

PlatformInfo
########

I'd like to have a common module for detecting, storing and retrieving 
infos about the current host architecture, (platform, processor type, 
vendor, kernel, OS,...). It should be used throughout the whole source 
tree, including the test framework and the tests themselves.

Toolchains/Tools
##########

A Toolchain class should get added, as an abstraction for a series 
(=list) of tools being initialized by a single keyword.
It should be possible to:

     + Check whether we can load/use a toolchain or single tool in our 
current environment (as given by os.environ['PATH']).
     + Check whether we can load/use a toolchain or single tool in a 
special environment.
     + Get a list of possible toolchains for a "Tooltopic" (can't think 
of another name right now). A "Tooltopic" would be "C++", and possible 
toolchains include "mingw" and "msvs". This selection would probably 
depend on the PlatformInfo.
     + Get a default toolchain for a topic, that is actually installed 
in the current system.
     + Dynamically add new toolchains.
     + Dynamically add new tools to a toolchain.
     + Dynamically change the preferred order in which toolchains are tried.
     + Add new Toolchains system-wide, that contain parameterized 
versions of existing tools, e.g. a "cxx-embedded" for a cross-compiling 
gcc that requires special options.

The single Tool support as it is now should still be available and work 
as expected.

External Tools
#########

For better support of the external tools we could use:

     + A way to install a Tool in the local SCons distribution and
     + to deinstall it again.
     + SCons should be able to display (--version) that there are 
external tools installed and, on request (--list-external) which Tools 
exactly (and in which version!).
     + For this, single tools should support a version string.


Questions!
=======

- How do we want our Tools to be organized for the standard SCons 
implementation? Especially, when we have a Tool like "latex" that 
basically looks the same for all distributions (miktex, texlive, ...) in 
terms of command calls.
   Do we want a "miktex" Tool under the covers, that gets automatically 
selected by the "latex" Toolchain? Should there be a LatexCommon.py in 
cases like this? And how do we go about tests for these toolchains? Do 
we stop using live tests and always provide a fake LaTeX distribution, 
like for the C compiler and linking tests?

- Is a Tool ultimately responsible for detecting paths to possible 
alternatives of executables? Or is this the task of a configure context?



If you decide to contribute to this discussion, please try to keep it at 
a higher level and to the point. I plan to come up with a concrete 
design in the form of a SEP, that can then be further discussed on the 
Wiki. Thanks a lot in advance for your efforts.

Best regards,

Dirk