[Scons-dev] Support for Tools via a pip package / python module in sys path

Tue Aug 1 11:04:10 EDT 2017

Just to follow on the below
I've recently found I can do what I want to do without making any
functional changes to scons
but I put in a pull request for some changes to the user manual to make it
clear on how to use toolpath since it doesn't seem to be documented all
that much
the trick I found was just to put sys.path into the toolpath argument so
that it can pick up on tools installed via a package management system like
pip

https://bitbucket.org/scons/scons/pull-requests/493/updates-to-user-manual-tests-for-the-use/diff#comment-None

I'm still having problems generating the docs under the head sources though
I had to copy the xml files to a 2.5.1 tree just to build them into html to
check how they would come out

I had a root through the list of bugs but I don't think this one is listed
yet
Under the latest sources

doc/user/tasks.xml is built to build/doc/generated/examples/tasks_ex1_1.xml

tasks.xml is the same / no change between 2.5.1 and 3.0 latest
but the resultant tasks_ex1_1.xml is different

under 2.5.1 it comes out as
```
<?xml version="1.0" encoding="UTF-8"?>
<screen xmlns="http://www.scons.org/dbxsd/v1.0" xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://www.scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd">%
<userinput>scons -Q</userinput>
cc -o app main.cpp
cat < foo.bar2 > foo.cpp
cc -o app2 main2.cpp foo.cpp
cat < test.bar > test.h
</screen>
```

under 3.0 latest it comes out as
```
<?xml version="1.0" encoding="UTF-8"?>
<screen xmlns="http://www.scons.org/dbxsd/v1.0" xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://www.scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd">%
<userinput>scons -Q</userinput>
[?1034hcat < test.bar > test.h
cc -o app main.cpp
cat < foo.bar2 > foo.cpp
cc -o app2 main2.cpp foo.cpp
</screen>
```

I think the difference leads to an error of
```
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
__xinclude_lxml(["scons_xi.xml"], ["main.xml"])
__build_lxml(["scons_ex.xml"], ["scons_xi.xml"])
scons: *** [scons_exi.xml] XSLTApplyError : Cannot resolve URI
../generated/examples/tasks_ex1_1.xml
scons: building terminated because of errors.
scons: *** [build/doc/user/index.html] Error 2
scons: building terminated because of errors.
```

Since the source file is the same I think it's something inbetween
if you want me to file this as a bug then let me know

Many Thanks
Richard

On 20 July 2017 at 21:21, RW <garlicbready at googlemail.com> wrote:

> Hi Bill,
>
> # Current Search Mechanism
>
> From what I understand of the toolpath search mechanism it searches:
> 1. within scons\engine\SCons\Tool for the tools bundled with scons
> 2. A list of directories specified within the toolpath parameter
> 3. A variable called DefaultToolpath within the scons source (by default
> is empty)
>
> I checked experimentally and it looks as if the syspath is not included in
> the search for tools so I don't think it can pick up on modules at present
> when searching for tools.
>
> I've already added in the dot syntax in a prior commit to do something
> different
> The dot syntax is currently coded to search sub-directories within
> toolpaths
> so if you have a toolname like 'Test1.TestBuilder1'
> and a list of toolpaths like
> C:\Lib\toolpath1
> C:\Lib\toolpath2
> C:\Lib\toolpath3
>
> The places it's going to search will be something like
> scons\engine\SCons\Tool\Test1\TestBuilder1
> C:\Lib\toolpath1\Test1\TestBuilder1
> C:\Lib\toolpath2\Test1\TestBuilder1
> C:\Lib\toolpath3\Test1\TestBuilder1
>
> I could change that around so that the dot means something else
> and use another character such as forward slash (pythonesque) as the
> directory separator instead.
> That might be a better idea
>
> # Overrinding
>
> For overriding, it's currently just currently adding to the toolpath
> parameter behind the scenes
> so the behaviour will be the same as that, I'm not sure how that works
> currently
> If toolpath overrides the inbuilt tools then this should work the same way
>
>
> # Suggested syntax
>
> There's a few different ways of changing the syntax around
> To give some examples of what I'm thinking based on what you've said
>
> 1. With the below example I've change the dot to a forward slash within
> the toolname
> "TestBuilder1" - look for tool in root of toolpath directory
> "ExampleDir1/TestBuilder1" - look for tool in sub-dir of toolpath directory
> "ExampleDir1/ExampleDir2/TestBuilder1" - look for tool in sub-dir of
> toolpath directory
>
> 2. This example uses a # prefix in the toolname to specify to use a python
> module as the base search directory (add to the toolpath)
> "scons_toollib_example#TestBuilder1"
> "scons_toollib_example.submod1#TestBuilder1"
> "scons_toollib_example#ExampleDir1/TestBuilder1"
> "scons_toollib_example#ExampleDir1/ExampleDir2/TestBuilder1"
>
> 3. We could also keep the toolmods option to avoid using a prefix as well
> all that will do is find the directory where the module is located and add
> that directory to the toolpath list
>
>
> Any thoughts?
>
>
> Many Thanks
> Richard
>
>
> On 20 July 2017 at 20:05, Bill Deegan <bill at baddogconsulting.com> wrote:
>
>> Wouldn't this work and not require any changes?
>> Or if it doesn't already work, wouldn't that be easier syntax? no
>> additional arguments, if it's got a '.' in it then you know it's part of a
>> package?
>>
>> toollist = ['scons_toollib_example.TestBuilder1']
>> env = Environment(ENV = os.environ, tools = toollist)
>> env.TestBuilder1('testdest1.txt', 'testsrc1.txt')
>>
>> How would you handle overloaded tool names in a module?
>> For example
>> scons_toollib_example has a gcc tool?
>>
>> -Bill
>>
>>
>>
>> On Thu, Jul 20, 2017 at 2:53 PM, RW via Scons-dev <scons-dev at scons.org>
>> wrote:
>>
>>> Hi,
>>> One of the features I'm interested in is the ability to have tools
>>> loaded externally via a package installed via pip.
>>> This way if you wanted to, you could ether install tools as a separate
>>> repo via pip, or move some of the existing tools into separate repo's
>>>
>>> I think I've come up with a way to do it which doesn't involve much code.
>>> But before I add in any tests or additions to the man page / user docs I
>>> wanted to check if this is an okay way to implement this
>>>
>>> I've put the commit here:
>>> https://bitbucket.org/grbd/scons/commits/af3ea9f2d4468479f7f
>>> 7be73afd455b0f2067409
>>>
>>> With this method there's an additional option on the environment
>>> constructor called toolmods
>>>
>>> ```
>>> toollist = ['TestBuilder1']
>>> env = Environment(ENV = os.environ, tools = toollist, toolmods =
>>> ['scons_toollib_example'])
>>> env.TestBuilder1('testdest1.txt', 'testsrc1.txt')
>>> ```
>>>
>>> This option looks up the directory where the module is located in the
>>> sys path and adds it to the toolpath
>>> (e.g. C:\Python27\Lib\site-packages\scons_toollib_example)
>>>
>>> A side effect is that the module file gets imported at the same time
>>> (scons_toollib_example\__init__.py is run)
>>> I just do this to get the full path to the module when adding to the
>>> toolpath, although you could probably use that to modify scons or do
>>> something else when the tool is referenced if you wanted to.
>>>
>>> Is this implementation is a good idea? if so I'll look into adding some
>>> tests and user doc entries for it before doing a pull request
>>>
>>> Many Thanks
>>> Richard
>>>
>>> _______________________________________________
>>> Scons-dev mailing list
>>> Scons-dev at scons.org
>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20170801/42a614b9/attachment.html>