[Scons-dev] Support for Tools via a pip package / python module in sys path
RW
garlicbready at googlemail.com
Wed Aug 2 05:26:19 EDT 2017
Thanks for the info that's quite useful
it turns out I already had python-libxml2 installed but after installing
python-libxslt1
(apt-get install python-libxslt1) that seemed to resolve that problem
I can now get as far as generating the user docs in html (which is good
enough for what I need)
For info there it's highlighted another problem further down the line when
generating the pdf's. It's probably something I have or haven't got
installed under Ubuntu mate but as long as I can get as far as html output
that's good enough for me anyways
There's quite a few of these
```
+---------------------------------------------------------------------
| In /home/ric/scons-temp/scons_patched/build/scons/engine/SCons/
| Variables/PathVariable.py:
| Import failed (but source code parsing was successful).
| Error: ImportError: No module named compat (line 43)
|
[..............
+---------------------------------------------------------------------
| In /home/ric/scons-temp/scons_patched/build/scons/engine/SCons/
| Warnings.py:
| Import failed (but source code parsing was successful).
| Error: ImportError: No module named compat (line 43)
|
[...................
```
Followed by this at the end
```
Warning: No information available for SCons.Util.UniqueList's base
collections.UserList
[..............................
Warning: No information available for
SCons.Variables.ListVariable._ListVariable's base
collections.UserList
[......................................Traceback (most recent call last):
File "/usr/bin/epydoc", line 13, in <module>
cli()
File "/usr/lib/python2.7/dist-packages/epydoc/cli.py", line 971, in cli
main(options, names)
File "/usr/lib/python2.7/dist-packages/epydoc/cli.py", line 791, in main
write_latex(docindex, options, options.action)
File "/usr/lib/python2.7/dist-packages/epydoc/cli.py", line 870, in
write_latex
latex_writer.write(options.target)
File "/usr/lib/python2.7/dist-packages/epydoc/docwriter/latex.py", line
199, in write
self._write(self.write_module, directory, filename, val_doc)
File "/usr/lib/python2.7/dist-packages/epydoc/docwriter/latex.py", line
217, in _write
write_func(f.write, *args)
File "/usr/lib/python2.7/dist-packages/epydoc/docwriter/latex.py", line
384, in write_module
self.write_class(out, var_doc.value)
File "/usr/lib/python2.7/dist-packages/epydoc/docwriter/latex.py", line
412, in write_class
out(self.base_tree(doc))
File "/usr/lib/python2.7/dist-packages/epydoc/docwriter/latex.py", line
504, in base_tree
width = self._find_tree_width(doc)+2
File "/usr/lib/python2.7/dist-packages/epydoc/docwriter/latex.py", line
538, in _find_tree_width
width = max(width, self._find_tree_width(base)+2)
File "/usr/lib/python2.7/dist-packages/epydoc/docwriter/latex.py", line
537, in _find_tree_width
for base in doc.bases:
TypeError: iteration over non-sequence
scons: *** [build/doc/scons-api/api.pdf] Error 1
scons: building terminated because of errors.
```
Many Thanks
Richard
On 1 August 2017 at 16:29, Bill Deegan <bill at baddogconsulting.com> wrote:
> Once we merge your pull request I'll try building on my system.
> I know there is occasionally some issue using lxml vs libxml2/libxslt.
>
> If you like I have a script to build python with libxml2 and libxslt.
> Just change the version string at the top and it should build a python
> which should work for the doc flow.
>
> https://github.com/bdbaddog/python_build_script
>
> -Bill
>
> On Tue, Aug 1, 2017 at 8:04 AM, RW <garlicbready at googlemail.com> wrote:
>
>> Just to follow on the below
>> I've recently found I can do what I want to do without making any
>> functional changes to scons
>> but I put in a pull request for some changes to the user manual to make
>> it clear on how to use toolpath since it doesn't seem to be documented all
>> that much
>> the trick I found was just to put sys.path into the toolpath argument so
>> that it can pick up on tools installed via a package management system like
>> pip
>>
>> https://bitbucket.org/scons/scons/pull-requests/493/updates-
>> to-user-manual-tests-for-the-use/diff#comment-None
>>
>>
>> I'm still having problems generating the docs under the head sources
>> though
>> I had to copy the xml files to a 2.5.1 tree just to build them into html
>> to check how they would come out
>>
>> I had a root through the list of bugs but I don't think this one is
>> listed yet
>> Under the latest sources
>>
>> doc/user/tasks.xml is built to build/doc/generated/examples/t
>> asks_ex1_1.xml
>>
>> tasks.xml is the same / no change between 2.5.1 and 3.0 latest
>> but the resultant tasks_ex1_1.xml is different
>>
>> under 2.5.1 it comes out as
>> ```
>> <?xml version="1.0" encoding="UTF-8"?>
>> <screen xmlns="http://www.scons.org/dbxsd/v1.0" xmlns:xsi="
>> http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www
>> .scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd">%
>> <userinput>scons -Q</userinput>
>> cc -o app main.cpp
>> cat < foo.bar2 > foo.cpp
>> cc -o app2 main2.cpp foo.cpp
>> cat < test.bar > test.h
>> </screen>
>> ```
>>
>> under 3.0 latest it comes out as
>> ```
>> <?xml version="1.0" encoding="UTF-8"?>
>> <screen xmlns="http://www.scons.org/dbxsd/v1.0" xmlns:xsi="
>> http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www
>> .scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd">%
>> <userinput>scons -Q</userinput>
>> [?1034hcat < test.bar > test.h
>> cc -o app main.cpp
>> cat < foo.bar2 > foo.cpp
>> cc -o app2 main2.cpp foo.cpp
>> </screen>
>> ```
>>
>> I think the difference leads to an error of
>> ```
>> scons: Reading SConscript files ...
>> scons: done reading SConscript files.
>> scons: Building targets ...
>> __xinclude_lxml(["scons_xi.xml"], ["main.xml"])
>> __build_lxml(["scons_ex.xml"], ["scons_xi.xml"])
>> scons: *** [scons_exi.xml] XSLTApplyError : Cannot resolve URI
>> ../generated/examples/tasks_ex1_1.xml
>> scons: building terminated because of errors.
>> scons: *** [build/doc/user/index.html] Error 2
>> scons: building terminated because of errors.
>> ```
>>
>> Since the source file is the same I think it's something inbetween
>> if you want me to file this as a bug then let me know
>>
>> Many Thanks
>> Richard
>>
>> On 20 July 2017 at 21:21, RW <garlicbready at googlemail.com> wrote:
>>
>>> Hi Bill,
>>>
>>> # Current Search Mechanism
>>>
>>> From what I understand of the toolpath search mechanism it searches:
>>> 1. within scons\engine\SCons\Tool for the tools bundled with scons
>>> 2. A list of directories specified within the toolpath parameter
>>> 3. A variable called DefaultToolpath within the scons source (by default
>>> is empty)
>>>
>>> I checked experimentally and it looks as if the syspath is not included
>>> in the search for tools so I don't think it can pick up on modules at
>>> present when searching for tools.
>>>
>>> I've already added in the dot syntax in a prior commit to do something
>>> different
>>> The dot syntax is currently coded to search sub-directories within
>>> toolpaths
>>> so if you have a toolname like 'Test1.TestBuilder1'
>>> and a list of toolpaths like
>>> C:\Lib\toolpath1
>>> C:\Lib\toolpath2
>>> C:\Lib\toolpath3
>>>
>>> The places it's going to search will be something like
>>> scons\engine\SCons\Tool\Test1\TestBuilder1
>>> C:\Lib\toolpath1\Test1\TestBuilder1
>>> C:\Lib\toolpath2\Test1\TestBuilder1
>>> C:\Lib\toolpath3\Test1\TestBuilder1
>>>
>>> I could change that around so that the dot means something else
>>> and use another character such as forward slash (pythonesque) as the
>>> directory separator instead.
>>> That might be a better idea
>>>
>>> # Overrinding
>>>
>>> For overriding, it's currently just currently adding to the toolpath
>>> parameter behind the scenes
>>> so the behaviour will be the same as that, I'm not sure how that works
>>> currently
>>> If toolpath overrides the inbuilt tools then this should work the same
>>> way
>>>
>>>
>>> # Suggested syntax
>>>
>>> There's a few different ways of changing the syntax around
>>> To give some examples of what I'm thinking based on what you've said
>>>
>>> 1. With the below example I've change the dot to a forward slash within
>>> the toolname
>>> "TestBuilder1" - look for tool in root of toolpath directory
>>> "ExampleDir1/TestBuilder1" - look for tool in sub-dir of toolpath
>>> directory
>>> "ExampleDir1/ExampleDir2/TestBuilder1" - look for tool in sub-dir of
>>> toolpath directory
>>>
>>> 2. This example uses a # prefix in the toolname to specify to use a
>>> python module as the base search directory (add to the toolpath)
>>> "scons_toollib_example#TestBuilder1"
>>> "scons_toollib_example.submod1#TestBuilder1"
>>> "scons_toollib_example#ExampleDir1/TestBuilder1"
>>> "scons_toollib_example#ExampleDir1/ExampleDir2/TestBuilder1"
>>>
>>> 3. We could also keep the toolmods option to avoid using a prefix as well
>>> all that will do is find the directory where the module is located and
>>> add that directory to the toolpath list
>>>
>>>
>>> Any thoughts?
>>>
>>>
>>> Many Thanks
>>> Richard
>>>
>>>
>>> On 20 July 2017 at 20:05, Bill Deegan <bill at baddogconsulting.com> wrote:
>>>
>>>> Wouldn't this work and not require any changes?
>>>> Or if it doesn't already work, wouldn't that be easier syntax? no
>>>> additional arguments, if it's got a '.' in it then you know it's part of a
>>>> package?
>>>>
>>>> toollist = ['scons_toollib_example.TestBuilder1']
>>>> env = Environment(ENV = os.environ, tools = toollist)
>>>> env.TestBuilder1('testdest1.txt', 'testsrc1.txt')
>>>>
>>>> How would you handle overloaded tool names in a module?
>>>> For example
>>>> scons_toollib_example has a gcc tool?
>>>>
>>>> -Bill
>>>>
>>>>
>>>>
>>>> On Thu, Jul 20, 2017 at 2:53 PM, RW via Scons-dev <scons-dev at scons.org>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>> One of the features I'm interested in is the ability to have tools
>>>>> loaded externally via a package installed via pip.
>>>>> This way if you wanted to, you could ether install tools as a separate
>>>>> repo via pip, or move some of the existing tools into separate repo's
>>>>>
>>>>> I think I've come up with a way to do it which doesn't involve much
>>>>> code.
>>>>> But before I add in any tests or additions to the man page / user docs
>>>>> I wanted to check if this is an okay way to implement this
>>>>>
>>>>> I've put the commit here:
>>>>> https://bitbucket.org/grbd/scons/commits/af3ea9f2d4468479f7f
>>>>> 7be73afd455b0f2067409
>>>>>
>>>>> With this method there's an additional option on the environment
>>>>> constructor called toolmods
>>>>>
>>>>> ```
>>>>> toollist = ['TestBuilder1']
>>>>> env = Environment(ENV = os.environ, tools = toollist, toolmods =
>>>>> ['scons_toollib_example'])
>>>>> env.TestBuilder1('testdest1.txt', 'testsrc1.txt')
>>>>> ```
>>>>>
>>>>> This option looks up the directory where the module is located in the
>>>>> sys path and adds it to the toolpath
>>>>> (e.g. C:\Python27\Lib\site-packages\scons_toollib_example)
>>>>>
>>>>> A side effect is that the module file gets imported at the same time
>>>>> (scons_toollib_example\__init__.py is run)
>>>>> I just do this to get the full path to the module when adding to the
>>>>> toolpath, although you could probably use that to modify scons or do
>>>>> something else when the tool is referenced if you wanted to.
>>>>>
>>>>> Is this implementation is a good idea? if so I'll look into adding
>>>>> some tests and user doc entries for it before doing a pull request
>>>>>
>>>>> Many Thanks
>>>>> Richard
>>>>>
>>>>> _______________________________________________
>>>>> Scons-dev mailing list
>>>>> Scons-dev at scons.org
>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20170802/4cbed41d/attachment-0001.html>
More information about the Scons-dev
mailing list