[Scons-dev] python3-port branch merged to default and then closed

[Tim Jenness]

> On Jun 7, 2016, at 04:46 , Russel Winder <russel at winder.org.uk> wrote:
> 
> On Mon, 2016-06-06 at 15:59 -0700, Tim Jenness wrote:
>>>  […]
>> Russell — I’m taking a look again at python3 scons. Am I now meant to
>> be looking at the default branch for scons on bitbucket? Is there
>> work happening on a new branch anywhere?
> 
> I assume that means me (Russel) :-)
> 

Yes. Sorry.

> Yes. Bill did a whole lot of work, and so now there is only the
> mainline repository with it's default branch. Any and all changes must
> ensure that all the tests pass when run on Python 2.7.x (I am guessing
> there is a minimum for x here as 2.7 changes so much!), and should
> improve things when run on Python 3.4+.
> 
> All pull requests should be against mainline default, and one of the
> core team will either commit it or call for a discussion. 
> 

I’ve had a quick go at running the bootstrap.py with python3. Here is what I’ve learned:

Start by running “futurize -w .” on the entire tree.


	• When running bootstrap.py it fails when trying to load “rpm” because there is a directory called “rpm” in the scons root and that gets imported as a namespace package. Renaming “rpm” to “rpm-spec” fixes that.
	• When reading from subprocess you need to decode the bytes (futurize does not do that for you). (using .decode() works most of the time).
	• SCons_revision deliberately opens files as binary so the strings being inserted have to be encoded to bytes first.
		• similarly write_src_files
		• I’m not entirely sure why there are so many files being opened in binary mode that seem to be text files. It would be much less confusing if text files were opened in text mode.
	• I can’t build the windows binary targets on my Mac because the mbcs encoding is missing so I am skipping that.
	• os.path.walk is missing in python3 and replaced with os.walk (which is iterable). os.walk also works on python2.7 and the fix is very easy (and it is much cleaner) so should be applied independently of all this other python3 work.

With these changes I can get quite a long way through bootstrap and fail at :

scons: *** [build/scons-src-2.5.0-stamp] TypeError : can't use a string pattern on a bytes-like object

This is after the zipping runs. I don’t know where that error is coming from and there is no context other than the attempt to write the stamp file. In the bootstrap environment how would I work out how to get a better error message? The fix will be easy once I know what line is causing the exception.

Weirdly, in another checkout it gets stopped at the Digestify stage with md5. (md5 needs bytes: I’m looking at that problem at the moment).

The main issue I see is all the binary file opening going. Almost all the problems are related to binary vs str mismatches.

Of course, the futurized scons doesn’t work on python2 as it complains early on about an issue with metaclasses. Since I don’t know the scale of the python3 issues I looked at those first.

Apologies for this but I’ve been unable to quickly get my head round hg so I’ve put scons on github and opened a pull request there. If you want to see what I’ve done take a look at: https://github.com/timj/scons/pull/1 <https://github.com/timj/scons/pull/1>
I felt it was better to put the work somewhere rather than have everyone rediscover the issues.

As an aside, sometimes kw=[] keyword arguments are used. These are dangerous as the same default will be passed into the function each time and any modifications to it will persist. Should be kw=None and then assign a fresh [] (unless you want it to be persistent of course).

— 
Tim Jenness

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20160609/e12f2898/attachment.html>