[Scons-dev] SCons 3.0, sconsign files and Py2 vs Py3.. how to handle sconsign imcompatabilities

[Bill Deegan]

Likely the rebuild py2 vs py3 is the action signature for functions now
uses the bytecode. The bytecode is different between py2 and py3.

Previously it was either pickling the function (which didn't adequately
check for changes in the python functions), or dumping the string
representation :
<function a at 0x10e88fc80>
For example.



On Sun, Sep 10, 2017 at 1:29 PM, RW via Scons-dev <scons-dev at scons.org>
wrote:

> It doesn't look as if PICKLE_PROTOCOL is changeable during the SConscript
> phase so that might be an idea for a future version of scons
> Keeping it at 2 for scons 3.0 sounds like a good idea
>
> There's a couple of settings of
>
> src\engine\SCons\compat\__init__.py
> PICKLE_PROTOCOL = pickle.HIGHEST_PROTOCOL
> (which is 4, for py3 and 2 for py2)
>

> I also noticed
> ACTION_SIGNATURE_PICKLE_PROTOCOL = 1 in
> src\engine\SCons\Action.py
>
> but I don't think that's used anymore
>
> I tried setting the pickle protocol to 1 for PICKLE_PROTOCOL just to see
> if that would avoid the rebuild behavior when switching between versions.
> But it doesn't seem to.
>
>
> On 10 September 2017 at 16:52, Tim Jenness <tjenness at lsst.org> wrote:
>
>>
>>
>> > On Sep 10, 2017, at 08:32, RW via Scons-dev <scons-dev at scons.org>
>> wrote:
>> >
>> > Okay so changing
>> > pickle.loads(p)
>> > to
>> > pickle.loads(p, encoding='bytes')
>> >
>> > seems like a partial fix
>>
>> Ok. That would be desirable.
>>
>> > if there's a .sconsign.dblite file saved in python 2 with pickle
>> protocol 2
>> > python 3 won't crash now, instead it'll just rebuild the sources /
>> ignore the current state
>> >
>>
>> I think that’s a reasonable short term fix.
>>
>>
>> > going the opposite way from python 3 to python 2 gives me the error
>> > scons: *** [mkdocs.yml] ValueError : unsupported pickle protocol: 4
>> >
>> > I suppose you could force it to always use pickle protocol 2 (not sure
>> how to do that)
>> > the end result would be that it would always rebuild when going from
>> py2 -> py3 or py3 -> py2
>> > so it wouldn't crash but ignore the build state (perhaps the file
>> hashes are calculated differently?)
>> >
>>
>> How about protocol 2 is used for SCons 3.0 to allow a workable
>> transition, but then say that in the release in 6 months or a year’s time
>> py3 will start using 4?
>>
>> It might be worth having a quick look at the hashing to work out why the
>> rebuilds happen.
>>
>>
>> > pickle protocol 4 might be a lot quicker than pickle protocol 2 btw I'm
>> not sure
>>
>> Can this be benchmarked?
>>
>> —
>> Tim Jenness
>
>
>
> _______________________________________________
> Scons-dev mailing list
> Scons-dev at scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20170911/d3a8222f/attachment-0001.html>