[Scons-dev] SCons 3.0, sconsign files and Py2 vs Py3.. how to handle sconsign imcompatabilities

[RW]

It doesn't look as if PICKLE_PROTOCOL is changeable during the SConscript
phase so that might be an idea for a future version of scons
Keeping it at 2 for scons 3.0 sounds like a good idea

There's a couple of settings of

src\engine\SCons\compat\__init__.py
PICKLE_PROTOCOL = pickle.HIGHEST_PROTOCOL
(which is 4, for py3 and 2 for py2)

I also noticed
ACTION_SIGNATURE_PICKLE_PROTOCOL = 1 in
src\engine\SCons\Action.py

but I don't think that's used anymore

I tried setting the pickle protocol to 1 for PICKLE_PROTOCOL just to see if
that would avoid the rebuild behavior when switching between versions. But
it doesn't seem to.


On 10 September 2017 at 16:52, Tim Jenness <tjenness at lsst.org> wrote:

>
>
> > On Sep 10, 2017, at 08:32, RW via Scons-dev <scons-dev at scons.org> wrote:
> >
> > Okay so changing
> > pickle.loads(p)
> > to
> > pickle.loads(p, encoding='bytes')
> >
> > seems like a partial fix
>
> Ok. That would be desirable.
>
> > if there's a .sconsign.dblite file saved in python 2 with pickle
> protocol 2
> > python 3 won't crash now, instead it'll just rebuild the sources /
> ignore the current state
> >
>
> I think that’s a reasonable short term fix.
>
>
> > going the opposite way from python 3 to python 2 gives me the error
> > scons: *** [mkdocs.yml] ValueError : unsupported pickle protocol: 4
> >
> > I suppose you could force it to always use pickle protocol 2 (not sure
> how to do that)
> > the end result would be that it would always rebuild when going from py2
> -> py3 or py3 -> py2
> > so it wouldn't crash but ignore the build state (perhaps the file hashes
> are calculated differently?)
> >
>
> How about protocol 2 is used for SCons 3.0 to allow a workable transition,
> but then say that in the release in 6 months or a year’s time py3 will
> start using 4?
>
> It might be worth having a quick look at the hashing to work out why the
> rebuilds happen.
>
>
> > pickle protocol 4 might be a lot quicker than pickle protocol 2 btw I'm
> not sure
>
> Can this be benchmarked?
>
> —
> Tim Jenness
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20170910/bd285111/attachment.html>