[Scons-dev] Subst() replacement?

[Jason Kenny]

Hi Bill,

I know in the past we have talked about various things to do to fix up Scons. One of the items was the subst engine we have. Currently for me the issue with Parts is that I use the Subst engine to pass data around between different components in a automated way. This works well minus two issues:

1) it can take time. However I have various tweaks in the object I pass to the subst call to help cache a number of cases.
2) stack depth. This is the major issue for me at the moment.

The issue with with 2) is seen with -j based builds which can lead to seg faults. Depending on the build, it is common to have some depth of 10-20+ components. This can lead to a stack of depth per thread of 250 to 300+. When doing a -j based build this blows the stack in a bad way.

one can easily get stack with this pattern:

...
File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 543, in substitute
    return self.expand(args, lvars)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 495, in expand
    return list(map(func, s))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 494, in func
    return conv(substitute(l, lvars))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 526, in substitute
    result = _dollar_exps.sub(sub_match, args)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 525, in sub_match
    return self.conv(self.expand(match.group(1), lvars))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 491, in expand
    return self.substitute(s, lv)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 543, in substitute
    return self.expand(args, lvars)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 509, in expand
    return self.substitute(s, lvars)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 543, in substitute
    return self.expand(args, lvars)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 495, in expand
    return list(map(func, s))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 494, in func
    return conv(substitute(l, lvars))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 543, in substitute
    return self.expand(args, lvars)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 495, in expand
    return list(map(func, s))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 494, in func
    return conv(substitute(l, lvars))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 526, in substitute
    result = _dollar_exps.sub(sub_match, args)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 525, in sub_match
...

I had mentioned in the past I want to not use the subst call as much, but change this with something else is non trivial at the moment. I do recall you had done some work on this. I am looking at trying to fix the subst logic in Scons . What I propose to do it to override the subst module in SCons with Parts. The goal would to be to show a better version we could the move in to Scons with minimum effort as well as allow me to move forward with out having to block on a new SCons drop.

I believe you had done work on this already. I was hoping I could try to start with that with hope this can be used to reduce the recursion issues in SCons. This seemed to be better than start from scratch.

Oh course if you have a better idea I am

Jason


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20190218/d99b765a/attachment.html>