[Scons-dev] Subst() replacement?

[Jason Kenny]

Do you have a link for me to look at?

Else I was going to start by making a expression parser to allow the breaking up of the $ ${} $() $$ and ${${}} ( this last one is a feature improvement) style expression and then make tweaked versions of the existing classes in subst module to try to avoid recursion when it can.

Jason
________________________________
From: Scons-dev <scons-dev-bounces at scons.org> on behalf of Bill Deegan <bill at baddogconsulting.com>
Sent: Monday, February 18, 2019 5:05 PM
To: SCons developer list
Subject: Re: [Scons-dev] Subst() replacement?

Jason,

The code I have is still a work in progress.
Putting it in parts first doesn't make sense to me as when it's ready I'll be dropping it into SCons itself, though likely as a switchable alternative initially.

I'll be back to working on it in about a month I expect. Maybe sooner.

-Bill

On Mon, Feb 18, 2019 at 11:18 AM Jason Kenny <dragon512 at live.com<mailto:dragon512 at live.com>> wrote:
Hi Bill,

I know in the past we have talked about various things to do to fix up Scons. One of the items was the subst engine we have. Currently for me the issue with Parts is that I use the Subst engine to pass data around between different components in a automated way. This works well minus two issues:

1) it can take time. However I have various tweaks in the object I pass to the subst call to help cache a number of cases.
2) stack depth. This is the major issue for me at the moment.

The issue with with 2) is seen with -j based builds which can lead to seg faults. Depending on the build, it is common to have some depth of 10-20+ components. This can lead to a stack of depth per thread of 250 to 300+. When doing a -j based build this blows the stack in a bad way.

one can easily get stack with this pattern:

...
File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 543, in substitute
    return self.expand(args, lvars)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 495, in expand
    return list(map(func, s))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 494, in func
    return conv(substitute(l, lvars))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 526, in substitute
    result = _dollar_exps.sub(sub_match, args)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 525, in sub_match
    return self.conv(self.expand(match.group(1), lvars))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 491, in expand
    return self.substitute(s, lv)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 543, in substitute
    return self.expand(args, lvars)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 509, in expand
    return self.substitute(s, lvars)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 543, in substitute
    return self.expand(args, lvars)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 495, in expand
    return list(map(func, s))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 494, in func
    return conv(substitute(l, lvars))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 543, in substitute
    return self.expand(args, lvars)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 495, in expand
    return list(map(func, s))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 494, in func
    return conv(substitute(l, lvars))
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 526, in substitute
    result = _dollar_exps.sub(sub_match, args)
  File "/build-env/lib/python3.6/site-packages/scons/SCons/Subst.py", line 525, in sub_match
...

I had mentioned in the past I want to not use the subst call as much, but change this with something else is non trivial at the moment. I do recall you had done some work on this. I am looking at trying to fix the subst logic in Scons . What I propose to do it to override the subst module in SCons with Parts. The goal would to be to show a better version we could the move in to Scons with minimum effort as well as allow me to move forward with out having to block on a new SCons drop.

I believe you had done work on this already. I was hoping I could try to start with that with hope this can be used to reduce the recursion issues in SCons. This seemed to be better than start from scratch.

Oh course if you have a better idea I am

Jason


_______________________________________________
Scons-dev mailing list
Scons-dev at scons.org<mailto:Scons-dev at scons.org>
https://pairlist2.pair.net/mailman/listinfo/scons-dev<https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist2.pair.net%2Fmailman%2Flistinfo%2Fscons-dev&data=02%7C01%7C%7Cde0bea34038744b51bbc08d695f59d7b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636861279472300731&sdata=75yXbVakjeXi61d0cgZwJSSHZzYjgS3wSViRtqR0lBw%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20190218/12c0fd10/attachment.html>