[Scons-dev] Cross-language support

William Blevins wblevins001 at gmail.com
Wed Jul 29 22:50:30 EDT 2015 

On Wed, Jul 29, 2015 at 10:05 PM, Jason Kenny <dragon512 at live.com> wrote:

>   Ya I see this...
>
> So I trying to take a little time tonight and figure out what happens..
> Could use some help.
>
> I currently looking at test/CPPSUFFIXES.py
>
> This fails at line 107. From what I can tell the test expects output of
>
> C:\Python27\python.exe mycc.py test1.o test1.c
>
> but instead we get
>
> C:\Python27\python.exe mycc.py test1.o test1.c
> Install file: "test1.c" as "test1_c"
> Install file: "test1.h" as "test1_h"
>
> This happens because foo.h was changed causing these files to be copied
> again.
>
> So problems I have with this test.
>
> 1) no one would copy headers that depend on a header not passed to the
> install area. I am not sure if we should treat <> different from “”
> includes. It a common practice to use “” for headers in your library and <>
> for stuff outside your library. I don’t this really helps either way
> 2) the copy/install builder does not generate new file if the source is
> different. What we have here is a good scanner that would do the right
> thing if the “binary” needs to be rebuilt, but redundant is the copy has to
> happen.
>

Gary and I were of the same opinion.  The functionality is correct and
matches all other scanner/builder behavior, but it is inefficient...


> The issue in a way is that SCons makes the installed test1_c out of date
> because foo.h is out of date, when in fact this is a case in which we would
> want to update logic to care more about the csig of the source being the
> same as what is stored and not doing anything as technically this is
> correct behavior for copy like builders and ignoring that the scanner
> header is out of date.
>
>


> I see few solutions to this:
>
> 1) we say current behavior is correct and update the test.
>

This is the easiest, but I was hoping to resolve this performance issue.


> 2) we make a change to allow a install/copy builder to say the scanner
> items are Required() which remove the Depends() state
>

I looked into doing this, but I didn't see an easy way to do this.  I made
some notes:
https://bitbucket.org/scons/scons/pull-requests/237/issue-2264-cross-language-scanner-support/diff#comment-8502930


> 3) we change builder to apply a custom sig check for the install nodes to
> ignore that a file dependent from the scanner.
>

This seems like a reasonable approach unless this means updating every
scanner.  The fundamental problem here is that language scanners already
exist that exhibit correct behavior.  I don't want to duplicate any part of
the scanner code.  I feel like going this route will require some level of
duplication.


> 4) remove the scanner from the install builder as we only want to copy
> files that change
>

I assume this is functional equivalent to disabling recursive scanning in
the install builder.  I already considered this and think its the best
choice.  See some notes here about our previous conversation:
https://bitbucket.org/scons/scons/pull-requests/237/issue-2264-cross-language-scanner-support/diff#comment-8521558

With that change only 3 tests are left failing for me.  See copa-pasta
snippet below:

test/IDL/IDLSUFFIXES.py # If I understand the test correctly, this
test checks to see that the bug we are trying to fix works???
test/explain/basic.py # Contains code with build dependencies on
installed files as sources: is this something SCons explicitly
supports?
test/explain/save-info.py # Contains code with build dependencies on
installed files as sources: is this something SCons explicitly
supports?

Please pay special attention to test/IDL/IDLSIFFIXES.py because this test
requires the recursive scanner behavior to work which leads me to believe
that the recursive scanner problem already existed, and there used to be a
hack for this with the CScanner...


>
> I lean toward 4) logic given I how I understand the CCopy builder in Parts
> works. and that the install builder for the most part is a copy builder
> that adds nodes to a list to be used by the Scons version of the packaging
> builders.
>

Agreed. See above.


>
> Jason
>
>
>  *From:* William Blevins <wblevins001 at gmail.com>
> *Sent:* Wednesday, July 29, 2015 7:19 PM
> *To:* SCons developer list <scons-dev at scons.org>
> *Subject:* Re: [Scons-dev] Cross-language support
>
>   Jason.
>
> FYI.  The failing tests you listed match the list from June. The
> action-test.py may be new.
>
>
> https://bitbucket.org/scons/scons/pull-requests/237/issue-2264-cross-language-scanner-support/diff#comment-7323143
>
> V/R,
> William
>
> On Wed, Jul 29, 2015 at 8:16 PM, William Blevins <wblevins001 at gmail.com>
> wrote:
>
>>  Jason,
>>
>> As previously stated in the pull request comments, I expect some tests to
>> fail currently because of the recursive header install issue (topic #1):
>> <lang>SUFFIXES, HeaderInstall.py.  The other tests may or may not be
>> related.
>>
>> There is more value in looking at those tests first.  It may also be
>> valuable to go back 3-4 commits to the closest ancestor of the default
>> branch and run tests there, so that we can see if some of the tests failed
>> previously.  Also, if we think its a problem of the current location of
>> that divergent head, then we can see if a rebase fixes some of them.
>>
>> V/R,
>> William
>>
>> On Wed, Jul 29, 2015 at 7:09 PM, Jason Kenny <dragon512 at live.com> wrote:
>>
>>>   ok not any better...
>>>
>>>
>>> Failed the following 16 tests:
>>>         test\Batch\action-changed.py
>>>         test\CPPSUFFIXES.py
>>>         test\DSUFFIXES.py
>>>         test\Fortran\FORTRANSUFFIXES.py
>>>         test\HeaderInstall.py
>>>          test\MSVS\vs-9.0-exec.py
>>>         test\Win32\mingw.py
>>>         test\exitfns.py
>>>         test\implicit\IMPLICIT_COMMAND_DEPENDENCIES.py
>>>         test\import.py
>>>         test\long-lines\signature.py
>>>         test\scons-time\run\config\python.py
>>>         test\scons-time\run\option\python.py
>>>         test\sconsign\script\SConsignFile.py
>>>         test\sconsign\script\Signatures.py
>>>         test\sconsign\script\no-SConsignFile.py
>>>
>>> I would need to dig down to see what is failing and why
>>> Jason
>>>
>>>
>>>  *From:* Jason Kenny <dragon512 at live.com>
>>> *Sent:* Wednesday, July 29, 2015 5:38 PM
>>>  *To:* SCons developer list <scons-dev at scons.org>
>>> *Subject:* Re: [Scons-dev] Cross-language support
>>>
>>>    ok I have it fixed... I am rerunning the tests now
>>> Jason
>>>
>>>  *From:* Jason Kenny <dragon512 at live.com>
>>> *Sent:* Wednesday, July 29, 2015 5:30 PM
>>> *To:* SCons developer list <scons-dev at scons.org>
>>> *Subject:* Re: [Scons-dev] Cross-language support
>>>
>>>   I must have messed up. I only see debugCount and JniHeaderDir
>>> bookmarks at the moment.
>>>
>>> Jason
>>>
>>>  *From:* William Blevins <wblevins001 at gmail.com>
>>> *Sent:* Wednesday, July 29, 2015 4:50 PM
>>> *To:* SCons developer list <scons-dev at scons.org>
>>> *Subject:* Re: [Scons-dev] Cross-language support
>>>
>>>  That's a SCons fork.  Not an HG branch (in case its a terminology
>>> problem).  Make sure you are using the CrossLanguage bookmark.
>>>
>>> V/R,
>>> William
>>>
>>> On Wed, Jul 29, 2015 at 5:45 PM, Jason Kenny <dragon512 at live.com> wrote:
>>>
>>>>   branch *SCons_20150323 *
>>>>
>>>>  I can will setup from scratch again and try again.
>>>>
>>>> Jason
>>>>
>>>>  *From:* William Blevins <wblevins001 at gmail.com>
>>>> *Sent:* Wednesday, July 29, 2015 4:38 PM
>>>>  *To:* SCons developer list <scons-dev at scons.org>
>>>> *Subject:* Re: [Scons-dev] Cross-language support
>>>>
>>>>   Jason,
>>>>
>>>> Are you sure you are using the correct commit revision?
>>>>
>>>> None of those tests are failing on my Linux distro, and I haven't
>>>> modified any of those tests either.  Plus, tests that I am expecting to
>>>> fail are apparently passing.
>>>>
>>>> V/R,
>>>> William
>>>>
>>>> On Wed, Jul 29, 2015 at 5:06 PM, Jason Kenny <dragon512 at live.com>
>>>> wrote:
>>>>
>>>>>    So a quick pass of the tests has these failures
>>>>>
>>>>> Failed the following 15 tests:
>>>>>         test\Batch\action-changed.py
>>>>>         test\MSVS\vs-9.0-exec.py
>>>>>         test\Win32\mingw.py
>>>>>         test\exitfns.py
>>>>>         test\implicit\IMPLICIT_COMMAND_DEPENDENCIES.py
>>>>>         test\import.py
>>>>>         test\long-lines\signature.py
>>>>>         test\option\debug-count.py
>>>>>         test\option\debug-multiple.py
>>>>>         test\option\debug-objects.py
>>>>>         test\scons-time\run\config\python.py
>>>>>         test\scons-time\run\option\python.py
>>>>>         test\sconsign\script\SConsignFile.py
>>>>>         test\sconsign\script\Signatures.py
>>>>>         test\sconsign\script\no-SConsignFile.py
>>>>>
>>>>>  *From:* Jason Kenny <dragon512 at live.com>
>>>>> *Sent:* Wednesday, July 29, 2015 2:32 PM
>>>>>  *To:* SCons developer list <scons-dev at scons.org>
>>>>> *Subject:* Re: [Scons-dev] Cross-language support
>>>>>
>>>>>    Ok let me update the branch and re-run everything.
>>>>>
>>>>> Jason
>>>>>
>>>>>  *From:* William Blevins <wblevins001 at gmail.com>
>>>>> *Sent:* Wednesday, July 29, 2015 10:30 AM
>>>>> *To:* SCons developer list <scons-dev at scons.org>
>>>>> *Subject:* Re: [Scons-dev] Cross-language support
>>>>>
>>>>>  Jason,
>>>>>
>>>>> I'm not sure.  I remember that you were helping me look into the
>>>>> recursive Install behavior.  Plus, possible parts incompatibility?
>>>>>
>>>>> I haven't ran the tests on a windows box at all.  I don't expect
>>>>> anything new, but it should be done before it ships :)
>>>>>
>>>>> V/R,
>>>>> William
>>>>>
>>>>> On Wed, Jul 29, 2015 at 8:44 AM, Jason Kenny <dragon512 at live.com>
>>>>> wrote:
>>>>>
>>>>>>   Which MS toolchain on windows do we need tested? I can do a test
>>>>>> run this afternoon.
>>>>>>
>>>>>> Jason
>>>>>>
>>>>>>  *From:* William Blevins <wblevins001 at gmail.com>
>>>>>> *Sent:* Wednesday, July 29, 2015 1:11 AM
>>>>>> *To:* SCons developer list <scons-dev at scons.org>
>>>>>>  *Subject:* Re: [Scons-dev] Cross-language support
>>>>>>
>>>>>>    Happy with it*
>>>>>>
>>>>>> Until the review is complete, and I can perhaps get a few more guinea
>>>>>> pigs to try it out, it's hard to make a concrete projection. I have 3 items
>>>>>> listed out that need to be given a thumbs up/down at a minimum:
>>>>>>
>>>>>> * Item #1 is still mostly outstanding.  I'm not sure how to address,
>>>>>> please see the discussion Jason and I started a few weeks ago under the
>>>>>> pull request comments.
>>>>>>
>>>>>> * Item #2 requires no changes (to my knowledge), but someone with
>>>>>> more QT knowledge may say otherwise.
>>>>>>
>>>>>> * I may already have most of the patch for Item #3 (IE. no scanner
>>>>>> for key) based on today's feedback.
>>>>>>
>>>>>> I also still need to request test runs on Windows, and for toolchains
>>>>>> that I may not have installed or instructions to install them (EG. D).
>>>>>>
>>>>>> V/R,
>>>>>> William
>>>>>>
>>>>>> On Wed, Jul 29, 2015 at 1:58 AM, William Blevins <
>>>>>> wblevins001 at gmail.com> wrote:
>>>>>>
>>>>>>> As soon as we are happen with it :)
>>>>>>>
>>>>>>> On Wed, Jul 29, 2015 at 12:30 AM, Bill Deegan <
>>>>>>> bill at baddogconsulting.com> wrote:
>>>>>>>
>>>>>>>>   O.k. let me push 2.3.5 with the visual studio 2015 stuff.
>>>>>>>> Then we'll changed to 2.4 merge slots. stabilize. release.
>>>>>>>> Then this code? (2.5?)
>>>>>>>>
>>>>>>>> -Bill
>>>>>>>>
>>>>>>>> On Tue, Jul 28, 2015 at 6:33 PM, Jason Kenny <dragon512 at live.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>   I have been using the slots drop for a while with Parts. I
>>>>>>>>> think it is ready. It does have a notable improvement in speed and memory
>>>>>>>>> size. I would before getting this out as officially earlier than later.
>>>>>>>>>
>>>>>>>>> Jason
>>>>>>>>>
>>>>>>>>>  *From:* William Blevins <wblevins001 at gmail.com>
>>>>>>>>> *Sent:* Tuesday, July 28, 2015 8:31 PM
>>>>>>>>> *To:* Dirk Bächle <dl9obn at darc.de> ; SCons developer list
>>>>>>>>> <scons-dev at scons.org>
>>>>>>>>> *Subject:* Re: [Scons-dev] Cross-language support
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  On Tue, Jul 28, 2015 at 7:16 PM, Dirk Bächle <tshortik at gmx.de>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi guys,
>>>>>>>>>>
>>>>>>>>>> sorry for chiming in so late.
>>>>>>>>>>
>>>>>>>>>> On 28.07.2015 23:44, Gary Oberbrunner wrote:
>>>>>>>>>>
>>>>>>>>>>> Yes, that's how we've done it in the past.  Sounds like doing it
>>>>>>>>>>> at the same time as slots would be perfect.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> Doing this in parallel with the "slots" change sounds good to me
>>>>>>>>>> too. +1
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'm not opposed to releasing in the same update as slots, but
>>>>>>>>> since the cross language code reviews haven't been finished, I don't want
>>>>>>>>> to delay slots since it is ready now.
>>>>>>>>>
>>>>>>>>> The pessimist in me as sees doing two major enhancements in the
>>>>>>>>> same release as higher risk; I don't foresee any issues, but its worth the
>>>>>>>>> thought if the release overhead isn't too bad.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -- Gary
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jul 28, 2015 at 2:01 PM, Bill Deegan <
>>>>>>>>>>> bill at baddogconsulting.com <mailto:bill at baddogconsulting.com>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>     Gary,
>>>>>>>>>>>
>>>>>>>>>>>     For such a change we should bump the second digit?
>>>>>>>>>>>     2.4?
>>>>>>>>>>>
>>>>>>>>>>>     I agree we should not turn down a change because it will
>>>>>>>>>>> cause rebuilds where the past didn't as long as it is now more correct
>>>>>>>>>>>     (which it should be with this change).
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Yes, "forward" is the way to go. ;)
>>>>>>>>>>
>>>>>>>>>>     Also agree we should be verbose in our notification of the
>>>>>>>>>>> impacts of the new change to avoid (as much as we can) "surprises".
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> I think (better: hope) we did a good enough job for the "slots"
>>>>>>>>>> stuff on this. For the scanner changes, I see them more like a fix...so a
>>>>>>>>>> single announcement should be sufficient?
>>>>>>>>>>
>>>>>>>>>> Finally, and just in case I haven't done so already, I'd like to
>>>>>>>>>> thank William for all the work he's done on this issue. I couldn't help as
>>>>>>>>>> much as I would've liked, but with Gary's support you tackled this down and
>>>>>>>>>> brought it to a good end. Kudos to you...bravo!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Truly appreciated.  I have spent a lot of time on this issue
>>>>>>>>> despite what one might expect from the number of lines of code.
>>>>>>>>>
>>>>>>>>> It was honestly my first time working in a "real" python
>>>>>>>>> environment outside of scripting, and the SCons code base is rather
>>>>>>>>> complex.  It instills me greater appreciation for the work that has already
>>>>>>>>> done.
>>>>>>>>>
>>>>>>>>> I also want to thank everyone here for their help past and future,
>>>>>>>>> but don't pat me on the back until its done. I might get lazy :)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>>
>>>>>>>>>> Dirk
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Scons-dev mailing list
>>>>>>>>>> Scons-dev at scons.org
>>>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------
>>>>>>>>> _______________________________________________
>>>>>>>>> Scons-dev mailing list
>>>>>>>>> Scons-dev at scons.org
>>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Scons-dev mailing list
>>>>>>>>> Scons-dev at scons.org
>>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Scons-dev mailing list
>>>>>>>> Scons-dev at scons.org
>>>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>> _______________________________________________
>>>>>> Scons-dev mailing list
>>>>>> Scons-dev at scons.org
>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>
>>>>>> _______________________________________________
>>>>>> Scons-dev mailing list
>>>>>> Scons-dev at scons.org
>>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>>
>>>>>>
>>>>> ------------------------------
>>>>> _______________________________________________
>>>>> Scons-dev mailing list
>>>>> Scons-dev at scons.org
>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>> ------------------------------
>>>>> _______________________________________________
>>>>> Scons-dev mailing list
>>>>> Scons-dev at scons.org
>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>
>>>>> _______________________________________________
>>>>> Scons-dev mailing list
>>>>> Scons-dev at scons.org
>>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>>
>>>>>
>>>> ------------------------------
>>>> _______________________________________________
>>>> Scons-dev mailing list
>>>> Scons-dev at scons.org
>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>
>>>> _______________________________________________
>>>> Scons-dev mailing list
>>>> Scons-dev at scons.org
>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>
>>>>
>>> ------------------------------
>>> _______________________________________________
>>> Scons-dev mailing list
>>> Scons-dev at scons.org
>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>> ------------------------------
>>> _______________________________________________
>>> Scons-dev mailing list
>>> Scons-dev at scons.org
>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>> ------------------------------
>>> _______________________________________________
>>> Scons-dev mailing list
>>> Scons-dev at scons.org
>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>
>>> _______________________________________________
>>> Scons-dev mailing list
>>> Scons-dev at scons.org
>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>
>>>
>>
>
>
> ------------------------------
> _______________________________________________
> Scons-dev mailing list
> Scons-dev at scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
> _______________________________________________
> Scons-dev mailing list
> Scons-dev at scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20150729/571287f2/attachment-0001.html>

More information about the Scons-dev mailing list