[Scons-dev] scons-2.5.0 performance issue?

[Bill Deegan]

Thomas,

Can you run --debug=objects on the other versions as well?
Just want to see if the object counts have change significantly between
2.5.0 and the others.



Thanks,
Bill

On Mon, Apr 11, 2016 at 10:59 AM, Thomas Berg <merlin66b at gmail.com> wrote:

> Bill, below is the output of --debug=count, it is identical with
> scons-2.4.1 and scons-2.5.0.
>
> Since my case was about the no-op build (nothing is built), less
> parallelization should not be an issue. We do have about 900 Qt Moc
> generated files in our build, and I guess 1-2000 other generated
> files. When moving from 2.4.1 to 2.5.0, the I can see that the moc
> build steps are being re-run, nothing else. Still I find it mysterious
> if just a small percentage increase in the number of dependencies can
> triple the scons processing time.
>
> If others are not seeing these slowdowns, maybe our build is getting
> hit bad due to the form of our dependency graph or something.
>
> Thanks,
> Thomas
>
>
> Object counts:
>        pre-   post-    pre-   post-
>        read    read   build   build   Class
>           0    3277    3277   35628   Action.CommandAction
>           0     220     220     220   Action.CommandGeneratorAction
>           0     894     894     894   Action.FunctionAction
>           0    1670    1670    1670   Action.LazyAction
>           0       9       9      35   Action.ListAction
>           0    1563    1563    1563   Builder.BuilderBase
>           0     220     220     220   Builder.CompositeBuilder
>           0   28632   28632   28632   Builder.OverrideWarner
>           0      44      44      44   Environment.Base
>           0    1142    1142    1142   Environment.EnvironmentClone
>           0      16      16      16   Environment.OverrideEnvironment
>           0   40815   40815   42115   Executor.Executor
>           0       0       0   47872   Executor.Null
>           0   98922   98922  107638   Node.FS.Base
>           0    5285    5285    6580   Node.FS.Dir
>           0   44079   44079   51437   Node.FS.File
>           0       0       0       1   Node.FS.RootDir
>           0   99703   99703  108420   Node.Node
>
> On Mon, Apr 11, 2016 at 3:47 PM, Bill Deegan <bill at baddogconsulting.com>
> wrote:
> > Thomas,
> >
> > Can you run and post with "--debug=count"?
> >
> > Thanks,
> > Bill
> >
> > On Mon, Apr 11, 2016 at 7:14 AM, Thomas Berg <merlin66b at gmail.com>
> wrote:
> >>
> >> Hi all,
> >>
> >> First of all, thanks for the 2.5.0 release!
> >>
> >> I have tested it in the build at my dayjob now, where we have a large
> >> C++ application built with scons: around 7-8000 .cpp files, same
> >> amount of headers, some code generation steps including Qt moc-ing.
> >> Unfortunately it seems that something has happened in scons-2.5.0 that
> >> slows down the no-op build quite significantly. This is the output of
> >> scons --debug=time in an almost-no-op build of our source tree:
> >>
> >> ========================================================
> >> = 2.5.0
> >> ========================================================
> >>
> >> Total build time: 933.582000 seconds
> >> Total SConscript file execution time: 55.395000 seconds
> >> Total SCons execution time: 858.109000 seconds
> >> Total command execution time: 20.078000 seconds
> >>
> >> ========================================================
> >> = 2.4.1
> >> ========================================================
> >>
> >> Total build time: 337.473000 seconds
> >> Total SConscript file execution time: 62.303000 seconds
> >> Total SCons execution time: 256.638000 seconds
> >> Total command execution time: 18.532000 seconds
> >>
> >> ========================================================
> >> = 2.3.4
> >> ========================================================
> >>
> >> Total build time: 331.774000 seconds
> >> Total SConscript file execution time: 61.290000 seconds
> >> Total SCons execution time: 249.606000 seconds
> >> Total command execution time: 20.878000 seconds
> >>
> >> ------------------------------------------------------------
> >> This is with python-2.7.9 64bit on Windows 7.
> >>
> >> So it seems that the scons execution time has increased by a factor of
> >> 3 in our build, between scons-2.4.1 and 2.5.0 :(
> >>
> >> For us the 5.5 minute no-op build time has already been an issue, so
> >> the new 15 minute no-op build is bad news. If you have just changed a
> >> single file, and need to build from the top to check that everything
> >> is up to date, 15 minutes is a very long time... I'm seeing similar
> >> slowdowns for our partial builds where we only execute some of our
> >> SConscripts (build time going from 16 to 25 seconds for example).
> >>
> >> 1. Is anyone else seeing this?
> >> 2. Any idea what changes that could cause something like this? I could
> >> try bisecting or reverting some changes to narrow it down
> >> 3. Is there anything else I can do, like posting the output of
> profiling?
> >>
> >> I also have some other smaller hobby C++ projects that I could profile.
> >>
> >> Thanks for any hints,
> >> Thomas Berg
> >> _______________________________________________
> >> Scons-dev mailing list
> >> Scons-dev at scons.org
> >> https://pairlist2.pair.net/mailman/listinfo/scons-dev
> >
> >
> >
> > _______________________________________________
> > Scons-dev mailing list
> > Scons-dev at scons.org
> > https://pairlist2.pair.net/mailman/listinfo/scons-dev
> >
> _______________________________________________
> Scons-dev mailing list
> Scons-dev at scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20160411/806d8979/attachment-0001.html>