[Scons-dev] scons-2.5.0 performance issue?

Mon Apr 11 11:40:53 EDT 2016

Thomas,

Even on no-op, the difference in dependency generation could be
substantial. Consider comparing '--tree=prune' between a small QT target
between 2.5.0 and 2.4.1 and I think you will see that they are quite
different.

V/R,
William

On Mon, Apr 11, 2016 at 3:59 PM, Thomas Berg <merlin66b at gmail.com> wrote:

> Bill, below is the output of --debug=count, it is identical with
> scons-2.4.1 and scons-2.5.0.
>
> Since my case was about the no-op build (nothing is built), less
> parallelization should not be an issue. We do have about 900 Qt Moc
> generated files in our build, and I guess 1-2000 other generated
> files. When moving from 2.4.1 to 2.5.0, the I can see that the moc
> build steps are being re-run, nothing else. Still I find it mysterious
> if just a small percentage increase in the number of dependencies can
> triple the scons processing time.
>
> If others are not seeing these slowdowns, maybe our build is getting
> hit bad due to the form of our dependency graph or something.
>
> Thanks,
> Thomas
>
>
> Object counts:
>        pre-   post-    pre-   post-
>        read    read   build   build   Class
>           0    3277    3277   35628   Action.CommandAction
>           0     220     220     220   Action.CommandGeneratorAction
>           0     894     894     894   Action.FunctionAction
>           0    1670    1670    1670   Action.LazyAction
>           0       9       9      35   Action.ListAction
>           0    1563    1563    1563   Builder.BuilderBase
>           0     220     220     220   Builder.CompositeBuilder
>           0   28632   28632   28632   Builder.OverrideWarner
>           0      44      44      44   Environment.Base
>           0    1142    1142    1142   Environment.EnvironmentClone
>           0      16      16      16   Environment.OverrideEnvironment
>           0   40815   40815   42115   Executor.Executor
>           0       0       0   47872   Executor.Null
>           0   98922   98922  107638   Node.FS.Base
>           0    5285    5285    6580   Node.FS.Dir
>           0   44079   44079   51437   Node.FS.File
>           0       0       0       1   Node.FS.RootDir
>           0   99703   99703  108420   Node.Node
>
> On Mon, Apr 11, 2016 at 3:47 PM, Bill Deegan <bill at baddogconsulting.com>
> wrote:
> > Thomas,
> >
> > Can you run and post with "--debug=count"?
> >
> > Thanks,
> > Bill
> >
> > On Mon, Apr 11, 2016 at 7:14 AM, Thomas Berg <merlin66b at gmail.com>
> wrote:
> >>
> >> Hi all,
> >>
> >> First of all, thanks for the 2.5.0 release!
> >>
> >> I have tested it in the build at my dayjob now, where we have a large
> >> C++ application built with scons: around 7-8000 .cpp files, same
> >> amount of headers, some code generation steps including Qt moc-ing.
> >> Unfortunately it seems that something has happened in scons-2.5.0 that
> >> slows down the no-op build quite significantly. This is the output of
> >> scons --debug=time in an almost-no-op build of our source tree:
> >>
> >> ========================================================
> >> = 2.5.0
> >> ========================================================
> >>
> >> Total build time: 933.582000 seconds
> >> Total SConscript file execution time: 55.395000 seconds
> >> Total SCons execution time: 858.109000 seconds
> >> Total command execution time: 20.078000 seconds
> >>
> >> ========================================================
> >> = 2.4.1
> >> ========================================================
> >>
> >> Total build time: 337.473000 seconds
> >> Total SConscript file execution time: 62.303000 seconds
> >> Total SCons execution time: 256.638000 seconds
> >> Total command execution time: 18.532000 seconds
> >>
> >> ========================================================
> >> = 2.3.4
> >> ========================================================
> >>
> >> Total build time: 331.774000 seconds
> >> Total SConscript file execution time: 61.290000 seconds
> >> Total SCons execution time: 249.606000 seconds
> >> Total command execution time: 20.878000 seconds
> >>
> >> ------------------------------------------------------------
> >> This is with python-2.7.9 64bit on Windows 7.
> >>
> >> So it seems that the scons execution time has increased by a factor of
> >> 3 in our build, between scons-2.4.1 and 2.5.0 :(
> >>
> >> For us the 5.5 minute no-op build time has already been an issue, so
> >> the new 15 minute no-op build is bad news. If you have just changed a
> >> single file, and need to build from the top to check that everything
> >> is up to date, 15 minutes is a very long time... I'm seeing similar
> >> slowdowns for our partial builds where we only execute some of our
> >> SConscripts (build time going from 16 to 25 seconds for example).
> >>
> >> 1. Is anyone else seeing this?
> >> 2. Any idea what changes that could cause something like this? I could
> >> try bisecting or reverting some changes to narrow it down
> >> 3. Is there anything else I can do, like posting the output of
> profiling?
> >>
> >> I also have some other smaller hobby C++ projects that I could profile.
> >>
> >> Thanks for any hints,
> >> Thomas Berg
> >> _______________________________________________
> >> Scons-dev mailing list
> >> Scons-dev at scons.org
> >> https://pairlist2.pair.net/mailman/listinfo/scons-dev
> >
> >
> >
> > _______________________________________________
> > Scons-dev mailing list
> > Scons-dev at scons.org
> > https://pairlist2.pair.net/mailman/listinfo/scons-dev
> >
> _______________________________________________
> Scons-dev mailing list
> Scons-dev at scons.org
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist2.pair.net/pipermail/scons-dev/attachments/20160411/dafa6f3b/attachment.html>