[Scons-dev] debug explain, part 1

[Mats Wichmann]

I've been misspending my time looking at a problem which maybe isn't
that important, but it's bugging me since I started noticing it in
project builds a number of months ago, when I added --debug=explain to
try to get help for something which was not behaving as expected.
Thought maybe some discussion here might help.

This is part 1 of 2.

There's an explain method that is called if --debug=explain is
requested, and if scons already knows a file needs to be rebuilt.  That
is, the decider has already made a decision, not affected by explain,
which only tries to prepare a nicely readable description if you are
trying to debug. By the way, a comment on IRC recently noted that maybe
that's a little silly, you've already done some work to find a reason
once, and then you ask another method to sort of recreate that. In any
case, explain sometimes ends up not giving very useful explanations.

The most notable problem is in some circumstances, the source file to be
rebuilt is reported with a pair of lines, as being no longer a
dependency, and then as a new dependency. This happens even if the
source file itself did not change.  There are some other problems: if
there were several reasons for a rebuild, like a file changed and also
the dependency order and/or the build action changed, only the first of
those is reported.  Not sure whether that's important or not.  And there
is another scenario which will show in the example below that we can
discuss whether it is right or not.

After fiddling with tests on a build which is big/slow I've made up a
simple example (can be the basis of a test case) - a trivial C source
file sits in the src/ directory, along with a header file it includes;
SConstruct calls to src/SConscript which builds a Program from the
source file.  Here are the explain lines only from building this three
ways - the setups are identical except for the single-line SConstruct:

=== No variant dir:
+ SConscript('src/SConscript')
+ Initial build:
scons: building `src/hello.o' because it doesn't exist
scons: building `src/hello' because it doesn't exist
+ src/hello.c updated:
scons: rebuilding `src/hello.o' because `src/hello.c' changed
+ src/hello.h updated:
scons: rebuilding `src/hello.o' because `src/hello.h' changed

=== Variant dir build/ with duplication:
+ SConscript('src/SConscript', variant_dir='build', duplicate=1)
+ Initial build:
scons: building `build/hello.o' because it doesn't exist
scons: building `build/hello' because it doesn't exist
+ src/hello.c updated:
scons: rebuilding `build/hello.o' because `build/hello.c' changed
+ src/hello.h updated:
scons: rebuilding `build/hello.o' because `build/hello.h' changed

=== Variant dir build/ without duplication:
+ SConscript('src/SConscript', variant_dir='build', duplicate=0)
+ Initial build:
scons: building `build/hello.o' because it doesn't exist
scons: building `build/hello' because it doesn't exist
+ src/hello.c updated:
scons: rebuilding `build/hello.o' because:
           `src/hello.c' is no longer a dependency
           `src/hello.c' is a new dependency
+ src/hello.h updated:
scons: rebuilding `build/hello.o' because:
           `src/hello.c' is no longer a dependency
           `src/hello.c' is a new dependency
           `src/hello.h' changed

The first case looks as I'd expect.  The two variant cases ought to look
almost the same, except that built objects ought to be showing in the
variant dir.

The variant form without duplication shows the problem I started with.
If the source file itself changes, it is not reported as changed, but is
reported as removec + new dependency.  If the header file changed, it is
correctly reported as such, and the source file still is reported as
removed + new.

Meanwhile the variant form with duplication shows a different oddity:
even though the changes were to a file in src, it reports that the
rebuild is because of a change to a file in build.  That's technically
correct but violates the principle of "no surprises" in my opinion - if
you edited in src, it should say so, no? The other two cases do.

Note that the program itself is not relinked after the object is built -
the "changes" in the experiment made no functional changes, just updated
a comment, so I guess the csig of hello.o didn't change after rebuild.
I think that's as expected, just wanted to mention.

There are at least three github issues that include this removed/new
dependency pattern, and it comes up elsewhere on occasion.   So it seems
a number of people are running into it.

part 2 will have some of the internal details.