[Scons-dev] SCons and Java

Fri May 30 19:39:28 EDT 2014

Dirk,

Team,
> I have a couple points I'd like to discuss, but for the sake of
> organization, I intend to split them into separate emails.
> Java Part #2
> There has been some discussion on making Scons more ANT-like,
> http://www.scons.org/wiki/JavaSupport, and that might solve some issues
> like identifying Java targets.  There is merit here assuming that Java
> support stays SCons-like at the same time; otherwise, why not just use ant
> or maven?
>

>From this I get that you'd like to have some form of Java support in SCons
>> directly, and don't want to delegate things to an external program,
>> correct? At least it's my assumption for the following remarks...
>
> SCons already has Java support (right?).  I have been a SCons user for a
few years and am interested in trying to make the Java toolkit better
because I dislike Ant-like build system(s), but I feel that SCons still has
room for improvement.  I am braining storming ways to improve the setup.  I
have a bunch of idea(s) that I would like to discuss and this just happens
to be one of them (#2 in fact).  I know that the SCons teams has talked
about refactoring the SCons Java toolkit, but outside the notes from the
link above, I haven't seen any further discussions.  I am the new kid on
the scons-dev mailing list block, so feel free to give me a tour.  I would
be willing to work on SCons with some direction.

Problem(s) this will solve:
> 1. Determining the source -> target(s) from *.java -> *.class is painful,
> possibly java version dependent, and performance intensive (possibly);
> thus, we should optimize this out.
> 2. Java developers are plagued requiring manual cleaning of a jar far too
> often with ant-like system(s) and the current scon(s) methodology.
>    1. Clean could be required when deleting or moving a file since the
> last build output isn't removed otherwise; avoidable in current SCons
> depending on how sources are specified (not obvious and/or intuitive coming
> from ant-like system(s)).
> 3. Simplify the SCons Java API to allow building jar file(s) without
> having to interact with *.class file(s) at all.
> 4. Simplify the SCons Java support for resource file(s); currently, you
> have to do copying by hand (if it's supported I know not where or how).
> 5. Reduce the amount of code required to build a jar (by a lot).
>

This sounds like an even more simplified Java Builder than what we have
>> now. If you could come up with a SoSimpleJava() Builder, we'd probably add
>> it to the core and let people have their own choice.
>
> That's fair, but since I am talking about not tracking *.class file(s) at
all.  The old function(s) couldn't exist.

 How I imagine this should work:

1. Treat class file(s) like side-effect(s) and do not include them (at all)
> in the dependency tree.
>     1. This means we don't have to scan the Java file(s) and determine
> *.class targets.
>

I don't think that you can have both. If you treat files as SideEffects (in
>> the SCons sense), they have to be part of the dependency DAG. See next
>> comment...
>
> I meant this in the truer form of side-effect and not the
SCons.SideEffects sense.  Getting from A -> C by passing through B, but B
is just along the way.  The only way the *.class file(s) could be part of
the DAG is if they were identified after the Java compiler had already
created the *.class file(s) by basically Glob(*.class).

I would really like to get away from SCons being required to scan Java
source file(s) to determine the targets due to the 1 -> N nature of *.java
to *.class.  This would resolve several issues with the SCons Java toolkit
including incorrect dependency tree(s) with derived (generated) Java source
file(s) and some build issue(s) that require manual clean operation(s) on
Java code (which may be alleviated depending on how you setup your
SConscript, but may not be obvious coming from an ant-like system).

The Java DAG is already incomplete because SCons doesn't add item(s) in
JAVACLASSPATH automatically, so user(s) are required to do it manually
anyway.  At first I thought this was a bug, but it's even documented in the
User's Guide, so I assume it's intentional which confuses the crap out of
me?

2. Remove the Java(...) function from SCons; thus, do not interact with
> *.class file(s) directly ever.  There isn't a *good* reason to do this.
>

Here is just one scenario, where dealing with *.class files actually makes
>> sense: If you're using anonymous classes in a Java source, a bunch of
>> additional class files get generated. How do you remove these files on a
>> "clean", if they are located in a source folder (which people might prefer
>> for some reason)?
>
> Remove all *.class file(s) generated within the directory tree were the
Java compiler is expected to output it's source.  I'd almost prefer to
require the build directory and just delete it.  Making the build
configuration simple and clean is more important I think.

More experienced Java developers can probably come up with a dozen more
>> useful cases...I'm not sure how often people would want to integrate
>> pre-built *.class files (coming from Scala perhaps?) into their builds, for
>> example.
>
> Are you talking about coping *.class file(s) between build directories?
 Why not use a *.jar file? At that point you are just moving code
copy-pasta  from your source code to your build tool.

You may be able to work around the "clean" problem, by creating all
>> "*.class" files in a separate folder structure (pretty much like a
>> VariantDir). But what you'll end up with is a somewhat crippled DAG,
>> forcing you to remove this output dir completely and rebuild everything
>> from scratch...even when only a comment in a single Java source is changed.
>> Only to "be sure", just like in make or Ant...there is not much of a win
>> here for the user, I'm afraid.
>
> The win here is that Java compiler is much faster than the SCons source
emitter (I think that's the right term) for Java. There is a performance
bottleneck(s) in the Java toolkit which I will discuss in another email; I
hope the team is alright with email attachments because I have performance
logs.

Users might also accidentally delete "*.class" files in your output dir,
>> between two builds. And they expect SCons to rebuild those intermediate
>> targets on its next invocation. How do you want to handle this?
>
> SCons wouldn't rebuild since no source had changed.  This would be an
issue if the user was trying to share *.class file(s) which would only be
relevant with something like SCons.JavaH, but it supports the directory
parameter(s). SCons could check that the directory contents had changed and
mark as requiring rebuild; I am grabbing at straws on this one...

Finally, you should take into account that users might want to create
>> different versions of the same source code. Just as an example, let's
>> assume they want to create an "original" and "obfuscated" target(s), or a
>> "simple" and "extended" release where additional packages get included
>> based on environment variables. And for maximum speedup they do this in
>> parallel, by using the "-j" option...so a single dedicated output dir won't
>> do probably.
>
> There would be 1 target directory per Java compiler invocation which I
imagined would be 1 target directory per *.jar created.  The gain from
threading packages in a *.jar file shouldn't be observable until well into
1000+ *.class file(s) per *.jar file.  At this point, I imagine there are
bigger architecture concerns at hand than the compile time of 1 *.jar file.

V/R,
William

On Fri, May 30, 2014 at 7:13 AM, Dirk Bächle <tshortik at gmx.de> wrote:

>  Hi William,
>
> please find a few comments below.
>
>
> On 30.05.2014 02:42, William Blevins wrote:
>
> Team,
>
>  I have a couple points I'd like to discuss, but for the sake of
> organization, I intend to split them into separate emails.
>
>  Java Part #2
>
>  There has been some discussion on making Scons more ANT-like,
> http://www.scons.org/wiki/JavaSupport, and that might solve some issues
> like identifying Java targets.  There is merit here assuming that Java
> support stays SCons-like at the same time; otherwise, why not just use ant
> or maven?
>
>   From this I get that you'd like to have some form of Java support in
> SCons directly, and don't want to delegate things to an external program,
> correct? At least it's my assumption for the following remarks...
>
>
>  Problem(s) this will solve:
> 1. Determining the source -> target(s) from *.java -> *.class is painful,
> possibly java version dependent, and performance intensive (possibly);
> thus, we should optimize this out.
> 2. Java developers are plagued requiring manual cleaning of a jar far too
> often with ant-like system(s) and the current scon(s) methodology.
>     1. Clean could be required when deleting or moving a file since the
> last build output isn't removed otherwise; avoidable in current SCons
> depending on how sources are specified (not obvious and/or intuitive coming
> from ant-like system(s)).
> 3. Simplify the SCons Java API to allow building jar file(s) without
> having to interact with *.class file(s) at all.
>  4. Simplify the SCons Java support for resource file(s); currently, you
> have to do copying by hand (if it's supported I know not where or how).
> 5. Reduce the amount of code required to build a jar (by a lot).
>
>   This sounds like an even more simplified Java Builder than what we have
> now. If you could come up with a SoSimpleJava() Builder, we'd probably add
> it to the core and let people have their own choice.
>
>  How I imagine this should work:
> 1. Treat class file(s) like side-effect(s) and do not include them (at
> all) in the dependency tree.
>     1. This means we don't have to scan the Java file(s) and determine
> *.class targets.
>
> I don't think that you can have both. If you treat files as SideEffects
> (in the SCons sense), they have to be part of the dependency DAG. See next
> comment...
>
>
>  2. Remove the Java(...) function from SCons; thus, do not interact with
> *.class file(s) directly ever.  There isn't a *good* reason to do this.
>
> Here is just one scenario, where dealing with *.class files actually makes
> sense: If you're using anonymous classes in a Java source, a bunch of
> additional class files get generated. How do you remove these files on a
> "clean", if they are located in a source folder (which people might prefer
> for some reason)?
>
> More experienced Java developers can probably come up with a dozen more
> useful cases...I'm not sure how often people would want to integrate
> pre-built *.class files (coming from Scala perhaps?) into their builds, for
> example.
>
> You may be able to work around the "clean" problem, by creating all
> "*.class" files in a separate folder structure (pretty much like a
> VariantDir). But what you'll end up with is a somewhat crippled DAG,
> forcing you to remove this output dir completely and rebuild everything
> from scratch...even when only a comment in a single Java source is changed.
> Only to "be sure", just like in make or Ant...there is not much of a win
> here for the user, I'm afraid.
>
> Users might also accidentally delete "*.class" files in your output dir,
> between two builds. And they expect SCons to rebuild those intermediate
> targets on its next invocation. How do you want to handle this?
>
> Finally, you should take into account that users might want to create
> different versions of the same source code. Just as an example, let's
> assume they want to create an "original" and "obfuscated" target(s), or a
> "simple" and "extended" release where additional packages get included
> based on environment variables. And for maximum speedup they do this in
> parallel, by using the "-j" option...so a single dedicated output dir won't
> do probably.
>
> Best regards,
>
> Dirk
>
>
> _______________________________________________
> Scons-dev mailing list
> Scons-dev at scons.org
> http://two.pairlist.net/mailman/listinfo/scons-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://two.pairlist.net/pipermail/scons-dev/attachments/20140530/74bd4fb4/attachment-0001.html>