[Scons-dev] Java Development

[William Deegan]

William,
On July 25, 2014 at 8:27:02 PM, William Blevins (wblevins001 at gmail.com) wrote:

Team,

I want to get another thread going for SCons Java development.

The SCons Java tool has a ton of error reports on Tigris including 7 priority 1 issues.  At the moment, this tool doesn't stand a chance against other Java competitors, and not because they are great tools.  I frankly hate ANT.  I have used Java support from SCons and it's seriously painful; nothing like the C++ support.  Some other developers have made statements like "No one outside SCons builds Java programs more complicated than hello world."  The SCons tool framework is great, and I would really like to see the Java toolkit see some love.  It has potential to be a hidden gem, and I want help out with this, but I don't have the experience to do this on my own, so firstly I'd like to list some of the biggest hurdles to users SCons Java.  I'm not gonna try and propose any solutions at the moment.  I just want to see if I can get the group thinking about the problems.

1. Adding resource files to a jar causes SCons segfault: http://scons.tigris.org/issues/show_bug.cgi?id=2550

I have firsthand experience with this bug.  The only way I could figure out how to workaround it was to make a separate jar just for resources.

2. Java emitter almost never gets the java output correct.

One of the many things I hate about ANT is that ANT is stupid.  It always executes a build even if code is up-to-date and I usually have to explicitly clean.  SCons COULD resolve both problems if the emitters worked.  The only way to get remotely consistent working build is to call Jar( 'buildDir' ) when everyone wants to do Jar( [ 'class1', ... 'classN' ] ).

3. Dependencies:

SCons does not automatically add classpath items as dependencies.  Why do I need to do this manually?  This is what SCons does!  It's the heart and soul!

I believe this is because of item #6.

4. Consistency:

Classpath tokens (among other items) do not behave the same as other builders.
Example: I cannot use "#jar/item.jar" in the classpath without expanding via something like File(...).get_path().

5. Interfaces:

Java(...) parameters and internal handling aren't intuitive and only handles sources = 'directory' correctly.  It doesn't do lists of java files or list of directories in a sane manner.
http://scons.tigris.org/issues/show_bug.cgi?id=1772

Personally, I don't think that Java and Jar should be separate functions.  How do you get to the classes then?  What about Javah!  I have an idea, but that's outside the scope of this rant.

6. Performance:

The dependency structure for Java exposes class files in a way that creates tons of false positives.

SCons current:

classes1 = Java(...)

classes2 = Java(...)

Depends( classes2, classes1 ) # O( N^2 ) dependency graph with tons of false positives

SCons if I have anything useful to say about it

jar1 = Jar( classes1 )

jar2 = Jar( classes2 )

Depends( jar2, jar1 ) # O( 1 ) which obviously fails in parallel builds currently.

I am currently data mining a production Java codebase to prove my point. Dirk and I have already discussed this issue somewhat; thanks Dirk :)

This actually causes the Task Master thread to get blocked on large jars reducing parallel efficiency in builds to None.


The main issue here (if I understand SCons’ internals enough) is that SCons’s doing all dependencies on a per file basis. For many types of builds that works fine. For Java (building jars, and other issues) and some other types of builds, that’s very inefficient.

There’s no “blob” of files where you have N inputs and M outputs, and thus you get the N*M arcs in the graph.

Currently the only similar but not really similar enough is the Dir() Node type.  But that has it’s own problems, which could be solved by a N*M type node.



Other Java issues could likely be resolved building on top of such a new Node type.

Though resolving the anonymous and inner classes in a java file creating more than one class file and what it might be named is also still an issue which the scanner and emitter try to solve by parsing the java files and figuring out the proper naming.  This of course is not (as I understand it) formally defined as part of the java language and thus is a per compiler implementation detail.

That’s what I have in my head regarding the outstanding issues from Java.. 

-Bill





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://two.pairlist.net/pipermail/scons-dev/attachments/20140727/6d6bdbba/attachment.html>