Weblogs Forum - Binary Compatible Versus Open Source Java

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Weblogs Forum
Binary Compatible Versus Open Source Java

44 replies on 3 pages. Most recent reply: May 17, 2006 1:04 AM by Kondwani Mkandawire

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 44 replies on 3 pages [ « | 1 2 3 | » ]

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Binary Compatible Versus Open Source Java

Posted: Mar 31, 2006 11:42 AM

> > A perfect example of the advantage of the VM is getting
> > software for linux. You can either get the source, get
> > all the libraries it depends on, compile, maybe go on
> some
> > forums for help to get it working, or hope they have
> > binary for your distribution. With Java, once you have
> > the JRE, you can just get the jars and you are off and
> > running.
>
> And yet, the majority of software shipped by any GNU/Linux
> distribution is not in Java. By a far shot. ;)
>
> Why is that? Because deploying Java is a pain, due to the
> uses and abuses of JARs.

I'm not sure that's the reason. When I have a choice between a Java distribution and make, I'll pull the Java distribution. If you ask people why they use C or C++ over Java they almost invariably tell you that it's too slow.

> If you try to deploy several Java applications and
> libraries, they often end up requiring mutually
> incompatible versions of the same JARs, so that you have
> to make sure that your $CLASSPATH is properly sorted.

The setting the classpath is an environment variable is one of the biggest mistakes in Java for the reasons you mention. My advice is to completely override the classpath before starting the app. The $CLASSPATH variable is unecessary as is 'installing' the JRE and using an executable jar.

> In general, deploying Java works as long as you have the
> same VM as the author, and the author ships all the binary
> blobs himself, essentially turning Java applications into
> what C applications were back in the 80s, when static
> linking to Motif was en vogue. That's not a big issue,
> since most people currently only use Sun's VM,

We use several VMs from different vendors and this doesn't sound familiar. I've seen a few issues with rogue VMs but they come from companies whose products are bad in general.

> though we
> are changing that with GNU Classpath, Kaffe and gcj. Sort
> of how shipping C executables statically linked with Motif
> was probably not a big issue as long as most people used
> Sparc-Solaris.
>
> Switch the VM, and you're bound for surprises, since a lot
> of 'enterprisey' Java code out there uses undocumented
> classes in com.sun* packages which may or may not be
> present on the officially certified as compatible VM from
> IBM, or a even a newer release from Sun, for example.

I've found the surprises to be few and far between. THe exception, not the rule. I'm working on code right now that I am deploys on three VMs from three vendors.

> Nice idea, but Sun would never let that pas through the
> JCP.

I was thinking in terms of a more generic VM.

Dalibor Topic

Posts: 26
Nickname: robilad
Registered: Mar, 2006

Re: Binary Compatible Versus Open Source Java

Posted: Mar 31, 2006 12:05 PM

> I've found the surprises to be few and far between. THe
> exception, not the rule. I'm working on code right now
> that I am deploys on three VMs from three vendors.

That's because they all license Sun's class library, I assume. All proprietary VMs do, afaik.

The sloppy coding really shows when one tries to build the code with free runtimes, where com.sun.whatever.secret.Sun.api does not exist.

For an example of very bad code: Geronimo uses OpenEJB. OpenEJB's developer found it necessary to use reflection to get the actual VM classpath out of an undocumented, private field of Sun's ClassLoader implementation. Since that sort of things are undocumented, implementation specific details, it's next to impossible to get such applications to run correctly on implementations that are not derived from Sun's proprietary code base.

Such sloppy hacks are more of a rule, than an exception in the enterprise Java field. Just grep for imports of the unspecified sun.* and com.sun.* classes through your favourite open source J2EE server's code. They were all pretty bad last time I looked at them a few months ago.

cheers,
dalibor topic

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Binary Compatible Versus Open Source Java

Posted: Mar 31, 2006 12:25 PM

> > I've found the surprises to be few and far between.
> THe
> > exception, not the rule. I'm working on code right now
> > that I am deploys on three VMs from three vendors.
>
> That's because they all license Sun's class library, I
> assume. All proprietary VMs do, afaik.
>
> The sloppy coding really shows when one tries to build the
> code with free runtimes, where
> com.sun.whatever.secret.Sun.api does not exist.
>
> For an example of very bad code: Geronimo uses OpenEJB.
> OpenEJB's developer found it necessary to use reflection
> to get the actual VM classpath out of an undocumented,
> private field of Sun's ClassLoader implementation. Since
> that sort of things are undocumented, implementation
> specific details, it's next to impossible to get such
> applications to run correctly on implementations that are
> not derived from Sun's proprietary code base.

OK. Yeah, Even the JDK does this. All those concurrent libraries use the 'Unsafe' class if I recall correctly.

> Such sloppy hacks are more of a rule, than an exception in
> the enterprise Java field. Just grep for imports of the
> unspecified sun.* and com.sun.* classes through your
> favourite open source J2EE server's code. They were all
> pretty bad last time I looked at them a few months ago.

I think I must have gotten lost somewhere because I thought we were talking about the VM. This to me is about the classes. Sure, you can't expect two implementations of the JDK to work exactly the same. I think that a) bundling the JDK with the JRE is conveient but problematic (as you describe) b) the JDK is too big and we don't need all 32 MB of classfiles for any one Java application.

A lot of the stuff in the JDK used to be in separate jars. I think they put it into the JDK because people were complaining about getting all the necessary jars on the target machine. But this bloat is not sustainable. They need to at least stem off the growth of the JDK.

Dalibor Topic

Posts: 26
Nickname: robilad
Registered: Mar, 2006

Re: Binary Compatible Versus Open Source Java

Posted: Mar 31, 2006 3:52 PM

> I think I must have gotten lost somewhere because I
> thought we were talking about the VM.

Yeah, I seem to have been talking about classes. My aplogies for talking past each other.

> I think that a)
> bundling the JDK with the JRE is conveient but problematic
> (as you describe) b) the JDK is too big and we don't need
> all 32 MB of classfiles for any one Java application.
>
> A lot of the stuff in the JDK used to be in separate jars.
> I think they put it into the JDK because people were
> e complaining about getting all the necessary jars on the
> target machine. But this bloat is not sustainable. They
> need to at least stem off the growth of the JDK.

I agree. But it's more convenient this way for the companies on the J2SE EC, that way they don't have to ship CORBA, Swing, Rhino or something else as a separate download to their customers.

CPAN is to Perl what JCP is to Java. ;)

cheers,
dalibor topic

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: Binary Compatible Versus Open Source Java

Posted: Mar 31, 2006 9:57 PM

It seems to me that there are several issues mixed up in this exchange. Badly written Java software does not make Java bad. What it reflects is how some developers still don't understand the power of abstraction for simplifying software design and maintainability. Many still seem hellbent on performance, least amount of typing and perhaps most obscure implementation coupling.

Java is about take a step back from your bent on details and instead focus on portability, reliability and maintainability. There is a lot of power in that train of thinking.

Also, Java's mobile code is what provides the power in software maintainence. Being able to deploy updates directly to users, as soon as they are ready is great. The other aspect of mobile code that many miss out on is the Smart Proxy paradigm. A smart proxy is a serializable object that runs on the client. It might use one or more remote servers/services to do something for the client. The important thing is it lets you change the details without changing the interface(s). That is very powerful in environments where you need to optmize client/server interactions.

The consistency that the JLS provides is important to me too. I've had not so many good experiences with using multiple JVMs because of the fact that there are bugs in software, and well, a different JVM will have different bugs, in general, and thus we get the dreaded "write once test everywhere" problem when "where" means different JVM.

I've had great luck taking mainstream applications with 300 or more classes between windows and linux with sun's JVM.

I've made the plea with Sun to give src.zip to the classpath folks to remedy the "it's different, have to retest" problem.

I think that there's a lot of NIH in the opensource world, and while I appreciate their zeal for "having the source", I haven't found that I've had a problem with doing business without the source for the JVM. I do consult src.zip for understanding the context of some exceptions during development/debugging.

What we need is a software community that stops trying to reinvent things that already exist, and instead start creating real revolutions in development tools. The netbeans and eclipse platforms are finally providing a lot of opportunities for people to stop developing whole new IDEs, and instead create modules/plugins to solve specific problems.

I'm starting to spend a lot of effort toward Jini plugins for Netbeans to help generate configuration and template services.

The IDEs can help us to stop programming in low level detail and instead focus on specific patterns that promote good software designs.

The comment earlier about program correctness is interesting. The use of annotations is not the only piece of the solution here. There needs to be endpoint and invocation layer standards that support distributed authentication and authorization. All the WS-* efforts to do this with XML based invocation is not really adding any value, just trying to use an apparently neutral technology to do something that would be better done without all the overhead of XML. While the standards get developed, there will be proprietary implementations everywhere, and everyone will be paying to have transport technologies reinvented.

The JERI stack does this for Jini applications and its plugability allows Jini apps to talk just about any imaginable invocation layer and transport (via endpoints).

We'll have to see what Sun has to say at Java One...

Mark Thornton

Posts: 275
Nickname: mthornton
Registered: Oct, 2005

Re: Binary Compatible Versus Open Source Java

Posted: Apr 1, 2006 1:12 AM

We just might see improved support for dynamic languages in the JVM (targetted at Java 7):

JSR 292: Supporting Dynamically Typed Languages on the JavaTM Platform
http://www.jcp.org/en/jsr/detail?id=292

Mark Thornton

Posts: 275
Nickname: mthornton
Registered: Oct, 2005

Re: Binary Compatible Versus Open Source Java

Posted: Apr 1, 2006 2:03 AM

> And yet, the majority of software shipped by any GNU/Linux
> distribution is not in Java. By a far shot. ;)
>
> Why is that? Because deploying Java is a pain, due to the
> uses and abuses of JARs.

I don't think technical features or flaws of Java have much to do with this.

Dalibor Topic

Posts: 26
Nickname: robilad
Registered: Mar, 2006

Re: Binary Compatible Versus Open Source Java

Posted: Apr 2, 2006 8:51 AM

Invokedynamic, in my opinion, is more of a publicity stunt to keep the hope alive for potential deflectors to PHP/Ruby/$Anything_but_Java, than a serious attempt at turning the JVM into a general purpose platform.

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: Binary Compatible Versus Open Source Java

Posted: Apr 2, 2006 10:06 AM

> Invokedynamic, in my opinion, is more of a publicity stunt
> to keep the hope alive for potential deflectors to
> PHP/Ruby/$Anything_but_Java, than a serious attempt at
> turning the JVM into a general purpose platform.

In your mind, which features of the JVM are not desireable in software development? Which features, when turned off, would impact the Java security model, including mobile code?

So far, I've heard a lot of the dynamic language buffs talk about features of python, ruby etc. The primary theme is always about how much they have to type, and how verbose the Java language is. I'm not impressed by those arguments because I don't think that those, by themselves are a good reason to create a new language, and then ask the software development community to spend 10 years recreating all the software tools we have in one language for use in another.

Right now, we are on a 10 year yoyo cycle of software development. To date, we've yet to create any single language or runtime feature which can't be created in another language!

The emotional drive and charging dialog about the benefits of these dynamic languages is not providing any bang for the buck.

We need orders of magnitude changes in productivity, with corresponding improvements in program reliability. The vertical markets being attacked by APIs such as Ruby on Rails and now Grails are narrow from a general software perspective. I.e. those types of applications are focused on very specific types of software. Primarily they help with UI presentation, not with program correctness related to the collection, processing and managment of the information being presented.

Some might argue that simple loop interfaces and some other features help make the developers job easier. I've already stated above that those are largely typing issues. The syntax of the language and strong typing in Java make it difficult to get a loop and its execution wrong. If dynamic languages have problems managing loop execution because of lack of compile time typing, that might be an issue to discuss.

In the end, I'm not convinced that there's anything worth changing the whole world of software over. What Java needs, is better APIs for specific vertical markets!

J2EE is just too vertical in design. It provides a lot of things that are just not what should be done for the general case. However, it does provide some tools that are valuable for some types of complex enterprise applications. But, those tools come at a cost of complexity to the overall system that complicates the learning process and makes simple deployments extra verbose, which is one of the complaints of the dynamic language crowd.

I'm a Jini practitioner. I use Jini because it lets me deploy the smallest configuration and system I need, while supporting a much larger, more complex system design if needed. And, security and deployment configuration through the ConfigurationFile class is great because I can configure with real objects, and thus don't have to keep extending an XML configuration document with more types of data.

Java is not perfect, but the real value is there. It works, and provides a platform that is binary compatible across all the environments that I need.

Using other languages for simple tools is something I've done for years. But, I still create more reliable and useful software in Java than any other language.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Binary Compatible Versus Open Source Java

Posted: Apr 2, 2006 3:20 PM

> A) Binary compatibility is pretty well-defined for Java.
> It is not what many people think it is.
>
> BC only makes guarantees for linkage symbols, and makes no
> guarantees regarding the semantics of the symbols. That's
> why binary compatibility is pretty trivial: just implement
> all the symbols in an API with their access qualifiers,
> and you're done.
>
I think "binary compatibility" is overloaded in common use. In the JVM spec (at least the first edition), there's a chapter on binary compatibility that describes the kind of changes you can make to a type without requiring other types that previously linked to it to be recompiled. A change to type A is binary compatible with type B (which used to link to A just fine before the change), if B's old binary continues to link to A's binary after A has been changed and recompiled. I think that's the sense of "binary compatibility" you're talking about.

Gosling talked about that in one of the interviews I did with him:

James Gosling: I was searching everything I could find that I could come up with a decent way to implement that would allow you to evolve and still maintain binary compatibility. Probably the central thing in that is in the virtual machine spec. The central thing is really about dynamic binding rather than static binding. The whole notion of having a just-in-time compiler has a whole lot of different aspects to it. One of them is this issue that if you look at a Java binary, it doesn't have fixed offsets for fields. It doesn't have slot numbers for virtual function tables. It's got symbolic references. And they get bound essentially as late as possible. And that gives you a terrific amount of flexibility. Some things seem like they ought to be obvious, that you ought to be able to do them, like adding private members to classes. But in something that does static determination of offsets of fields, you can't do that. You have to do all of that stuff dynamically. It really makes the issue of evolution work a lot better.

http://www.artima.com/intv/gosling315.html

That binary compatibility is about making things independently evolvable, and you're right, it says nothing about semantics, just linking.

But the other way I've heard binary compability used in the context of Java is to differentiate that there's a binary standard, not just a source standard, for Java. And that standard does include the semantics. Having a binary standard means I can just plug in different JAR files and use them. I don't have to recompile them from source. That's a much less formal way people use the term, but it simply means that I can take Java binaries and plug them in and run them on any Java VM.

Rob Gingell, who used to chair the JCP, talked about this in this interview:

Rob Gingell: I think binary compatibility is much more market- and economically relevant than source compatibility independent of the technology. ... Having a shared space of binaries is much more vital and powerful than having a shared space of source. That's not to say that shared spaces of source are not valuable in their own right. It's just that the properties that attend to them are not the ones that have historically explained economic behavior in the industry.

http://www.artima.com/intv/standards5.html

That's the kind of binary compability that really helps me as a consumer of Java, and what I was referring to in this post. Is there a better term for that?

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Binary Compatible Versus Open Source Java

Posted: Apr 2, 2006 3:53 PM

> In general, deploying Java works as long as you have the
> same VM as the author, and the author ships all the binary
> blobs himself, essentially turning Java applications into
> what C applications were back in the 80s, when static
> linking to Motif was en vogue. That's not a big issue,
> since most people currently only use Sun's VM, though we
> are changing that with GNU Classpath, Kaffe and gcj. Sort
> of how shipping C executables statically linked with Motif
> was probably not a big issue as long as most people used
> Sparc-Solaris.
>
> Switch the VM, and you're bound for surprises, since a lot
> of 'enterprisey' Java code out there uses undocumented
> classes in com.sun* packages which may or may not be
> present on the officially certified as compatible VM from
> IBM, or a even a newer release from Sun, for example.
>
That hasn't been my experience. I used the Sun JVM on Linux for a long time, then at one point switched to BEA's because they had a nice management console. Somewhere along the way, I started getting OutOfMemoryErrors a lot, and for months I scratched my head trying to figure out what was wrong with my app. Then one day it occurred to me to try switching the VM. I tried about 5 VMs from 3 vendors (because I switched between Java versions too), and they all ran the app. Only the one I was using seemed to have the OutOfMemoryError problem, though that problem required that I run the app a while before it would surface. I really liked the fact that I had several vendor's JVM implementations to choose from.

I also right now am running BEA's JRockit 1.5 VM on the server, and I use Apple's VM on my Mac laptop. Are either of these derived from Sun's VM? I.e., do they have com.sun.* packages in there too? Regardless, I've never had a problem of not being able to run a JAR file on any JVM that I can remember.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Binary Compatible Versus Open Source Java

Posted: Apr 2, 2006 4:19 PM

> > forked now, but you just couldn't call it Java. I guess
> my
> > main desire is that the compatibility that comes out of
> > that Sun-controlled licensing process continues, because
>
>
> But... as long as Sun keeps this rule (ie. "You can only
> call it Java if it survives the TCK pounding"), then you
> don't have a problem, do you? Just use the thing with
> "Java" stamped on it, and be done.
> If someone does fork off something incompatible... they'll
> have a very tough time getting it used, because no one
> will want to use an incompatible version.
>
Yes, I think Sun requiring testing for compliance with the spec being to get to use the Java branding is independent of Sun releasing their implementation of the Java runtime open source. Whether or not someone would fork off something incompatible, and the extent to which that could damage my ability to just "sling binaries around" as Rob Gingell put it, is unknown. MS was forced through legal proceedings to not call their fork "Java," but it still created a lot of confusion and uncertainty in the marketplace. So forking can really even happen with the current licensing, it just isn't as easy as I think it would be if Sun's Java runtime were open sourced.

Nevertheless, nowadays Java has huge momentum, and it would be difficult for anyone to get much traction with a forked version even if their goal is to confuse things. IBM could conceivably get away with going their own way, but would they?

My main aim with this blog post was to counter Peter Yared's statements about LAMP and multiple languages on the JVM, and point out that the open source Java debate I think makes more sense is about research versus compability. I do believe that the current licensing approach of Java's runtime makes it harder for people outside Sun to innovate than an open source approach would, and I think that long term that can hurt me as a consumer of Java. I don't think that open source Java would lead to fragmentation, just that it would be easier. I wanted to point out that as a consumer of Java, I feel I benefit a lot from Sun's enforcement regime, and that the current licensing gives that more teeth than an open source license would.

Dalibor Topic

Posts: 26
Nickname: robilad
Registered: Mar, 2006

Re: Binary Compatible Versus Open Source Java

Posted: Apr 2, 2006 6:51 PM

> That's the kind of binary compability that really helps me
> as a consumer of Java, and what I was referring to in this
> post. Is there a better term for that?

I'd call that 'interoperability', i.e. different vendors' implementations use the same data (in your case JARs) in the same way.

That's what specification test suites are for. For open standards like XML, they are freely available to anyone without strings attached. For Java, they are not, though.

That's the main reason why the free software implementations of J2SE are still probably incompatible with Sun's implementation in some areas.

cheers,
dalibor topic

Dalibor Topic

Posts: 26
Nickname: robilad
Registered: Mar, 2006

Re: Binary Compatible Versus Open Source Java

Posted: Apr 2, 2006 7:19 PM

> I also right now am running BEA's JRockit 1.5 VM on the
> server, and I use Apple's VM on my Mac laptop. Are either
> of these derived from Sun's VM?

Yes.

> I.e., do they have
> com.sun.* packages in there too?
Yes, afaik. Both license J2SE from Sun.

In general, licensees seem to largely leave the class library alone, and concentrate upon porting the VM to their platforms.

IBM is a bit special since they are the only licensee I know that plays in all J2*E markets with their products, so they actually do tweak the class libraries a bit, update XML parsers, have their own crypto code, etc.

> Regardless, I've never
> had a problem of not being able to run a JAR file on any
> JVM that I can remember.

Well, one famous lamento about those things was in ANT's 1.6 release announcement message:

"Call to Inaction

A special message to whoever it is in charge of commands in tools.jar: stop moving your entry points! In Ant1.5 we had to deal with the 'classic' javac entry point going away in Java1.4.0, seemingly coming back later. In Java 1.4.2, the javah entry point moved. The traditional command line invocation mechanism has been replaced by hosted invocation -Ant, Maven, IDEs, etc, and moving entry points around breaks these host applications. Even if we get a bug fix out in Ant a few weeks after the Java release, it takes months for this to trickle down to end users, especially via IDEs and other distributions. For example, Sun's own Java Web Services Developer Pack ships with Ant1.5.1, and so cannot run <javah> on a 1.4.2 installation. "

That was particularly funny since those entry points are actually undocumented, implementation-specific classes a la com.sun.something.somewhere.javac.Main, which Sun reserved the right to shuffle around as they please (back then, changed somewhat with a recent JSR for javac).

That was what I mean above: a lot of popular Java software abuses implementation-specific details, i.e. it is not really portable. You just don't notice it much, since most proprietary implementations license the same code with the same behaviour, so things tend to behave similarly on the proprietary implementations.

cheers,
dalibor topic

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Binary Compatible Versus Open Source Java

Posted: Apr 2, 2006 7:50 PM

> That was what I mean above: a lot of popular Java software
> abuses implementation-specific details, i.e. it is not
> really portable. You just don't notice it much, since most
> proprietary implementations license the same code with the
> same behaviour, so things tend to behave similarly on the
> proprietary implementations.
>
I see. Thanks very much for posting to this discussion. The links Werner gave to the Open Source Java FAQ, and the two yhou gave to advogato.org, were quite helpful. And you have brought up something that I hadn't really thought of before: that the interoperability (as you suggested I call it) I like so much may arise not solely from the conformance test regime, but also to some extent from the fact that every licensee derives their Java API implementation from the same source, thereby inheriting most of the behavior.

That sounds a bit like an open source project already. Hibernate isn't a standard with multiple implementations and conformance tests, it is just a body of code that everyone shares. The Java API is a standard with multiple implementations and conformance tests, but you're saying it is also to a great extent a shared body of code.

Flat View: This topic has 44 replies on 3 pages [ « | 1 2 3 | » ]

Previous Topic

Next Topic