The Artima Developer Community
Sponsored Link

Weblogs Forum
Binary Compatible Versus Open Source Java

44 replies on 3 pages. Most recent reply: May 17, 2006 4:04 AM by Kondwani Mkandawire

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 44 replies on 3 pages [ « | 1 2 3 ]
Dalibor Topic

Posts: 26
Nickname: robilad
Registered: Mar, 2006

Re: Binary Compatible Versus Open Source Java Posted: Apr 2, 2006 11:51 PM
Reply to this message Reply
Advertisement
> In your mind, which features of the JVM are not desireable
> in software development?

I was talking about the invokedynamic instruction not being very useful in terms of turning the JVML into a general purpose programming language backend. We may be talking past each other here, as the JVML is obviously suited for software develpment in Java.

The JVML type system is very Java-specific. For example, it deliberately lacks support for unsigned primitives, since Gosling didn't think his target group would be able to deal with signed vs. unsigned types, from his experience with C developers.

That means if the type system of the language you want to run on top of the JVM is not JVML-like, you'll spend some time working around the limitations of the JVML.

That's why I am not really impressed by the invokevirtual JSR: it would just seem to save a few cycles otherwise burned on reflection for Java-like languages. It would not gain anything for languages that do not map nicely to JVML's type system, afaict, as you'd still implement method resolution & invocation manually.

> We need orders of magnitude changes in productivity, with
> corresponding improvements in program reliability.

I'd love to see something like spec# for Java.

Dalibor Topic

Posts: 26
Nickname: robilad
Registered: Mar, 2006

Re: Binary Compatible Versus Open Source Java Posted: Apr 3, 2006 12:25 AM
Reply to this message Reply
> The Java API is a standard with multiple
> implementations and conformance tests, but you're saying
> it is also to a great extent a shared body of code.

The java API is not a standard with multiple implementations.

There is only one, the proprietary RI, which is very much unlike open source, as should be quite obvious by reading its license, which among other things prohibits sharing the code.

You can, for a pretty hefty sum, buy the right to use the code from the proprietary RI in your own proprietary implementation, and that's how Sun makes money from licensing code to IBM, Apple, BEA, Oracle, and others.

While there are open source VMs like http://www.kaffe.org that use open source class libraries implementing the Java API like http://www.classpath.org they are probably not fully interoperable with the proprietary implementations yet because Sun refuses to make the specifications and the test suites available with no strings attached. Sun makes it impossible for a third party to verify how compatible these implementations are.

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: Binary Compatible Versus Open Source Java Posted: Apr 3, 2006 12:41 AM
Reply to this message Reply
> Well, one famous lamento about those things was in ANT's
> 1.6 release announcement message:

> That was particularly funny since those entry points are
> actually undocumented, implementation-specific classes a
> la com.sun.something.somewhere.javac.Main, which Sun
> reserved the right to shuffle around as they please (back
> then, changed somewhat with a recent JSR for javac).

This is what I call cutting one's throat. Trying to declare a standard (the class providing the "main(String[])" method) by assumption demonstrates the lack of understanding of what Java is all about. Clearly, one can, in fact, run javac, javah and all the other tools with Runtime.exec(String[]). So why, would you ever want to run it from the class directly?

You know absolutely nothing about the runtime environment of those tools, and including their "damage" inside of the ANT JVM just seems silly. I always use 'fork="true"'.

I just don't really understand how anyone could create a production tool that depended on a private entrypoint, and be mad when it broke.

The "must have native interface" crowd who has always had binary access to hack away at reverse engineering software are really good at creating very nasty behaving software.

I think it would have been a lot easier, to actually have fired up a JSR about 5 years ago and asked sun for a dependable interface to the tools at the class level if it was really necessary to run them within the same JVM. Though, I'm not convinced that it is really necessary for the most part.

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: Binary Compatible Versus Open Source Java Posted: Apr 3, 2006 1:11 AM
Reply to this message Reply
> I was talking about the invokedynamic instruction not
> being very useful in terms of turning the JVML into a
> general purpose programming language backend. We may be
> talking past each other here, as the JVML is obviously
> suited for software develpment in Java.

Yes, I understand your reference to invokedynamic. I was specifically asking what part of the JVML is not suited to software development. Or to be more direct, what key "features" of the JVML are the most limiting to software development?

> The JVML type system is very Java-specific. For example,
> it deliberately lacks support for unsigned primitives,
> since Gosling didn't think his target group would be able
> to deal with signed vs. unsigned types, from his
> experience with C developers.

Any variation in the behavior of a language based on the use or misuse of a keyword, simply doubles the number of bugs that can be created. Right now, volatile is a good example of a backwards keyword. It's bad to fail to use it when needed, and better, but not always optimal to always use it.

I fixed countless bugs in C programs over the years due to misuse of signed/unsigned declarations. One of the things that James decided was to fully specify the size of the native types. Since that also includes the number of usable bits, chosing signed values makes sure that the number of bits can never fall short of expectation.

> That means if the type system of the language you want to
> run on top of the JVM is not JVML-like, you'll spend some
> time working around the limitations of the JVML.

Only for languages that decided that an 'unsigned' type was somehow necessary. Unfortunately, some are specified that way. That extra bit doesn't really add much comfort. In the end, if one more bit solves a problem for you, you probably will need yet another before too long. So, using unsigned to get some extra range by adding a bit is kind of short sighted it seems to me. Especially given the wide range of problems and debates it has created.

> That's why I am not really impressed by the invokevirtual
> JSR: it would just seem to save a few cycles otherwise
> burned on reflection for Java-like languages. It would not
> gain anything for languages that do not map nicely to
> JVML's type system, afaict, as you'd still implement
> method resolution & invocation manually.

Yes, but the VMs of the dynamic languages are already doing this resolution themselves. It's a necessary step in finding the appropriate method to call. In the language I did at Bell Labs for the 5ESS switches Administrative Automation back in 1990, I grumbled about how much work I had to do in C to make the virtual calls work...

> > We need orders of magnitude changes in productivity,
> with
> > corresponding improvements in program reliability.
>
> I'd love to see something like spec# for Java.

Which specific features do you think will provide the most bang for the buck?

Bill Venners

Posts: 2249
Nickname: bv
Registered: Jan, 2002

Re: Binary Compatible Versus Open Source Java Posted: Apr 3, 2006 1:38 AM
Reply to this message Reply
> > The Java API is a standard with multiple
> > implementations and conformance tests, but you're
> saying
> > it is also to a great extent a shared body of code.
>
> The java API is not a standard with multiple
> implementations.
>
> There is only one, the proprietary RI, which is very much
> unlike open source, as should be quite obvious by reading
> its license, which among other things prohibits sharing
> the code.
>
Wait a second, doesn't the fact that it is called "reference" implementation imply that there is more than one implementation? Among the many implementations, the RI is the one to look at when there's a question about what behavior is "standard." Even if someone starts with the Sun code, they still add their own changes to it, which makes it a different implementation in my mind. They have to take that implementation back and have it pass the tests, don't they?

What I mean by "like open source," is that with an open source project, you don't have multiple implementations of a standard, you just have one shared body of code. That contrasts with one shared standard, and multiple bodies of code that implement that standard. What I was trying to say is that you pointed out to me that there's a shared body of code (Sun's implementation of the Java runtime, which Sun of course uses and others license) that may play a part in Java's interoperability, in addition to the standard enforcement through conformance testing.

> You can, for a pretty hefty sum, buy the right to use the
> code from the proprietary RI in your own proprietary
> implementation, and that's how Sun makes money from
> licensing code to IBM, Apple, BEA, Oracle, and others.
>
That's how I understood it to work.

> While there are open source VMs like http://www.kaffe.org
> that use open source class libraries implementing the Java
> API like http://www.classpath.org they are probably not
> fully interoperable with the proprietary implementations
> yet because Sun refuses to make the specifications and the
> test suites available with no strings attached. Sun makes
> it impossible for a third party to verify how compatible
> these implementations are.
>
I don't understand what you mean by "Sun makes it impossible for a third party to verify how compatible these implementations are." Can't a third party just go to Sun, pay some sum, and have the tests run on their implementation? Is the problem that the open source projects can't afford the money it costs to run the tests, or is it that their terms you would have to agree to even if they paid the testing fee would be incompatible with the open source license? Or perhaps something else. I don't quite understand what the block is.

Dalibor Topic

Posts: 26
Nickname: robilad
Registered: Mar, 2006

Re: Binary Compatible Versus Open Source Java Posted: Apr 3, 2006 1:46 AM
Reply to this message Reply
> Or to be more direct, what key
> "features" of the JVML are the most limiting to software
> development?

To software development in general? Probably none. I've got a few things I don't like very much with the design, though:

* the jsr instruction, which makes verification quite ugly. * the 64k size limit on methods makes it impossible to use certain techniques to implement continuations
* having a handful of different invoke* instructions
* having an explicit widening instruction, makes bytecode rewriting a bit more painful
* lack of a simple way to access Java code running on the JVM as library with "C" calling interface, though gcj+swig fixes that somewhat, judging by http://search.cpan.org/~rusekd/GCJ-Cni-0.03/lib/GCJ/Examples.pod
* finals are not really final

> I fixed countless bugs in C programs over the years due to
> misuse of signed/unsigned declarations. One of the things
> that James decided was to fully specify the size of the
> native types. Since that also includes the number of
> usable bits, chosing signed values makes sure that the
> number of bits can never fall short of expectation.

It creates a need to mask variables out when talking to native C code, though. So when you work with JNI to access native functions that accept or return unsigned values, you're back at having the problem.

> Which specific features do you think will provide the most
> bang for the buck?

* Adding pre and post conditions to the standard class library, and enforcing them within the compiler using a theorem prover. See ESC/Java & JML.

* Notation for non-null types. See nice.

Bill Venners

Posts: 2249
Nickname: bv
Registered: Jan, 2002

Re: Binary Compatible Versus Open Source Java Posted: Apr 3, 2006 2:41 AM
Reply to this message Reply
> This is what I call cutting one's throat. Trying to
> declare a standard (the class providing the
> "main(String[])" method) by assumption demonstrates the
> lack of understanding of what Java is all about. Clearly,
> one can, in fact, run javac, javah and all the other tools
> with Runtime.exec(String[]). So why, would you ever want
> to run it from the class directly?
>
> You know absolutely nothing about the runtime environment
> of those tools, and including their "damage" inside of the
> ANT JVM just seems silly. I always use 'fork="true"'.
>
People want to run it directly for speed, because it takes time to fire up the VM. I think the reason people usually go around the interface is performance. It makes it hard for the person who defined the interface to make changes to their implementation without breaking clients. And although you can often say, well, that's the fault of the people who went around the interface, so their stuff deserves to break. In practice sometimes you don't want that stuff to break, so you chose to limit your implementation changes to things that are compatible with clients that went around the interface.

Dalibor Topic

Posts: 26
Nickname: robilad
Registered: Mar, 2006

Re: Binary Compatible Versus Open Source Java Posted: Apr 3, 2006 2:52 AM
Reply to this message Reply
> Among the many implementations, the RI
> is the one to look at when there's a question about what
> behavior is "standard." Even if someone starts with the
> Sun code, they still add their own changes to it, which
> makes it a different implementation in my mind.

Sure. I meant different to mean "completely different", rather than "partially different", i.e. independent vs. derived code. My bad, sorry for the confusion.

> They have
> to take that implementation back and have it pass the
> tests, don't they?

The SCSL has a TCK (i.e. the test suite) section. According to it, they need to pass the tests before they release a compatible release. Afaict, licensees get to judge on compatibility themselves, but need to pass their results on to Sun. It's self-certification with a honors system, afaict from it.

See http://java.sun.com/j2se/1.5.0/scsl_5.0-license.txt for details.

> What I mean by "like open source," is that with an open
> source project, you don't have multiple implementations of
> a standard, you just have one shared body of code. That
> contrasts with one shared standard, and multiple bodies of
> code that implement that standard. What I was trying to
> say is that you pointed out to me that there's a shared
> body of code (Sun's implementation of the Java runtime,
> which Sun of course uses and others license) that may play
> a part in Java's interoperability, in addition to the
> standard enforcement through conformance testing.

That depends. Take an open standard like XML, where you have dozens of open and closed source implementations. They may or may not share code, but yet manage to interoperate quite alright.

If you need to know if the one you want to use implements the standard faithfully, you can grab the test suite from W3C, no strings attached, and see if the claims of the vendor hold water.

You can't do that with Java, since you don't have access to the test suite, and the test suite license does not allow you to test other people's implementations. It's a funny system, designed to extract cash from licensing the brand.

> I don't understand what you mean by "Sun makes it
> impossible for a third party to verify how compatible
> these implementations are." Can't a third party just go to
> Sun, pay some sum, and have the tests run on their
> implementation?

No, not really. Up to 1.5, any certified as compatible implementation was forced to include proprietary software from Sun, which squarely ruled open source implementations out, since Sun's license infected other, independent code. That's 9 years during which Sun actively made it impossible to have an officially compatible open source implementation.

Eventually, Sun published the test suite under a "read only" license, that only allows reading the tests, but forbids actually compiling or using them to test an implementation.
The license is available at https://jck.dev.java.net/jck-read-only-license.txt Publishing the TCK under the "read only" license was a nice PR stunt for Graham Hamilton, but it was completely useless to anyone else.

After we've rattled Sun's chains for 9 years, since 1.5, it is now theorethically possible for a non-commercial author to get access to the J2SE test suite through the TCK scholarship. Noone has got one yet, I've applied and I'm working my way through the legal booby traps. Unfortunately, it's an NDA-ridden process itself, so I can't tell you much about it yet.

> Is the problem that the open source
> projects can't afford the money it costs to run the tests,
> or is it that their terms you would have to agree to even
> if they paid the testing fee would be incompatible with
> the open source license?

In general, the problem with most of Sun's Java technology licensing is that Sun can pull an SCO on you any time, as the agreements are very asymetrical.

Here is an example: You can see Apache's deal with Sun at http://jakarta.apache.org/site/jspa-position.html and note how shaky the whole thing is.

It almost blew up in their face when Sun's legal temporarily forced Apache Geronimo to include weird 'Notices from Sun Microsystems' on their web site, saying that the all derivative works of the open source Geronimo code must be recertified and re-licensed by Sun, which contradicts Apache Foundation's own license. I've explained that issue at http://www.advogato.org/person/robilad/diary.html?start=52
Eventually Sun's legal division stopped whatever it was up to, though.

> Or perhaps something else. I
> don't quite understand what the block is.

There are no test available with no strings attached, simply put. What you can get, you are not allowed to use. What you can beg for, I am not allowed to talk about what the block is.

I do think, though, that having to beg for access to a compatibility test suite under non-discriminatory terms flies in the face of the idea of Java as an open standard.

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: Binary Compatible Versus Open Source Java Posted: Apr 3, 2006 11:55 AM
Reply to this message Reply
> > You know absolutely nothing about the runtime environment
> > of those tools, and including their "damage" inside of the
> > ANT JVM just seems silly. I always use 'fork="true"'.
> >
> People want to run it directly for speed, because it takes
> time to fire up the VM. I think the reason people usually
> go around the interface is performance.

Yes, that might be the motivation, but is that really the result? I've used Kawa for years to do java development and it just run javac from the commandline when I push F7 to compile the current file. For most source files, it's done compiling before I can shift my eyes down to the output window. Bigger source files take longer to compile, but that is not an issue related to new processes.

Back in the days of JDK1.1, I did use jikes for compiling individual files because it did start faster than javac. But starting with JDK1.3 and even more so in JDK1.4, javac started up fast enough with hotspot that I did not need to use jikes.

> It makes it hard
> for the person who defined the interface to make changes
> to their implementation without breaking clients. And
> although you can often say, well, that's the fault of the
> people who went around the interface, so their stuff
> deserves to break. In practice sometimes you don't want
> that stuff to break, so you chose to limit your
> implementation changes to things that are compatible with
> clients that went around the interface.

This is a trival interface into the main(String[]) method, and I don't want to over accentuate this issue as a big deal. But, I think it's a great example of how the implementation details of an interface, namely ant's <javac> task, crept into view in an unfortunate way.

Yes, sun should have left the old main(String[]) entry point in place. But, even sun, has only recently started to use ant. They used make for eons, and many visible signs of make still exist.

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: Binary Compatible Versus Open Source Java Posted: Apr 3, 2006 1:55 PM
Reply to this message Reply
> > Or to be more direct, what key
> > "features" of the JVML are the most limiting to
> software
> > development?
>
> To software development in general? Probably none. I've
> got a few things I don't like very much with the design,
> though:
>
> * the jsr instruction, which makes verification quite
> ugly.

It is complicated, but the spec has some information that can help guide one to get it right. JSR makes finally work more easily from the runtime environment than a "branch" of some sort, without return context could.

> * the 64k size limit on methods makes it impossible
> to use certain techniques to implement continuations

This does create some issues. I've not had to deal with this issue myself, so I don't know "how bad" it would be for continuations.

> * having a handful of different invoke* instructions

Is there a specific misfeature of this that bothers you?

> * having an explicit widening instruction, makes bytecode
> rewriting a bit more painful

Can you provide a specific example?

> * lack of a simple way to access Java code running on the
> JVM as library with "C" calling interface, though gcj+swig
> fixes that somewhat, judging by
> http://search.cpan.org/~rusekd/GCJ-Cni-0.03/lib/GCJ/Example
> s.pod

It is is trivially possible to create a JNI based application that advertises C language APIs that talk to Java. Calling into Java is the hard part though...

> * finals are not really final

Isn't that a sun implementation issue?

> > I fixed countless bugs in C programs over the years due
> to
> > misuse of signed/unsigned declarations. One of the
> things
> > that James decided was to fully specify the size of the
> > native types. Since that also includes the number of
> > usable bits, chosing signed values makes sure that the
> > number of bits can never fall short of expectation.
>
> It creates a need to mask variables out when talking to
> native C code, though. So when you work with JNI to access
> native functions that accept or return unsigned values,
> you're back at having the problem.

Depending on what you are doing with the values right? You can freely store the unsigned value in the signed java instance, do twos-complement math on it etc. When you need to reference the value as a numeric, then you need to perform the appropriate masking as in

long x = ((0xffffffffl)val32BitUnsigned);
or
int y = ((0xffff)val16BitUnsigned);

As I said the big issue for me is that if that single bit is all that makes it work for you, there are probably more important things to consider.

In addessing, it doubles the available ram space. That can be important, but 32bits hasn't been much better than 31bits was. So now we're all looking at 64bits instead.

I regularly use long values in places where I have native integer values and don't know the possible ranges. I also make use of java.lang.Number to beat back precision problems.

> > Which specific features do you think will provide the
> most
> > bang for the buck?
>
> * Adding pre and post conditions to the standard class
> library, and enforcing them within the compiler using a
> theorem prover. See ESC/Java & JML.

This is one of the possibilities that I think would be valuable to more people than any of the other features.

> * Notation for non-null types. See nice.

Would not pre conditions help with asserting non-null types with a single implementation of prereqs instead of a specialization specifically for one type?

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

Re: Binary Compatible Versus Open Source Java Posted: Apr 4, 2006 10:54 AM
Reply to this message Reply
> > * Notation for non-null types. See nice.
>
> Would not pre conditions help with asserting non-null
> types with a single implementation of prereqs instead of a
> specialization specifically for one type?

Would not pre conditions help with asserting String argument types or HashMap argument types or ... (instead of static type checking)?

The type system implemented for Nice allows more error checks at compile time rather than run time (NullPointerException) - the same arguments you might make for the benefits of static type checking in Java, can be made for the benefits of non-null types in Nice.

(Incidentally, Nice also provides requires pre-conditions and ensures post-conditions.)

"alt.lang.jre: Twice as Nice"
http://www-128.ibm.com/developerworks/library/j-alj10064.html

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: Binary Compatible Versus Open Source Java Posted: Apr 5, 2006 9:49 AM
Reply to this message Reply
> Would not pre conditions help with asserting String
> argument types or HashMap argument types or ... (instead
> of static type checking)?

I guess I'm not sure what you mean hear. If you type it into your source as a prereq formatted text string, or as a formal type argument formatted text string, haven't you specified the same thing, but are just requiring a new way to say the same thing?

> The type system implemented for Nice allows more error
> checks at compile time rather than run time
> (NullPointerException) - the same arguments you might make
> for the benefits of static type checking in Java, can be
> made for the benefits of non-null types in Nice.

Yes, I know the benefits of 'non-null' checks. The question is whether that 'constraint' on values passed into a method is any more important than another 'constraint', such as (> 0) or (10 <> 20) etc.

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

Re: Binary Compatible Versus Open Source Java Posted: Apr 5, 2006 1:37 PM
Reply to this message Reply
The question is whether that 'constraint' on values passed into a method is any more important than another 'constraint', such as (> 0) or (10 <> 20) etc.
imo the question is 'what more can we do to check that these and other constraints are satisfied'.

Dalibor Topic

Posts: 26
Nickname: robilad
Registered: Mar, 2006

Re: Binary Compatible Versus Open Source Java Posted: Apr 8, 2006 12:09 PM
Reply to this message Reply
> > * Notation for non-null types. See nice.
>
> Would not pre conditions help with asserting non-null
> types with a single implementation of prereqs instead of a
> specialization specifically for one type?

Writing

String! var,

is much nicer than writing

String var;
@NonNull var.

at least for me. ;)

cheers,
dalibor topic

Kondwani Mkandawire

Posts: 530
Nickname: spike
Registered: Aug, 2004

Re: Binary Compatible Versus Open Source Java Posted: May 17, 2006 4:04 AM
Reply to this message Reply
RE: Open sourcing Java; for those of you who haven't read this yet:

http://www.vnunet.com/vnunet/news/2156205/sun-promises-open-source-java

Flat View: This topic has 44 replies on 3 pages [ « | 1  2  3 ]
Topic: Binary Compatible Versus Open Source Java Previous Topic   Next Topic Topic: Another Notch for JavaScript: HD DVD


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us