The Artima Developer Community
Sponsored Link

Weblogs Forum
Going all in...

22 replies on 2 pages. Most recent reply: Aug 16, 2005 6:47 PM by Mike Petry

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 22 replies on 2 pages [ « | 1 2 ]
Alexander Jerusalem

Posts: 36
Nickname: ajeru
Registered: Mar, 2003

Re: Going all in... Posted: May 17, 2005 7:58 AM
Reply to this message Reply
Advertisement
> I believe mobile code is a better solution for these
> business solutions because it hides the data and presents
> only the public interface to that data, no matter how the
> data evolves.

I agree that it is possible to replace declarative constraints on data with mobile code that does the same checks procedurally. But apart from my personal preference for declarative constraint languages, there is of course the fact that code has to be executed to enforce procedurally defined constraints. There is no separation of the logical definition of constraints and their executable form.

If I consider the technolgy landscape in a typical company and if I look at trends like on demand computing and if I hear that integration issues are among the biggest concerns of most IT organisations today, then I feel that locking away business rules and business data inside executables that are bound to a particular runtime, is just the wrong way to go.

> > That's true. But there are many
> > people who think that using POJOs (e.g. RMI) to
> > communicate within a distributed system is somehow
> simpler
> > than using XML web services.
>
> I am one of those people. It is vital to understand my
> point of view on why. It has to do with how many times
> the data changes forms, and the amount of bandwidth and
> text based manipulations the data goes through.

But that's more of a performance argument than a simplicity argument.

> > However, if we accept that
> > the wire format has to be managed then that argument
> > breaks down. It's never simple to manage the versioning
> of
> > serialisation formats or any data format for that
> matter.
> > It boils down to the question, what is the best
> language
> > to define message formats that provide some degree of
> > backward and forward compatibility, so that older
> senders
> > can communicate with more recent receivers and vice
> versa.
>
> I think that you have to be ready to decide is the design
> so wrong that the new version will be difficult to manage.
> If the difficulty is high and the likelyhood of breakage
> e is dominating the conversations, then it might be best
> to decide that there will not be a compatibility.

I agree with that. But you can certainly make this situation less frequent by using a flexible data model and schema language.

> Mobile
> code makes it much easier to make this decision because
> you can send the client a smart proxy that provides the
> same interface contract, but understands only the new data
> stream format.

But that means you have to be in a position to control _all_ participating applications. If that is the case, I entirely agree that you can solve many (if not all) versioning problems in this way. But the lack of central control is exactly the problem that data oriented protocols aim to solve. If you push (or pull) a dynamic proxy into some application and that proxy modifies the wire format, the result could be that the application communicates perfectly well with your service, but stops working with other over which you don't have control. Unless of course you manage that as well, so your service gives the app a different proxy than some other service. Also, you can't communicate with anything not written in Java and you can't do something like switch to J2SE 5 unless all your clients have switch their runtime as well.

> However, I strongly believe that this interface to HTTP is
> keeping people from using more trivial solutions such as
> those provided by the RMI programming model. It is harder
> to get started with mobile code, but only because we don't
> have deployment automation tools sitting around which
> build jar and deploy them to webservers and plug in
> codebase URLs to servers as needed.

I think the problems are not of a purely technical nature. There's a trust and management issue as well. I don't think there are many companies who will let you deploy code into their ERP systems. The scenario you describe may be viable in a typical thin client situation, but not where business systems interact with each other like in a supply chain management system. But even with something as simple as Java Applets, it hasn't worked and there are constant troubles with JVM versions. Many people say they always supply a JVM with their Java app for exactly those versioning and management reasons.


> > Thankfully, you have already accepted that Java's
> default
> > serialisation is unsuitable if you need to manage
> > versioning in a distributed system without central
> > control, so I don't have to come up with an example for
> > that one ;-)
>
> It is unsuitable if you think that it should be used for
> long term storage of data exchanged by disparately managed
> systems where change is common. It works fine for local
> systems long term storage and for disparately managed
> systems which never change.
>
> Put another way, if you use simple valued data items,
> serialization works fine.

The problem I see with default serialisation is that it causes version breaks even when there are no logical problems. For example, if you change the order of instance variables in the code. The addition of an optional instance variable shouldn't cause much trouble really, but it does if you put it in the wrong place lexically, even though the order of instance variables is insignificant in the Java language. This is a bad mixup of logical and low level representational concerns.

> But, if you use mobile code, instead of just data streams,
> you'll find that you can make better choices on the
> evolution of your data, and probably minimize the number
> of versions you have to deal with being active
> simultaneously.
>
> Mobile code and interface based programming reduce your
> dependencies to just the operations that are needed,
> instead of formatting the data for local use and then code
> to use that data.

You gain some functionality but at a high cost. You depend on particular versions of particular runtime environments and you lock data away behind application APIs. Exactly the kind of silos that cause so many integration troubles. That may be a solution for a small set of in-house applications, but it will never work without close cooperation between all concerned. The coordination overhead for this kind of solution grows quickly and scales badly

Alexander Jerusalem

Posts: 36
Nickname: ajeru
Registered: Mar, 2003

Re: Going all in... Posted: May 17, 2005 8:07 AM
Reply to this message Reply
> Except for almost trivial cases it is not reasonable to
> publish contracts as interfaces. If Your input depends on
> previous input, on different other parameters or has to be
> parsed first the only practical solution may be testing.

There are static constraints and there are dynamic constraints. As a user of an interface, I want to know what the static constraints are. Not just for static validation and local testing, but also because it enables me to give users feedback as early as possible. It makes no sense to let a user type in 50 characters in a field where only 20 are allowed. I don't want to throw test messages at a web service to find out what number range I am allowed to put into some field. Yes they are trivial cases, but these trivial cases make up 80 % of all constraints. If these things are not fomally expressed in the interface contract, it causes coordination overhead. Developers will want to find out what the rules are _before_ testing, simply because it's less work.

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: Going all in... Posted: May 17, 2005 1:59 PM
Reply to this message Reply
> If I consider the technolgy landscape in a typical company
> and if I look at trends like on demand computing and if I
> hear that integration issues are among the biggest
> concerns of most IT organisations today, then I feel that
> locking away business rules and business data inside
> executables that are bound to a particular runtime, is
> just the wrong way to go.

No matter what technology you choose, you will be locked into the power, capabilities and vendor(s) of that technology. For me, it is a matter of least lock in. J2SE is the least lockin from a computing platform and vendor perspective. J2EE vendors lock you into particular JVMs and there are numerous other issues with J2EE.

.Net locks you into Windows. perl, python etc., are for all practicle purposes, single vendor (the community supported release).

So, for me, J2SE is currently the best platform for my needs.

> > I am one of those people. It is vital to understand my
> > point of view on why. It has to do with how many times
> > the data changes forms, and the amount of bandwidth and
> > text based manipulations the data goes through.
>
> But that's more of a performance argument than a
> simplicity argument.

No, it's simplicity. I don't have to deal with all the associated tools and APIs and manage code and interfaces that require this conversion back and forth.

> > I think that you have to be ready to decide is the design
> > so wrong that the new version will be difficult to manage.
> > If the difficulty is high and the likelyhood of breakage
> > e is dominating the conversations, then it might be best
> > to decide that there will not be a compatibility.
>
> I agree with that. But you can certainly make this
> situation less frequent by using a flexible data model and
> schema language.

I don't see how you can optimize your data flows by only have data based interfaces. How do you change an interface to get its data from two sources instead of one without changing the clients everywhere. With mobile code, the downloaded smart proxy can make the coallescing happen without the client having to change.

> > Mobile
> > code makes it much easier to make this decision because
> > you can send the client a smart proxy that provides the
> > same interface contract, but understands only the new
> data
> > stream format.
>
> But that means you have to be in a position to control
> _all_ participating applications.

No, it means you have to be in charge of the server. The participating applications download new/updated code that handles the change dynamically, never knowing that something under the covers has changed.

> If that is the case, I
> entirely agree that you can solve many (if not all)
> versioning problems in this way. But the lack of central
> control is exactly the problem that data oriented
> protocols aim to solve.

Data oriented protocols demand that everyone agree on the data format don't they? Doesn't that require the server to then be in charge of making the clients change at the right moment to use a new format of data?

> If you push (or pull) a dynamic
> proxy into some application and that proxy modifies the
> wire format, the result could be that the application
> communicates perfectly well with your service, but stops
> working with other over which you don't have control.

Okay, I sense you're not think about codebases. In the Java RMI programming model, downloaded code can include a codebase URL which specifies where the downloaded code comes from. This means that each server that a client references can supply a different implementation of the same interface. Thus, one service doesn't necessarily break another as long as the codebases don't overlap in a way that creates breakage. It's possible to have a poorly designed codebase that doesn't provide correct isolation, but it's just as possible to create a well isolated codebase.

In Jini2.0, we have things in the JERI stack and in the APIs that simplify many issues. The HTTPMD protocol includes and MD5 checksum in the URL structure. This means that a client can grant permissions to a particular version of the code to isolate itself from unexpected changes. It also means that you'll get a new classloader when the jar changes so that the refering code won't be subject to URL caching that might make it use the old code.

> Unless of course you manage that as well, so your service
> gives the app a different proxy than some other service.
> Also, you can't communicate with anything not written in
> Java and you can't do something like switch to J2SE 5
> unless all your clients have switch their runtime as
> well.

I don't have problems serializing data between JDK 1.5 and JDK 1.4 when JDK1.4 only features are used... It is true that you can't download JDK1.5 specific code into a JDK1.4 VM. That's a technical issue with implementation, not a practicle issue with downloaded code in general.

> > However, I strongly believe that this interface to HTTP is
> > keeping people from using more trivial solutions such as
> > those provided by the RMI programming model. It is harder
> > to get started with mobile code, but only because we don't
> > have deployment automation tools sitting around which
> > build jar and deploy them to webservers and plug in
> > codebase URLs to servers as needed.
>
> I think the problems are not of a purely technical nature.
> There's a trust and management issue as well. I don't
> think there are many companies who will let you deploy
> code into their ERP systems. The scenario you describe may
> be viable in a typical thin client situation, but not
> where business systems interact with each other like in a
> supply chain management system.

The httpmd protocol handler make sure that once I approve a particular jar to be downloaded into my system, I don't have to worry about that jar changing. This is a much better situation that granting by URL and/or Signed jar. The result is that an updated jar file, will need a specific policy grant and perhaps approval depending on the granting policy practices of the receiving JVM.

> But even with something as
> simple as Java Applets, it hasn't worked and there are
> constant troubles with JVM versions. Many people say they
> always supply a JVM with their Java app for exactly those
> versioning and management reasons.

We do try and manage the version of the JVM that we use because typically, we change the jre/lib/ext content and don't want to contaminate another JVM.

The quality of the JVM releases can be a big issue for some types of applications. The amount of software that one depends on being "right" is quite large. But, if you are not willing to buy into a support contract, then you will have to deal with problems with work arounds or wait until fixes arrive. With contracts, you can get help and fixes fairly quickly.

> > > Thankfully, you have already accepted that Java's
> > default
> > > serialisation is unsuitable if you need to manage
> > > versioning in a distributed system without central
> > > control, so I don't have to come up with an example
> for
> > > that one ;-)
> >
> > It is unsuitable if you think that it should be used for
> > long term storage of data exchanged by disparately managed
> > systems where change is common. It works fine for local
> > systems long term storage and for disparately managed
> > systems which never change.
> >
> > Put another way, if you use simple valued data items,
> > serialization works fine.
>
> The problem I see with default serialisation is that it
> causes version breaks even when there are no logical
> problems. For example, if you change the order of instance
> variables in the code. The addition of an optional
> instance variable shouldn't cause much trouble really, but
> it does if you put it in the wrong place lexically, even
> though the order of instance variables is insignificant in
> the Java language. This is a bad mixup of logical and low
> level representational concerns.

Humm, I've never had this problem. I just quickly checked JDK1.5 and JDK1.4 and neither seems to have this behavior.

> > But, if you use mobile code, instead of just data streams,
> > you'll find that you can make better choices on the
> > evolution of your data, and probably minimize the number
> > of versions you have to deal with being active
> > simultaneously.
> >
> > Mobile code and interface based programming reduce your
> > dependencies to just the operations that are needed,
> > instead of formatting the data for local use and then code
> > to use that data.
>
> You gain some functionality but at a high cost. You depend
> on particular versions of particular runtime environments
> and you lock data away behind application APIs. Exactly
> the kind of silos that cause so many integration troubles.

I've been using RMI for rpc and integration since jdk1.3 (I used Voyager in JDK1.X, and never used JDK1.2 due to its bugs). I've never had a JVM dependency problem. Perhaps my usage patterns eluded the issues that you have experience with?

> That may be a solution for a small set of in-house
> applications, but it will never work without close
> cooperation between all concerned. The coordination
> overhead for this kind of solution grows quickly and
> scales badly

Code is downloaded by the users of the data. Thus, the producer doesn't have to worry about these integration issues. The users of the service will be presented with a codebase associated with the downloaded code. They can retrieve the current, correct version of the code and will be very much able to use the service as it changes transports or other internal implementation details.

Alexander Jerusalem

Posts: 36
Nickname: ajeru
Registered: Mar, 2003

Re: Going all in... Posted: May 18, 2005 12:41 PM
Reply to this message Reply
> > If I consider the technolgy landscape in a typical
> company
> > and if I look at trends like on demand computing and if
> I
> > hear that integration issues are among the biggest
> > concerns of most IT organisations today, then I feel
> that
> > locking away business rules and business data inside
> > executables that are bound to a particular runtime, is
> > just the wrong way to go.
>
> No matter what technology you choose, you will be locked
> into the power, capabilities and vendor(s) of that
> technology. For me, it is a matter of least lock in.
> J2SE is the least lockin from a computing platform and
> d vendor perspective. J2EE vendors lock you into
> particular JVMs and there are numerous other issues with
> J2EE.
>
> .Net locks you into Windows. perl, python etc., are for
> all practicle purposes, single vendor (the community
> supported release).

Exactly, and that's why passing well modeled data around is preferable whenever there is no central control over deployment, versioning, etc or when it simply gets too complex to manage and you need more flexiblity or when you want to build systems in a way that is amenable to integration with other systems in the future.

> So, for me, J2SE is currently the best platform for my
> needs.

J2SE is certainly a very good tool for running procedural code on multiple platforms.

> > > I am one of those people. It is vital to understand
> my
> > > point of view on why. It has to do with how many
> times
> > > the data changes forms, and the amount of bandwidth
> and
> > > text based manipulations the data goes through.
> >
> > But that's more of a performance argument than a
> > simplicity argument.
>
> No, it's simplicity. I don't have to deal with all the
> associated tools and APIs and manage code and interfaces
> that require this conversion back and forth.

If I need to manage serialisation and use all these mapping tools like JAXB, I find it more complex than plain XML/XSLT. But that may be a matter of taste to some degree.


> > If that is the case, I
> > entirely agree that you can solve many (if not all)
> > versioning problems in this way. But the lack of
> central
> > control is exactly the problem that data oriented
> > protocols aim to solve.
>
> Data oriented protocols demand that everyone agree on the
> data format don't they?

Yes, more or less. The important thing is that they can agree on a meta model that allows you to define which particular parts of the data must be understood, which ones are respected if supplied, how other parts are ignored, and so on. If done right, there can be a lot of flexibility in that agreement.

> Doesn't that require the server
> to then be in charge of making the clients change at the
> right moment to use a new format of data?

That depends on what kind of change you're talking about and how that client integrates with other software. Of course, if the server makes a breaking change, and demands that all clients adhere to the new rules or are simply denied access, then it would be helpful to have a convenient way of shipping the updated code to the client. That's a very nice feature for closely coupled systems. Interestingly, it never worked outside of simple UI bits and pieces.

> > If you push (or pull) a dynamic
> > proxy into some application and that proxy modifies the
> > wire format, the result could be that the application
> > communicates perfectly well with your service, but
> stops
> > working with other over which you don't have control.
>
> Okay, I sense you're not think about codebases. In the
> Java RMI programming model, downloaded code can include a
> codebase URL which specifies where the downloaded code
> comes from. This means that each server that a client
> references can supply a different implementation of the
> same interface. Thus, one service doesn't necessarily
> break another as long as the codebases don't overlap in a
> way that creates breakage. It's possible to have a poorly
> designed codebase that doesn't provide correct isolation,
> but it's just as possible to create a well isolated
> codebase.

I know RMI, but the whole thing just isn't robust enough. Just look at some of the mailing lists that discuss deployment issues in J2EE applications. Just to make client program work with more than one type of application server can be troublesome. They compile proxies differently or not at all. They have particular ways of deploying JNDI drivers. They depend on various helper libraries or particular third party library versions. Some support JDK 1.3 while other already run on 5. There's huge number of security issues and so on. I totally agree that these are not fundamental logical problems at all and one should think that there's a better way to do it. But again, this industry has been trying to do it for many years and so far it always failed. Even if you limit interop to the Java only.


> In Jini2.0, we have things in the JERI stack and in the
> APIs that simplify many issues. The HTTPMD protocol
> includes and MD5 checksum in the URL structure. This
> means that a client can grant permissions to a particular
> version of the code to isolate itself from unexpected
> changes. It also means that you'll get a new classloader
> when the jar changes so that the refering code won't be
> subject to URL caching that might make it use the old
> code.

I have to admit that I don't know Jini very well. Maybe it has better solutions to some of the problems with N:M third party library versioning, JVM versions, etc. But still, it is a purely Java thing and integration between applications that weren't designed to interoperate is hard if everything is hidden behind APIs and accessible only on a single runtime platform.

> > Unless of course you manage that as well, so your
> service
> > gives the app a different proxy than some other
> service.
> > Also, you can't communicate with anything not written
> in
> > Java and you can't do something like switch to J2SE 5
> > unless all your clients have switch their runtime as
> > well.
>
> I don't have problems serializing data between JDK 1.5 and
> JDK 1.4 when JDK1.4 only features are used...
> It is true
> that you can't download JDK1.5 specific code into a JDK1.4
> VM. That's a technical issue with implementation, not a
> practicle issue with downloaded code in general.

Well, first of all it means that mobile code is not viable on the Java platform today. At least not across organisational borders. But I think there is a more fundamental reason for these practical issues. It is very hard indeed to make something like J2SE forward compatible. Forward compatibility is hard with data formats and almost impossible with large collections of procedural code. And it is even harder in statically compiled languages.

> > But even with something as
> > simple as Java Applets, it hasn't worked and there are
> > constant troubles with JVM versions. Many people say
> they
> > always supply a JVM with their Java app for exactly
> those
> > versioning and management reasons.
>
> We do try and manage the version of the JVM that we use
> because typically, we change the jre/lib/ext content and
> don't want to contaminate another JVM.
>
> The quality of the JVM releases can be a big issue for
> some types of applications. The amount of software that
> one depends on being "right" is quite large. But, if you
> are not willing to buy into a support contract, then you
> will have to deal with problems with work arounds or wait
> until fixes arrive. With contracts, you can get help and
> fixes fairly quickly.

All that doesn't sound like something that scales in terms of coordination and management overhead.

> > The problem I see with default serialisation is that it
> > causes version breaks even when there are no logical
> > problems. For example, if you change the order of
> instance
> > variables in the code. The addition of an optional
> > instance variable shouldn't cause much trouble really,
> but
> > it does if you put it in the wrong place lexically,
> even
> > though the order of instance variables is insignificant
> in
> > the Java language. This is a bad mixup of logical and
> low
> > level representational concerns.
>
> Humm, I've never had this problem. I just quickly checked
> JDK1.5 and JDK1.4 and neither seems to have this
> behavior.

OK, you're right, changing the order alone is not enough break it. But the point is that there is no separation between logical and purely technical changes that shouldn't cause version breakage. The default serialization leaks implementation details. You cannot change the private implementation without risking to break compatibility, which is the whole point of encapsulation. I think to make it work in a robust way, you have to manage the serialization format, one way or another.


> I've been using RMI for rpc and integration since jdk1.3
> (I used Voyager in JDK1.X, and never used JDK1.2 due to
> its bugs). I've never had a JVM dependency problem.
> Perhaps my usage patterns eluded the issues that you have
> e experience with?

The question is simply whether you are completely in charge of the deployment environment and whether you are integrating your own applications. I don't call it integration when I simply want to make two of my own applications maybe even client and server of one application talk to each other. That's not integration. Integration is making applications written by different people, without central coordination, talk to each other and share information. I cannot download mobile Java code into other people's applications. Even if all applications on the planet were written in Java, you'd need to rewrite them all based on a completely different architecure (maybe like Jini?).

I think we are talking about very different kinds of scenarios.

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Going all in... Posted: May 18, 2005 4:07 PM
Reply to this message Reply
> There are static constraints and there are dynamic
> constraints. As a user of an interface, I want to know
> what the static constraints are. Not just for static
> validation and local testing, but also because it enables
> me to give users feedback as early as possible. It makes
> no sense to let a user type in 50 characters in a field
> where only 20 are allowed. I don't want to throw test
> messages at a web service to find out what number range I
> am allowed to put into some field.

O.K. I agree with your taxonomy and see Your point. But I'm not shure if You misrepresent my attempt to communicate contracts via testcases. Here is a living example, though not in Java:

http://docs.python.org/lib/module-doctest.html

Regards,
Kay

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: Going all in... Posted: May 18, 2005 8:49 PM
Reply to this message Reply
> Exactly, and that's why passing well modeled data around
> is preferable whenever there is no central control over
> deployment, versioning, etc or when it simply gets too
> complex to manage and you need more flexiblity or when you
> want to build systems in a way that is amenable to
> integration with other systems in the future.

I don't consider the structure of that data any different than
the serializable form of a Java Object. In either case, you
can be in control, completely.

In the case of XML, you are locked into no dynamic granularity
control that mobile code gives you. Once you decide on XML,
changing to something different is pretty much impossible
because every single user is dependant on that format. If
you need to compress the stream, everyone has to take that
into account. With mobile code, a smart proxy can be used
to do that, after the fact, and no code has to change.

In the end, I think its a matter of do you really think that
using a single language is a problem, or a solution. And,
I mean language, not platform. J2EE is not a solution, it
is a problem. It creates too many limitations on exploiting
the power of the Java platform. It makes easy things easier,
and hard things harder (thread management), or impossible
(class loading and proxy management).

> If I need to manage serialisation and use all these
> mapping tools like JAXB, I find it more complex than plain
> XML/XSLT. But that may be a matter of taste to some
> degree.

I don't use XML for serialization if that was not clear...

> Yes, more or less. The important thing is that they can
> agree on a meta model that allows you to define which
> particular parts of the data must be understood, which
> ones are respected if supplied, how other parts are
> ignored, and so on. If done right, there can be a lot of
> flexibility in that agreement.

Yes, this is possible, but look at all the places that
those agreements can break either on purpose of by
oversight. All of this procedural based contract
enforcement is not a very safe way to manage important
contracts.

> > Doesn't that require the server
> > to then be in charge of making the clients change at the
> > right moment to use a new format of data?
>
> That depends on what kind of change you're talking about
> and how that client integrates with other software. Of
> course, if the server makes a breaking change, and demands
> that all clients adhere to the new rules or are simply
> denied access, then it would be helpful to have a
> convenient way of shipping the updated code to the client.
> That's a very nice feature for closely coupled systems.
> Interestingly, it never worked outside of simple UI bits
> and pieces.

This is working quite well in many Jini based applications.
Mobile code is heavily exploited. The Jini Technology
Starter Kit provides services that use the RMI programming
model. People really can just start those services and
use the downloaded code that they export for their
codebases. It works everywhere that the underlying
technologies (networking such as multicast, open ports
for service use etc) work.

> I know RMI, but the whole thing just isn't robust enough.
> Just look at some of the mailing lists that discuss
> deployment issues in J2EE applications.

J2EE is not at all a place to analyse and understand the
RMI programming model. The app servers do not have
support for mobile code included as part of their design.

> Just to make
> client program work with more than one type of application
> server can be troublesome. They compile proxies
> differently or not at all. They have particular ways of
> deploying JNDI drivers. They depend on various helper
> libraries or particular third party library versions. Some
> support JDK 1.3 while other already run on 5. There's huge
> number of security issues and so on.

In Jini2.0, there are several tools which solve a lot of
problems.

o Dynamic Proxy support in JDK1.4 and later, no RMIC.
o DynamicPolicy support to handle dynamic security needs.
o Kerberos and X.5XX transport for authentication and
mutual identity establishment.
o A plugable transport stack (JERI) that allows you to
talk whatever wire protocol you want so that you
can plug into SOAP or some other invocation layer.

> I totally agree that
> these are not fundamental logical problems at all and one
> should think that there's a better way to do it. But
> again, this industry has been trying to do it for many
> years and so far it always failed. Even if you limit
> interop to the Java only.

I guess that one industry may be having problems. I hear
alot of horror stories from J2EE users. I've heard of many
large Jini applications. Secure interworking was not
possible until the security layer added in Jini2.0. It's
still possible for trusted code that is downloaded to do
things in your JVM that you might not find acceptable.
It is now possible to validate the source and version
of the downloaded code as well as the calling party for
initial deserialization attack protection.

> > In Jini2.0, we have things in the JERI stack and in the
> > APIs that simplify many issues. The HTTPMD protocol
> > includes and MD5 checksum in the URL structure. This
> > means that a client can grant permissions to a particular
> > version of the code to isolate itself from unexpected
> > changes. It also means that you'll get a new classloader
> > when the jar changes so that the refering code won't be
> > subject to URL caching that might make it use the old
> > code.
>
> I have to admit that I don't know Jini very well. Maybe it
> has better solutions to some of the problems with N:M
> third party library versioning, JVM versions, etc. But
> still, it is a purely Java thing and integration between
> applications that weren't designed to interoperate is hard
> if everything is hidden behind APIs and accessible only on
> a single runtime platform.

I think we'll have to agree to disagree. Integration only
gets more difficult because we continue to diversify on the
runtime environment.

> > > Unless of course you manage that as well, so your service
> > > gives the app a different proxy than some other service.
> > > Also, you can't communicate with anything not written in
> > > Java and you can't do something like switch to J2SE 5
> > > unless all your clients have switch their runtime as
> > > well.
> >
> > I don't have problems serializing data between JDK 1.5 and
> > JDK 1.4 when JDK1.4 only features are used...
> > It is true
> > that you can't download JDK1.5 specific code into a JDK1.4
> > VM. That's a technical issue with implementation, not a
> > practicle issue with downloaded code in general.

> Well, first of all it means that mobile code is not viable
> on the Java platform today. At least not across
> organisational borders. But I think there is a more
> fundamental reason for these practical issues. It is very
> hard indeed to make something like J2SE forward
> compatible. Forward compatibility is hard with data
> formats and almost impossible with large collections of
> procedural code. And it is even harder in statically
> compiled languages.

I guess I'm not seeing the difference. Bytes of data flowing
over the network, no matter what the format have to arrive
in some predictable representation. Regardless of that
structure, changes in the format are problematic from a
general perspective. Java Serialization takes care of
announcing incompatibility quite literally through
serialVersionUID. An XML document could do this with a
DTD assertion. But, most people don't seem to be doing
this based on what I've seen/heard.

In either the case of serialized Java Objects or XML you
can, by mistake, break the format of the data for other
users. There is no protection against this. So, I don't
see how a printable text representation of "data" is any
better than a binary representation. In either case it
can either be right or wrong, usuable or unusable.

With Serialization, you can version the objects and manage
changes in the structure of the transported data. You may
not have ever implemented such support in your serialization
schemes, but it is possible. You can do

public void writeObject( ObjectOutputStream os ) {
os.writeInt(version);
os.writeInt(val1);
os.writeObject(val2);

// Everything after here must be in Object form so that
// StreamEnd search works...

os.writeObject(new Float(val3)); // added in version 2
os.writeObject(val4); // added in version 3

os.writeObject(new StreamEnd());
}

public void readObject( ObjectInputStream is ) {
int ver = is.readInt();
val1 = is.readInt();
val2 = (SomeCast)is.readObject();
if( ver >= 2 ) {
val3 = ((Float)is.readObject()).floatValue();
} else {
val3 = 0;
}
if( ver >= 3 ) {
val4 = (SomeObject)is.readObject();
} else {
val4 = new SomeObject();
}

while( (is.readObject() instanceof StreamEnd) == false );
}

This creates forward and backward compatibility if that is what
you need. Old versions of the software don't have to care about
the new data items. This might not be necessary in cases where
you use a downloaded factory for creating new objects, instead
of instantiating then literaly. I.e. the interface is the
visible type, not a particular object class. Then, every new
object in the system would always be the most recent version.
If you use MarshalledObject to store/transport serialized values,
then the included codebase could be updated to include the
newest implementation and every object that gets unmarshalled
would be reinstantiated with the last implementation.

It's the whole business of correctly manage the serialized
objects lifecycles that needs to be correctly designed.

> OK, you're right, changing the order alone is not enough
> break it. But the point is that there is no separation
> between logical and purely technical changes that
> shouldn't cause version breakage. The default
> serialization leaks implementation details. You cannot
> change the private implementation without risking to break
> compatibility, which is the whole point of encapsulation.
> I think to make it work in a robust way, you have to
> manage the serialization format, one way or another.

The serialVersionUID changes when you add an instance
variable, add a method, or change the signature of an
existing method. If you miagrate a serialized object
in your system design, you should, at least:

o set serialVersionUID to a fixed value at the start of
the life of the object, and don't change it, ever.
o Use readObject/writeObject to manage the changes you
make over time which are backward compatible changes.
o Use a version number in the serialization stream that
will identify which version of the stream is being
read.

> > I've been using RMI for rpc and integration since jdk1.3
> > (I used Voyager in JDK1.X, and never used JDK1.2 due to
> > its bugs). I've never had a JVM dependency problem.
> > Perhaps my usage patterns eluded the issues that you have
> > e experience with?
>
> The question is simply whether you are completely in
> charge of the deployment environment and whether you are
> integrating your own applications.

I don't have a problem with premise of this statement.
However I do think its important to identify what makes
integration possible. It's only through common use and
knowledge of some underlying transport and data formatting
that you can make two disparately located systems talk
to each other. XML is one data format and HTTP is one
transport technology. Neither has any technical advantage
over other documented data formats and transport technologies.

Practically, there are toolsets and hardware technologies
that provide assistance to XML deployments in some situations.
It seems like this will likely be more common rather than
less common in the future. ESB is another layer that is said
to be needed because of features that don't exist in the
area of XML based services.

It is the mere existance of these external supporting
items that says the technology in and of itself is not
an complete solution. There are lots of problems still
to solve on many fronts with many different integration
technologies. The question for me is are those
problems making things possible, or are they making
things seem easier, but actually coupling applications
more directly to the limits of the technology.

> I don't call it
> integration when I simply want to make two of my own
> applications maybe even client and server of one
> application talk to each other. That's not integration.
> Integration is making applications written by different
> people, without central coordination, talk to each other
> and share information. I cannot download mobile Java code
> into other people's applications.

Integration, to me, is taking previously disparate
applications and making them work together to create
a new application that provides something previously
not possible. If I get to do this locally, that isn't
necessarily any easier than if I have to do it with
someone elses code.

> Even if all applications
> on the planet were written in Java, you'd need to rewrite
> them all based on a completely different architecure
> (maybe like Jini?).

I think this is the ultimate question. Is it easier and
cheaper, in the long run, to continue supporting multiple
languages and multiple platforms, or is a single platform
and/or language a better choice?

I believe that software vendors are moving toward the
one platform notion. Microsoft's CLR provides a single
set of primatives expressed in multiple languages, which
are restricted by the intersection of the features of
those languages. The Java language and the J2?E platforms
provide a fairly common set of features with J2EE and J2ME
restricting what J2SE features you can use to provide
a container that is targeted at solutions for specific
types of problem solving.

Sun has met with the dynamic language creators for Perl,
Python, and others to see what they might do to the JVM
to allow it to support these languages natively.

Unification is happening. There is value in it, much
beyond the simple comforts that "choice" brings to some
people who want to make decisions independently.

I think that if you don't see this coming, you might
be horribly surprised when your competition is suddenly
based on a highly portable and featureful platform that
provides them a lot more choices than your choice does
you.

> I think we are talking about very different kinds of
> scenarios.

Perhaps. I sense a lot of J2EE and similar notions in your thoughts and comments. I've stayed away from that market. There are perhaps something like 10,000 large businesses that need b2b J2EE based technologies. There are hundreds of millions of people that need mobile desktops with security and flexibility, home integration and other similar
solutions. Those are the interesting scenarios for me! Providing a common, flexible base with portability as the primary emphasis is one of the dominating design principals.

Alexander Jerusalem

Posts: 36
Nickname: ajeru
Registered: Mar, 2003

Re: Going all in... Posted: May 19, 2005 5:51 AM
Reply to this message Reply
> > Exactly, and that's why passing well modeled data
> around
> > is preferable whenever there is no central control over
> > deployment, versioning, etc or when it simply gets too
> > complex to manage and you need more flexiblity or when
> you
> > want to build systems in a way that is amenable to
> > integration with other systems in the future.
>
> I don't consider the structure of that data any different
> than
> the serializable form of a Java Object. In either case,
> you
> can be in control, completely.


We're talking in circles. I never said you couldn't be in control of serialisation. But when you are in control you need to have the right tools and abstractions to describe and manage the wire representation. Java and RMI don't help me with that in any way, whereas XML web services provide a set of languages and rules for exactly this purpose.


> In the case of XML, you are locked into no dynamic
> granularity
> control that mobile code gives you. Once you decide on
> XML,
> changing to something different is pretty much impossible
> because every single user is dependant on that format.

Well, let's be clear: No matter what technology you use, there will always be something you commit to, that you cannot simply rip out and replace. The question is how broad this commitment is. It's no different when you invent your very own serialisation format. The reason why XML is so popular for interop is exactly because the commitment is very small. It doesn't determine the programming language, the runtime or particular runtime versions and it carries a lot of metadata which makes versioning very flexible and granular. It doesn't even mandate a particular programming style like RPC or OO, you can do messaging as well, store and forward, etc. It's easy for any system to send and receive text over some network. Much easier than to embed a Java virtual machine and comply with the fluffy notion of an "interface" that Java offers.

> If
> you need to compress the stream, everyone has to take
> that
> into account.

No, not at all. It's a question of protocol design to have a way of negotiating the capabilities of sender and receiver. HTTP does exactly that with its various accept headers.

> > Yes, more or less. The important thing is that they can
> > agree on a meta model that allows you to define which
> > particular parts of the data must be understood, which
> > ones are respected if supplied, how other parts are
> > ignored, and so on. If done right, there can be a lot
> of
> > flexibility in that agreement.
>
> Yes, this is possible, but look at all the places that
> those agreements can break either on purpose of by
> oversight. All of this procedural based contract
> enforcement is not a very safe way to manage important
> contracts.

How is this "procedural based"? It's a declarative formal contract. You can mandate and describe a lot of things that you cannot describe formally in a Java interface.

> In Jini2.0, there are several tools which solve a lot of
> problems.
>
> o Dynamic Proxy support in JDK1.4 and later, no RMIC.
> o DynamicPolicy support to handle dynamic security
> needs.
> o Kerberos and X.5XX transport for authentication and
> mutual identity establishment.
> o A plugable transport stack (JERI) that allows you to
> talk whatever wire protocol you want so that you
> can plug into SOAP or some other invocation layer.
>
> > I totally agree that
> > these are not fundamental logical problems at all and
> one
> > should think that there's a better way to do it. But
> > again, this industry has been trying to do it for many
> > years and so far it always failed. Even if you limit
> > interop to the Java only.
>
> I guess that one industry may be having problems. I hear
> alot of horror stories from J2EE users. I've heard of
> many
> large Jini applications. Secure interworking was not
> possible until the security layer added in Jini2.0. It's
> still possible for trusted code that is downloaded to do
> things in your JVM that you might not find acceptable.
> It is now possible to validate the source and version
> of the downloaded code as well as the calling party for
> initial deserialization attack protection.

So does Jini find out dynamically which J2SE versions and which versions of particular third party libraries are present in a particular deployment and download the rest? And is this done in a way that doesn't conflict with the set of libraries and J2SE versions deployed by a different service provider my systems relies on?

> I think we'll have to agree to disagree. Integration
> only
> gets more difficult because we continue to diversify on
> the
> runtime environment.

The only way to solve integration problems is by reducing coupling. You will never be able to mandate a completely homogenous environment. And that's not just because one group uses Java and another one .NET and yet others Perl, PHP, Oracle PL/SQL, etc. It's also because each particular platform and each particular application has its own versioning cycles. You can't force them all into lockstep. It's not a matter of choice whether we have diverse IT environments, it's just a fact of life.

> > Well, first of all it means that mobile code is not
> viable
> > on the Java platform today. At least not across
> > organisational borders. But I think there is a more
> > fundamental reason for these practical issues. It is
> very
> > hard indeed to make something like J2SE forward
> > compatible. Forward compatibility is hard with data
> > formats and almost impossible with large collections of
> > procedural code. And it is even harder in statically
> > compiled languages.
>
> I guess I'm not seeing the difference. Bytes of data
> flowing
> over the network, no matter what the format have to
> arrive
> in some predictable representation. Regardless of that
> structure, changes in the format are problematic from a
> general perspective.


No, I'm afraid you're totally wrong in this case. Different representations have different logical properties. The semantics of procedural code can only be established by means of flow analysis, which is infinitely harder than dealing with declarative data or rules. It is quite often logically impossible to find out if two pieces of procedural code have the same effect for the same input, without running it. And running it, is exactly what you cannot do before finding out if you have compatible version or not.

> Java Serialization takes care of
> announcing incompatibility quite literally through
> serialVersionUID. An XML document could do this with a
> DTD assertion. But, most people don't seem to be doing
> this based on what I've seen/heard.

That's because XML has more granular ways of dealing with versioning than just success or fail based on a version flag.

> In either the case of serialized Java Objects or XML you
> can, by mistake, break the format of the data for other
> users. There is no protection against this. So, I don't
> see how a printable text representation of "data" is any
> better than a binary representation. In either case it
> can either be right or wrong, usuable or unusable.

It's not about text versus binary, that's another debate. But as I said, different representations have different logical properties. One representation may allow me to make declarative assertions about what I require, what I recognise, what I reject and another representation wouldn't allow me to do that. You can't just ignore the logic and think of everything in terms of blobs. There are some properties you really want to have in a representation. One is that you need to be able to judge the semantic equality of two lexically different expressions. So you need a deterministic algorithm that produces a canonical representation that can be compared. If you encode rules in a sequence of procedural steps and you have variables that are manipulated along the way, you are very likely to end up in a situation where canonicalisation is logically impossible.

And since Java's type system is so weak, you have to write procedural rules for things that you assert declaratively in other languages like XML Schema or RELAX-NG or OWL. The consequence is that Java code cannot be analysed for version compatibility. You have to set that "make or break" compatibility flag (serialVersionUID) manually for the whole class. There's no way of judging a particular incoming request based on the data that it carries and decide if you can deal with it or not. And there is no way of communicating these rules to others so they can automatically validate their messages or implementations. But that's exactly what you need in a loosely coupled system.

> With Serialization, you can version the objects and
> manage
> changes in the structure of the transported data. You
> may
> not have ever implemented such support in your
> serialization
> schemes, but it is possible. You can do [...]

I know that, and I said many times, that you can manage your serialisation format. But Java doesn't help you to do that. There is no schema language to declaratively state rules about that format and there is no standard for describing the technical metadata of a service in a way similar to SOAP/WSDL. You can invent your own serialisation format and your own processing rules and so forth, and then you communicate them to each and every partner you want to interact with.

But you know, I see a contradiction in your argument: On the one hand you stress that you can manage serialisation, but on the other you tell me that it is not necessary to do so, because you can send mobile code everywhere anyway. The thing is this: If you manage the wire format, you do it because _other_ people depend on it and _they_ have to manage their side of the system. You are absolutely right, that this is not necessary if their side doesn't depend on the wire format because you send them a proxy that manages communications to your service.

> It's the whole business of correctly manage the
> serialized
> objects lifecycles that needs to be correctly designed.

Sure, I don't deny the value of this at all. You just have to recognise what it can and cannot do with blobs of downloadable code. It has logical limits (as I explained above) and above all, it has organisational limits because you mandate a homogenous IT environment that you rarely have.

> > OK, you're right, changing the order alone is not
> enough
> > break it. But the point is that there is no separation
> > between logical and purely technical changes that
> > shouldn't cause version breakage. The default
> > serialization leaks implementation details. You cannot
> > change the private implementation without risking to
> break
> > compatibility, which is the whole point of
> encapsulation.
> > I think to make it work in a robust way, you have to
> > manage the serialization format, one way or another.
>
> The serialVersionUID changes when you add an instance
> variable, add a method, or change the signature of an
> existing method.

Sure, but the problem is that you have no clear separation of interface and implementation. If you change a data structure inside your implementation, you break the serial version without there being a logical necessity. That makes the whole system brittle. The only way of managing it is putting only relatively simple value objects in your interfaces, but then you lose any benefit of sending Java objecs in the first place.

> > > I've been using RMI for rpc and integration since
> jdk1.3
> > > (I used Voyager in JDK1.X, and never used JDK1.2 due
> to
> > > its bugs). I've never had a JVM dependency problem.
> > > Perhaps my usage patterns eluded the issues that you
> have
> > > e experience with?
> >
> > The question is simply whether you are completely in
> > charge of the deployment environment and whether you
> are
> > integrating your own applications.
>
> I don't have a problem with premise of this statement.
> However I do think its important to identify what makes
> integration possible. It's only through common use and
> knowledge of some underlying transport and data
> formatting
> that you can make two disparately located systems talk
> to each other. XML is one data format and HTTP is one
> transport technology. Neither has any technical
> advantage
> over other documented data formats and transport
> technologies.

I think I have expressed my opinion about the advantages of data based integration. To sum it up: Uncoupling of interface contract and implementation. Uncoupling of runtime environments and their particular versions. Flexible handling of message schema evolution based on widely used standards.

> Practically, there are toolsets and hardware technologies
> that provide assistance to XML deployments in some
> situations.
> It seems like this will likely be more common rather than
> less common in the future. ESB is another layer that is
> said
> to be needed because of features that don't exist in the
> area of XML based services.

So far I wasn't aware that ESB is anything more than a marketing term. What are the characteristics of ESB that contradict data based protocols?

> > I don't call it
> > integration when I simply want to make two of my own
> > applications maybe even client and server of one
> > application talk to each other. That's not integration.
> > Integration is making applications written by different
> > people, without central coordination, talk to each
> other
> > and share information. I cannot download mobile Java
> code
> > into other people's applications.
>
> Integration, to me, is taking previously disparate
> applications and making them work together to create
> a new application that provides something previously
> not possible. If I get to do this locally, that isn't
> necessarily any easier than if I have to do it with
> someone elses code.

Of course it is much easier if you can just go in and rewrite parts of the application in a language/runtime of your choice than having to integrate applications written by others.

> > Even if all applications
> > on the planet were written in Java, you'd need to
> rewrite
> > them all based on a completely different architecure
> > (maybe like Jini?).
>
> I think this is the ultimate question. Is it easier and
> cheaper, in the long run, to continue supporting multiple
> languages and multiple platforms, or is a single platform
> and/or language a better choice?

This is one of the important questions, yes. Another one would be the logic of describing interfaces, separating interface contracts from implementations, and if it is advisable to hide business rules in procedural code. But the problem with your question is, in my opinion, that no matter how you answer it, the world will not comply.

I would even think that it'd be horrible if it would comply because there would not be any progress. When I think back to 1996, when I first used Java in a large scale project, the context was something like AS400, RPG, VB, VC++/MFC/COM, Tuxedo, Delphi, Oracle PL/SQL, just to name a few of the things that our prospective customers were using. There was no homogenous environment and if we had decided we wanted to deploy mobile code into these systems, we would've had to hire an army, topple the government, nationalise all the banks and force them at gun point to reimplement everything in Java.

> I believe that software vendors are moving toward the
> one platform notion. Microsoft's CLR provides a single
> set of primatives expressed in multiple languages, which
> are restricted by the intersection of the features of
> those languages. The Java language and the J2?E
> platforms
> provide a fairly common set of features with J2EE and
> J2ME
> restricting what J2SE features you can use to provide
> a container that is targeted at solutions for specific
> types of problem solving.
>
> Sun has met with the dynamic language creators for Perl,
> Python, and others to see what they might do to the JVM
> to allow it to support these languages natively.
>
> Unification is happening. There is value in it, much
> beyond the simple comforts that "choice" brings to some
> people who want to make decisions independently.
>
> I think that if you don't see this coming, you might
> be horribly surprised when your competition is suddenly
> based on a highly portable and featureful platform that
> provides them a lot more choices than your choice does
> you.

I agree with you that more and more code is built on top of hardware platform independent virtual machines. But as in the past, there are a number of them, some linked to a particular OS, and over time you have to deal with a large number of versions of each of them. You have at least 4 technology stacks right now that don't run each other's code and I don't see them converging any time soon. But this is not only about compatibility on VM level, it's also about the frameworks on top of them. An EJB doesn't run as a Jini service or vice versa (without jumping through some hoops). Every day new frameworks come along with their particular APIs and depending on what services a piece of code needs, it has to make use of these APIs to some degree. Yes we are somewhat less hardware dependent than 10 years ago, but overall there is more heterogeneity not less.

Mike Petry

Posts: 34
Nickname: mikepetry
Registered: Apr, 2005

Re: Going all in... Posted: Aug 16, 2005 6:47 PM
Reply to this message Reply
Cool, it looks like I have discovered evidence of a holy war! This stuff really is holy war material. It seems like Jini is (or could be construed to be) diametrically opposed to the document-centric, SOA, web serviced approach to distributed systems.
I tend to agree with the SOA approach, use XML to serialise, and marshall your data. This creates a very, very simple way to distribute functionality, and simple always wins.
However, maybe I just don't grok Jini. I just started to think about it and maybe I am missing the point.
Have you noticed that the waters around the IT industry have become becalmed? I don't even think that software development even can be termed as "high-tech" anymore. Its just the same old stuff. Store data, read data, update data, add data, delete data, display data.
I sense revolution is standing around the corner and I am not sure how it will form but I think it has something to do with lots of networked computers.
Maybe Jini is one of those technologies that every one will go "ahhh thats what they where try to do" 5 years down the road.

Flat View: This topic has 22 replies on 2 pages [ « | 1  2 ]
Topic: When it comes to Computer Science, don't reference Wikipedia Previous Topic   Next Topic Topic: High Performance Dynamic Typing in C++


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us