This article describes a solution to the problem of building a highly available and scalable system that is not only inexpensive to build, but requires minimal ongoing system administration and maintenance

http://www.artima.com/lejava/articles/dynamic_clustering.html

What do you think of Boomi's approach to clustering?

More important, Boomi's customers are mid-market and smaller firms that lack in-house technical knowledge to operate a full-fledged J2EE environment.

Bogus. How is a "full-fledged J2EE environment" (i.e. one process, one shell script) any more difficult than a "full-fledged Jini environment" (i.e. typically a half dozen processes and a half dozen shell scripts)? Do you think that the Glassfish team would agree with this assertion?

Plus, with several free and open source J2EE application servers coming with dynamic clustering built-in, what is the business model behind building all this stuff (e.g. their custom JMS replacement) from scratch for each application?

I really don't understand why Sun is sponsoring an article that steers people away from using the J2EE standard, and instead steers them toward writing a whole bunch of low-level wiring themselves.

Peace.

Our product is not a development platform like a J2EE application server. We do not develop individual applications per customer, we ship a product with a set of pre-built Jini services that are configured as required by the user. Our typical end user is not a Java developer nor a person who has Java-related experience.

This article is not meant to steer people away from J2EE. It is just introducing an approach to building a clustered system using Jini technology. The goal of the design is to abstract away the clustering technology, while still utilizing various J2EE technologies such as Servlets, XML, etc. without incurring the overhead of developing within the EJB 2.0 (or EJB 3.0) specification. Jini provides us with the "low level wiring" to achieve this, without needing to build clustering technology ourselves.

The "Why not J2EE?" was posed to us because of the obvious clustering abilities of various J2EE application services. We were looking for a pure software clustering technology that would fit in with our Service Oriented Architecture. While we did look at J2EE, the Jini Technology combined with the Rio Dynamic Provisioning framework allowed us to develop the application without being constrained by the EJB Container specification. When doing the research, we realized that the two technologies can be complementary to each other. A relevant analogy perhaps would be Macromedia's JRun and their use of Jini technology for their J2EE application server (http://www.artima.com/intv/cluster4.html). Jini technology plays a role in JRun's J2EE server and also plays a similar role in our Integration server. Just as the JRun administrator typically has no knowledge of the underlying Jini technology, a typical Boomi administrator has no knowledge of Jini technology existing inside our server. Coincidentally, the JRun article describes a mechanism to reduce the complexity of starting and maintaining a Jini container without the need for multiple scripts.

We are in the Business Integration market along with every major J2EE vendor. Nearly all J2EE vendors provide an integration stack and/or middleware that can be used to develop and integrate different applications. However, while the J2EE vendor provides a "development framework" solution where components are developed/deployed by a J2EE programmer, our product requires no previous Java experience or development knowledge. Consequently, this allows us to market our product to a larger base of customers who might not have an in-house development staff.

"(i.e. typically a half dozen processes and a half dozen shell scripts)"

I think that needs clarification.......

That may have been true once upon a time and some people still do it this way but you don't need to. There are several containers that make this all go away and since JINI 2.0 it's also possible to bring up your entire JINI infrastructure in one process with one script.

> "(i.e. typically a half dozen processes and a half dozen
> shell scripts)"
>
> I think that needs clarification.......
>
> That may have been true once upon a time and some people
> still do it this way but you don't need to. There are
> several containers that make this all go away and since
> JINI 2.0 it's also possible to bring up your entire JINI
> infrastructure in one process with one script.


Re: the single script startup thing, I meant to say that Blitz's Installer (http://www.dancres.org/blitz/blitz_inst.html) provides exactly this configuration (amongst others). i.e. You can start up codebase, lookup, transaction manager and javaspace all in one JVM with one script.

> "(i.e. typically a half dozen processes and a half dozen
> shell scripts)"
>
> I think that needs clarification.......
>
> That may have been true once upon a time and some people
> still do it this way but you don't need to. There are
> several containers that make this all go away and since
> JINI 2.0 it's also possible to bring up your entire JINI
> infrastructure in one process with one script.

Dan, you're right, I was exaggerating to make a point. (Maybe I should have added a grin. ;-) The general perception of J2EE development and the administration of J2EE servers espoused by Jini developers is that it is hopelessly complex and impossible to use. I used the "six processes" analogy because it is one that has been used by people who last tried using Jini back in the same epoch that J2EE was a complete mess to administer. That Jini has become possible to use with a PhD is not surprising (things improve over time!), but it's not going to help the perception of Jini to belittle the platform (J2EE) that businesses actually use to get things done.

The truth is that most "packaged" (i.e. not internal) J2EE applications, if they are packaged well, are so simple to administer that a receptionist can do 99% of the administration. I should know, because our non-technical receptionist used to administer a half dozen Java packaged apps that we used ourselves.

Fair enough? ;-)

Peace.

> but it's not going to help
> the perception of Jini to belittle the platform (J2EE)
> that businesses actually use to get things done.

I don't think anyone is belittling the J2EE platform. Like Jini, the J2EE platform has improved greatly since we originally visited this decision and I don't necessarily have the expertise on the J2EE platform today to claim whether it's easy or hard to develop or administer. But what I can do is describe the thoughts that went into the decision to not use J2EE when the system was originally designed.

I think the decisions that were described in the article have to be placed in the proper time context in order to be understood. When the system was originally designed, J2EE 1.3 was out on the market and J2EE 1.4 was about to be released as a final spec. The big players in the market were BEA and IBM (in that order) who commanded large prices for their technology. JBoss was available, but had not gained the market share that it claims today. Since then we've seen new releases of Java and a new release of J2EE (5.0) that was aimed towards making development easier and a large growth of the free and open source application server market.

Bottom line, the article is describing how we leverage Jini Technology to achieve a dynamic clustering system and I don't think the goal of it was to attack the J2EE platform as a horribly complex platform that is impossible to administer (which it is not). Like you said, the same thing could be said of Jini Technology prior to the improvements that have been added in 2.0.

Mitchell,

Your original response was very helpful. I'd really like to see the application sometime.

Peace.

Hi there,

Is your application composed only of stateless services? Jini is quite handy to cluster these, but any ideas about clustering statefull ones?

Cheers,
Piotr

> Hi there,
>
> Is your application composed only of stateless services?
> Jini is quite handy to cluster these, but any ideas about
> clustering statefull ones?
>
> Cheers,
> Piotr

Have to be pedantic here - do you mean clustering for horizontal scaling or for reliability or both?

What kind of state? Does it all have primary keys? Is it POJO's, big objects or small?

Any clustered solution is quite specific to the applications requirements and whilst there are certain levels of generality that mean one can provide software frameworks and appropriate hardware, the combination required varies.

Lastly whilst many a developer has a tendency toward software-only solutions, mixed software/hardware solutions can often be cheaper, more effective and easier to maintain.

> > Is your application composed only of stateless services?
> > Jini is quite handy to cluster these, but any ideas about
> > clustering statefull ones?

> Have to be pedantic here - do you mean clustering for
> horizontal scaling or for reliability or both?

Nowadays it seems common to expect both aspects to be addressed, and more. I recently wrote a piece describing how clustering state can help to address the aspects of availability, reliability, scalability, performance, serviceability and manageability:

http://wiki.tangosol.com/display/COH31UG/Cluster+your+objects+and+your+data

The concepts should sound familiar ;-)

> whilst many a developer has a tendency toward
> software-only solutions, mixed software/hardware solutions
> can often be cheaper, more effective and easier to
> maintain.

I haven't seen this, other than in some specific database environments related to dual-ported disks. Having said that, you're already into the big bucks even with dual-ported disks, which are considered just about the cheapest hardware option ;-)

> What kind of state? Does it all have primary keys? Is it
> POJO's, big objects or small?

What I've witnessed is that applications architected for distributed shared state will tend toward using explicit primary keys and small POJOs, and applications retrofitted for distributed shared state will tend to have larger object graphs and fewer natural identities.

Peace,

Cameron Purdy
http://www.tangosol.com/

Please note that the original article described quite a generic, software-based solution. Thus my generic questions :)
I'm not looking for any ready-made solution, only to satisfy my curiosity.

> Lastly whilst many a developer has a tendency toward
> software-only solutions, mixed software/hardware solutions
> can often be cheaper, more effective and easier to
> maintain

Hmm. That means actually 'no' for generic, software-based clustering functionalities on JINI. Doesn't it?

Cheers,
Piotr

> > Lastly whilst many a developer has a tendency toward
> > software-only solutions, mixed software/hardware
> solutions
> > can often be cheaper, more effective and easier to
> > maintain
>
> Hmm. That means actually 'no' for generic, software-based
> clustering functionalities on JINI. Doesn't it?

If you were to create a software only solution, the fastest and easiest would be to use a transaction and a smart proxy to perform parallel writes to duplicate services, and to then distribute reads across multiple services in a load balanced way, failing to the next when one is not accessible.

This would be about 50-100 lines of code in a smart proxy for the simplest cases, and perhaps twice that many lines of code in a more complicated environment.

However, creating and validating the actions of the transaction participants would be the hard part. That is what Dan is refering to. In this day and age, persistent data on disk is the most commonly managed transactional state.

Making the hardware do the hard work, at the bit managed level, where it has no knowledge of the application, programming platform etc, is typically much cheaper, except when your developers work for free.