Summary
The Shoal project released this week an open-source JXTA-based clustering framework. Shoal can be plugged into any product to provide cluster-based fault tolerance and availability. A unique feature of Shoal is its ability to employ any group communication toolkit in a cluster via a service provider API.
Advertisement
Shoal is a JXTA-based project, recently made available on java.net, that provides a plug-in clustering infrastructure to any project that wishes to scale to a collection of servers. According to Mohamed Abdelaziz, a Sun engineer and senior architect of Sun's JXTA framework:
The main goal for the shoal project [is] to provide a clustering framework for the GlassFish project, however it does not preclude it from being used for purposes, such as the foundation for a grid deployment, or a deployment of a highly available service...
Shoal/GMS utilizes the JxtaManagement component, a JXTA based group service provider, for dynamic cluster configuration, formation, and monitoring.
Shoal facilitates cluster communication with two public APIs. The client API communicates cluster events between individual cluster members, while the group communication provider API lets you plug in any group communication protocol into the cluster. Such a group communication protocol may be JXTA's own group communication mechanism, using JXTA peer groups, or even the popular JGroups framework.
According to Abdelaziz, on top of group communications and the client API, Shoal's cluster management is provided by five components:
The NetworkManager uses a SHA-1 hash to encode the cluster GroupID, and NodeID, [and] in addition it also defines a set of predefined communication identifiers which are used for formation, monitoring and messaging...
The SystemAdvertisement [component is] an extensible XML document describing system characteristics, [such as] hardware and software configuration... This information is exchanged during cluster formation, and monitoring...
MasterNode is a lightweight protocol allowing a set of nodes to discover one another, and autonomously elect a master for the cluster. The protocol is resilient to multi node collisions and employs an autonomous mechanism to avoid network chatter to resolve collisions.
ClusterView maintains an ordered view of the Cluster.
HealthMonitor is a lightweight protocol allowing a set of nodes to monitor the health of a cluster. HealthMonitor relies on a tunable heart beat, which is acted upon by the MasterNode to notify the group of failures, and by other members to elect a new master if the master node fails.
ClusterManger manages lifecycle of [the] SPI.
What do you think of Shoal's unique approach to clustering?
> > "Shoal's unique approach to clustering" > > What(which) is the unique aspects of Shoal ? >
Based on the project documentation, an intriguing aspect of this project is that you can plug in your choice of group communication protocols. Many clustering (not caching) solutions I know of are in some ways tied to their own communication mechanism, be that simple UDP packets or some reliable multicast group communication.
Thanks for posting details of Shoal here and calling out the uniqueness factor.
Our goal with Shoal is to provide a uniform API that distributed systems applications can talk to leaving the complexity of discovering cluster group and members, managing group composition changes, monitoring members, and divining the network locational details to Shoal's Group Management Service layer, and the underlying group communication service provider without having to tightly bind to a specific provider.
The Jxta based group service provider referred to in Mohamed's blog provides all of the required functionalities that Group Management Service needs. The Group Management Service, besides providing the client API and the communication provider SPI, also provides recovery oriented support that selects one of the surviving members to perform recovery operations on a failed member's resources. This is particularly useful for automating administrative work for an application such as recovering in flight transactions that were partially committed. GMS also provides the basis for build a distributed cache through the DistributedStateCache interface.
The project is the result of a collaborative effort between the GlassFish team and the Jxta team.
We welcome contributions through views, suggestions, critique, code and patch submissions, code review, spread the word, use cases, testimonials and any other way that will result in a widely used and useful quality product.
> Based on the project documentation, an intriguing aspect > of this project is that you can plug in your choice of > group communication protocols. Many clustering (not > caching) solutions I know of are in some ways tied to > their own communication mechanism, be that simple UDP > packets or some reliable multicast group communication.
There are a number of significant shortcomings in Shoal at first blush. First of all, its membership and messaging are both driven externally (not by the pluggable group comms lib), which means that "members" are just String objects and messages are just serializables.
The message part would be OK if the SPI let you receive. It does expose the ability to send a message to _a_ member (not quite what you'd expect in a _group_ comms SPI), but it doesn't expose receiving. I looked around a bit, but didn't have the time to unravel the mystery.
The implementations (which may just be example code) were fairly hastily thrown together. In one instance:
private synchronized void sendMessage ( final List<String> members, final DSCMessage msg ) throws GMSException { if(members != null && !members.isEmpty()){ for(String member : members){ sendMessage(member, msg); } } }
So for each concrete implementation of a service, we'll hard-code the group comms logic? Again, this belongs in the SPI, otherwise the multicast-based impls (jgroups being one) will just repeat the same mcast message n times for a group of n. Obviously not the desired outcome (unless you're trying to make jgroups look bad.)
Shreedhar et al, please don't take this as anything but _constructive_ criticism. Sooner or later, this part of technology (clustering, group comms) needs a standard plug. I don't want to discourage you from your efforts, even if my comments seem overly critical.
My first suggestion is to plug in a couple more group comms libs .. after you get three or four plugged in, start again with a blank sheet of paper and see what design you come up with. Maybe it's unchanged .. and maybe its completely different.
Also, if you're based in Sun's Burlington campus, drop me an email (first initial last name at tangosol dot com) and I'd be glad to have you in to our lab to see some of our clustering work.
Hi Cameron We very much welcome your critique. What good is an open source project if constructive observations are not taken into account ? :)
At the outset, let me mention our motivation behind open sourcing this project : a. to get a community developed clustering API that incorporates many differing views into the design b. collaborate with more segment players such as yourself and group comm lib providers such as JGroups, and many many other folks/projects to evolve a standard plug. c. potentially explore the possibility for a JSR in this area as we find this lacking
What we have currently is the beginnings of an attempt towards this and as such one will find shortcomings
I would very much welcome collaborative engagement with you/Tangosol to evolve this SPI/project.
Unfortunately, I am not in Burlington. Perhaps a virtual conference ? My mailing address is shreed AT sun DOT com I will send you an email on your address to follow up.
I have created issues in our Issue Tracker out of your observations in this thread.
Previous Topic
