The Artima Developer Community
This article is sponsored by Sun Microsystems, I nc..

Leading-Edge Java
Dynamic Clustering with Jini Technology
by Frank Sommers
January 31, 2006

<<  Page 3 of 4  >>


Service-Oriented System Integration

Boomi's cluster architecture exploits the dynamic nature of Jini networking to provide high availability and failover. Boomi defines system integration services with Java interfaces, and registers implementations of those services in Jini lookup services on the cluster.

Interaction between services starts by a client performing a Jini service lookup based on the required service's type. Services can be deployed on the cluster in a redundant fashion. Since each service implementation registers a proxy that implements the service's Java interface, the client can retrieve and use any implementation's proxy. If a node fails, or is powered down, the service implementations on that node will not renew their lookup service registrations, causing the cluster's lookup services to purge those service proxies. Since a client's access to a service proxy is also based on leases, the client will have to locate a new service's proxy from the Jini lookup services. Dynamic service location and lease-based access to resources inherent in Jini networking thus provide the foundations of Boomi's high-availablity and failover infrastructure.

In Boomi's system integration architecture, a layer of transport services represents the gateway between the Boomi server and external data sources. Transport services act as I/O for the Boomi system. For instance, HTTP transport occurs via an HTTP-specific transport service, FTP transport through an FTP transport service, and input into a database is arranged via a database transport.

As messages enter the system, they are placed into a queue managed by a queue service. The queue service acts as a message delivery system, and is responsible for persisting data received from various transports to guarantee message delivery. Thus, the queue service is used by all other services. The queue service relies on a relational database to persist message data.

Figure 3. Data-flow parallelism on a cluster in support of system integration

While the database may form a single point of failure, in practice Boomi's system is deployed on a clustered database. All incoming data, as well as interim processing results, are saved in the database. When the queue service restarts after a crash, it first looks in the database for yet unprocessed data, and invokes services responsible for processing those messages.

Messages in the queue are eventually processed according to message type, message origin, and rules corresponding to additional message parameters. Such processing occurs by process services that not only transform data, but also route data to destinations specified by rules. Thus, process services form the core of the system integration functionality.

Boomi's process services implement about fifteen system integration patterns, such as data transformation, intelligent routing, decision validation, exception handling, or dynamic messaging. For each process service, users specify via a graphical rules designer what input data to expect, what transformation actions to take on what message types, and where results should be routed.

Events in Boomi's system occur not only in response to arriving messages, but also in an automated fashion induced by a Jini scheduling service. This service is analogous to the Unix cron daemon, and fires off events at specified intervals. General administrative responsibilities, in turn, are performed via a Jini administration service that controls configuration-level properties, such as where the database is located, and provisions security access. The administration service is also responsible for logging.

Finally, Boomi's Jini archiving service takes data out of the live data set maintained by the queue, and puts that data on a secondary storage device for historic preservation. For instance, the archiving service may pull data out of the database if a data item is older than a specified number of days, preventing the live database from becoming too large.

Figure 4. System integration services in Boomi's cluster architecture.

When a service boundary is crossed, Jini discovery, lookup, and remote method invocation facilitates service interaction. As an example, suppose that data needs to be read from a directory, transformed to a different format, and then written out to the directory again. In this example, the transport service reads the data in from the directory, and hands that data off to the queue service. The queue service persists the data in the database management system so that the data has guaranteed delivery: If the system fails at any point, that data item can be resent. Next, the queue service sends that data to the process service, which transforms the data to the desired format. The process service then places the data back into the queue. Lastly, the queue hands the results back to a transport service, which then writes out the data to the directory.

In this scenario, the queue service must find the administration service to obtain information about the system's database, and to download the needed JDBC driver. The queue service also has to locate transport services to send and retrieve data, and process services to hand data to for processing. As well, the queue service finds the archive service and passes to it data that is ready to be archived. Once the data is in the archive service, the archive service might seek additional archive services on the network in order to replicate the data.

Figure 5: Service interaction in Boomi's integration cluster

Boomi's cluster-based service deployment relies on the open-source Jini Rio project. Rio defines the cluster's desired operation in a declarative fashion. A Rio cluster configuration might specify how many instances of the process or transport services should be running. Rio's runtime infrastructure monitors the cluster and tries to ensure that desired cluster state. Suppose, for instance, that an administrator requests that four instances of the HTTP transport service be running. If a cluster node executing an HTTP transport service crashes, Rio starts a new HTTP transport service instance on an available node.

Boomi's administration, schedule, and queue services are considered singleton services: while they are deployed to cluster nodes, only one instance runs at a given time. On the other hand, the Boomi process service that does most of the heavy lifting on incoming data, is often clustered. As well, clustering the archive service provides multiple backup mechanisms across different cluster nodes. That way, archived data is replicated across multiple machines.

Figure 6: Cluster-based distribution of Boomi integration services

<<  Page 3 of 4  >>

This article is sponsored by Sun Microsystems, I nc..

Sponsored Links

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us