The Artima Developer Community
Sponsored Link

Weblogs Forum
How to think like a Jinition

21 replies on 2 pages. Most recent reply: Jan 31, 2005 11:51 AM by Gregg Wonderly

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 21 replies on 2 pages [ « | 1 2 ]
guest

Posts: 4
Nickname: guestsnota
Registered: Apr, 2004

Re: How to think like a Jinition Posted: Apr 22, 2004 4:51 PM
Reply to this message Reply
Advertisement
> Okay, so if I read you correctly, you're saying that all
> you care about is getting from A to B?

Yes.

> You don't care about what kind of solution?

I care that the solution correctly implements the requirements.

> If that's really your position, what is there left to discuss?

Explain why implementing the common (80% use case) middleware problems would benefit if done using Jini rather than J2EE.

If there's no clear answer to that (and please, please no car analogies, just technical arguments) then why should Jini gain more widespread adoption?

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

What's the common task for J2EE developers? Posted: Apr 22, 2004 11:42 PM
Reply to this message Reply
> Explain why implementing the common (80% use case)
> middleware problems would benefit if done using Jini
> rather than J2EE.

It appears that everyone has a different idea about what everyone is using J2EE for in their environment. Why don't you give us a specific application in J2EE terms that you think is a common thing. Then, I'll try and find something that Jini can do with that application that J2EE can't.

But, I think we also need to get past the basic issue of which hammer is better. J2EE is not a hammer and Jini is not a hammer. They are tool sets with specific attributes and capabilities. They are based on the Java programming language, and as such do share a great deal of common capabilities.

One of my arguments is that the Jini APIs for doing some of the things that they do in common feel more natural, and provide a better opportunity for extensability, security and maintainability. But, just like many J2EE developers that don't have Jini experience, my J2EE experience is limited. I've poured over the APIs in years past, but things continue to change and I can't keep up with stuff that I am not using actively.

> If there's no clear answer to that (and please, please no
> car analogies, just technical arguments) then why should
> Jini gain more widespread adoption?

Well, I think there is an answer and there is some technical argument to be made. The question is, is the different so black and white as to be obvious one way or the other?

Guest

Posts: 2
Nickname: cincygamer
Registered: Jan, 2005

Re: How to think like a Jinition Posted: Jan 14, 2005 1:22 AM
Reply to this message Reply
I use Jini for its JavaSpace implementation, not as a facade for database or to connect to J2ME or a toaster.
UPnP is better for that and much more industry supported.

The problems I have run into with Jini (really JavaSpace) are:

1. Jini 2.0 is too hard to setup and get the sample projects running. Yet, I dont want to stay on Jini 1.2.1. THat has really taken alot of the wind out of my sails. It just is too hard for me to work with. Jini 1.2.1 works great.

2. JavaSpaces is dog slow. Have you ever ran real benchmarks inserting and taking empty enities? It really sucks. It tried v2.0, its still orders of magnitute slower than using sockets or a database. And much slower than EJB.

3. To effectively use JavaSpaces in a "real" production application you need a lot more real pieces. Operations (Help Desk) needs to be able to support Jini. So, YOU the developer end up writing all these little utilities to monitor the server or to start restart services. In Jini 2.0 the StartService utility is gone and replaced with scripts. The problem I ran into was I had to write so much utility tools to monitor, I couldnt make progress on the application.

4. Jini is so unreliable. Try running the sample ray trace app and kill the Space (service) and restart. THe clients (workers) freak out and the dreaded stack trace exceptions appear. The workers need to be restarted most of the time.

5. Jini is unreliable (pt 2). Run the ray tracing app, sometimes not all the blocks make it. (COuld be a bug in the demo app). Makes you look like crap in front of your bosses to see those missing pieces when you showcase the app. I work in insurance and if these are claims.. oops theres a check that didnt get sent out.

6. Bottlenecks. The server the Space is running on only goes so fast serving objects, you need mulitple servers for producers or the space is starved and multiple consumers to pull results from Space, bottleneck.
So we are up to: 2 CPUS for workers, 2 LUS/Space, 1 CPU producer, 1 CPU consumer = 6 computers .. you need 2 spaces fr performance and failover for a real app.


7. Errors. They are really hard to debug. You must really understand RemoteObject and RMI errors and know what is going on. Not fun when you are in downtime. All apps have downtime and severe crash. I had a production C program crap out halfway thru execution with no reason. The same program with same data file works fine on all other re-tests.

8. Politics. I cant even talk my boss into letting me code Java anymore. Management purposely bought 12 .Net licenses even though we have 2 good Java devs and 0 .Net experience. The reason: Operations cannot support Java servers and "they will learn how to operate Microsoft software easier"

I am even Java certified (SCJP) and boss could care less. At my Java developer job at Lexis Nexis, I tried to push Jini and its too much to explain when everyone knows the J2EE model. I tried to push Jini for HPC while in in grad school.. "Too slow.. must use MPI" and have no apps to show because MPI is so hard to program. Java can be hard to sell is my point.


9. In the enterprise environment, the data is already in a DBMS. Its better just to use stored procs. Good SPs are a second or so (sub second at Lexis), and database operations are basically constant time O(1). Properly indexed tables act like a big hashtable. To run a INSERT INTO (SELECT ...) to dump rows in a table takes about the same amount of time versus Jini O(n) + all space overhead. Space gets much slower as fills. DBs cache the data. You have to write your own cache mechanism in JavaSpace.

10. "Just buy a faster box" Idiots I work with are not distributed computing fans and believe in buying a faster dell server.

11. Jini strengths are not to be for database access. It was designed for autonomous networking. As Keith Edwards said in Core Jini, "plug it in and it just works..like a phone"

12. Lack of data. To get to the point where you need that much CPU to process something, its hard to find enough raw data that is CPU bound, not DB bound, easily parallizible. We process some large jobs at umr.com (insurance company) so it runs for awhile. I dont have any processes that are 100% CPU all the time. If so, I really would use something faster such as C based MPI or Java + sockets or J2EE clustering.

I could ramble on.. thanks for reading my verbage. YMMV, I really tried to get Jini in-house and failed.

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: How to think like a Jinition Posted: Jan 14, 2005 1:14 PM
Reply to this message Reply
> The problems I have run into with Jini (really JavaSpace)
> are:
>
> 1. Jini 2.0 is too hard to setup and get the sample
> projects running. Yet, I dont want to stay on Jini 1.2.1.
> THat has really taken alot of the wind out of my sails. It
> just is too hard for me to work with. Jini 1.2.1 works
> great.

Humm. I do see a lot of questions on the Jini-users list about getting 2.0 running. But, I am not sure what the total issue is. I use the ServiceStarter framework in 2.0 and it works just fine for me. I can trivially (for me) add or remove services from the configuration, switch out endpoints etc.

Tell me more about the specific issues that you've encountered. The community, overall, wants to help make Jini easier to get started with. But, there are numerous concepts that one has to get their thinking in line with.

> 2. JavaSpaces is dog slow. Have you ever ran real
> benchmarks inserting and taking empty enities? It really
> sucks. It tried v2.0, its still orders of magnitute slower
> than using sockets or a database. And much slower than
> EJB.

I think that this is an issue that a lot of people run up against when they try to use a Javaspace like an RDBMS. The use of Javaspaces is really as a short term asynchronous buffer. It is tailored and designed to not deal with an extraordinary number of Entry objects. Also, the bandwidth through the space needs to be metered. But, this is try of any distributed system.

In J2EE and other single server approaches, the load on the server will limit the speed at which additional work can enter. So, there is a somewhat natural balancing of load that can occur. Though, you can still have poor balancing due to cheap inbound work compared to expensive outbound work.

Using acknowledgements and other out of band information to control the rate that work is presented to the system can provide a much better opportunity for success in distributed system design.

Any distributed system really must be able of balancing its activities against available resources in a deterministic way. There are some very direct ways to deny access to an overloaded part of the system and to stage data in a way that throttles the system.

In my broker, I use a disk based queue balanced by a per queue thread limit. What this allows is for the normal latency in the system to throttle my access to the destination servers. But, also increasing the thread count can increase system throughput by letting additional threads be processing on either end of the network while data is in transit.

So, on any system that the broker is installed, we only adjust the number of threads processing in a queue to balance the system. Bursty inbound data, is archived to disk. Stuck data that can't be delivered is left on disk, and placed into an alternate part of the queue since it is now old data.

> 3. To effectively use JavaSpaces in a "real" production
> application you need a lot more real pieces. Operations
> (Help Desk) needs to be able to support Jini. So, YOU the
> e developer end up writing all these little utilities to
> monitor the server or to start restart services. In Jini
> 2.0 the StartService utility is gone and replaced with
> scripts. The problem I ran into was I had to write so much
> utility tools to monitor, I couldnt make progress on the
> application.

Are you aware of the configuration based startup and the ServiceStarter framework? This makes it possible to start all of the JTSK services in a single batch file/shell script that doesn't have to change much. The hello world stuff in Jini2.0 is pretty detailed in how to use this stuff.

> 4. Jini is so unreliable. Try running the sample ray
> trace app and kill the Space (service) and restart. THe
> clients (workers) freak out and the dreaded stack trace
> exceptions appear. The workers need to be restarted most
> of the time.

It's not unreliable. You have to understand what it is trying to accomplish. It is not trying to provide a 10ms recovery of errors. It is trying to guarentee that long term failures are noticed and reported by the service not being visible in the LUS any longer. What is at issue is that the Sun teams experience with long term network behavior smacks in the face of what people want to experience based on their own local network experiences where CPUs are never overloaded and bandwidth is always available.

In large scale server environments, you have to be willing to wait for real errors. The TCP spec suggests times as long as 5min for timeouts in connection shutdown. These timeouts are part of the history of TCP and its development in the time of .75mip machines running over 2400-9600 baud dialup slip links or at best over DS0 or DS1 trunks.

So, while they are very conservative, they will eventually provide the desired actions. The timers are configurable for most of the places where this occurs. There was an announcement on the porter.jini.org list today saying that they were going to issue a lease cancel() call when a service entry is removed from the LUS. Currently, the service entry is removed from the LUS, but the lease is not cancelled. Thus, this lease time is the upper limit on the time to failure detection.

> 5. Jini is unreliable (pt 2). Run the ray tracing app,
> sometimes not all the blocks make it. (COuld be a bug in
> the demo app). Makes you look like crap in front of your
> bosses to see those missing pieces when you showcase the
> app. I work in insurance and if these are claims.. oops
> theres a check that didnt get sent out.

In any data routing application, you have to provide appropriate behavior for any guarantees the system must make. Requiring intervening nodes of a routing system to guarantee anything more than "Best Effort" raises the bar tremendously. If the data is that important, and you can't deal with out of order, out of date, sometime later delivery, then you need transactions from end to end.

On top of that, you'll need data archiving and backup on every node along the path. I've found that it is much easier to only have the originator and the final receiver aware of these needs, and they should have a mechanism to let the receiver know that data is missing and be able to identify to the sender which piece needs to be resent.

The result of such duplication is a form of offsite backup as well as a much easier way to reprocess or recover data after the fact.

> 6. Bottlenecks. The server the Space is running on only
> goes so fast serving objects, you need mulitple servers
> for producers or the space is starved and multiple
> consumers to pull results from Space, bottleneck.
> So we are up to: 2 CPUS for workers, 2 LUS/Space, 1 CPU
> producer, 1 CPU consumer = 6 computers .. you need 2
> spaces fr performance and failover for a real app.

If your problem is simple enough to solve on a single CPU, then you probably don't need to build a distributed system. However if your problem is open ended in complexity and load, then it is probably a good idea to start with a distributed system.

> 7. Errors. They are really hard to debug. You must really
> understand RemoteObject and RMI errors and know what is
> going on. Not fun when you are in downtime. All apps have
> downtime and severe crash.

The configuration of your system, your network, and your adminstrative practices all play a part in how you recover from a crash. RemoteObject and RMI semantics can be difficult to understand if you don't know what you are learning about. Here's my list of things that I think are important to know about RMI as implemented in Jini 2.0 and JERI in particular.

1. Remote objects have to have code somewhere to be
deserialized and used. The codebase setting is
what tells a receiving JVM where it might find
any needed class file definitions. You have to
understand ClassLoader and you have to understand
why looking up the classloader chain first is bad
for downloaded code. The PreferredClassLoader lets
you say when you want the classloader to look up
the chain first so that you can eliminate the
ClassCastExceptions that happen when a client
defines a class that is also resolved to the jar
file that the service comes from.
2. Proxy classes are automatically generated for you,
BUT proxy classes are not automatically used to
replace Remote implementing classes. You have to
explicitly export everything. This might seem like
a real pain, but it actually provides a great
opportunity for centralizing exporting. I use the
factory pattern in my applications and create a
export managing class that has static methods for
exporting. This allows me to handle specific classes
differently than I may have originally thought I
would because all the exports happen in the same
place instead of being strewn throughout the code.
also, consider passing another parameter to the
exporter that identifies the specfic source of
the export in case you want to 'count' exports
to know how many of which type of remote reference
you are generating which can help you manage the
load on a server by restricting clients.

There are some other things about RMI such as distributed garbage collection which you need to understand too. For instance in the above point #2, clients that come and go probably need to use DGC to reclaim local resources, or you'll need to use another leasing mechanism to trigger the unexport of the object and reclamation of other resources.
This is not a simple subject.

> 8. Politics. I cant even talk my boss into letting me code
> Java anymore. Management purposely bought 12 .Net licenses
> even though we have 2 good Java devs and 0 .Net
> experience. The reason: Operations cannot support Java
> servers and "they will learn how to operate Microsoft
> software easier"

The limitations of the .NET platform are the simplifying factors. Of course there are many issues about .NET which make it seem exciting. The WS-* standards make you feel vendor independent because you can use SOAP everywhere. But, that's not where you application is adding value. Your time investment and monetary commitment is to the programming platform. So, if you find you don't like that platform and can't continue down that path, then, you are no better off than if you had chosen Jini or CORBA or something else. You still get to recreate your value added software to run on a different platform.

> I am even Java certified (SCJP) and boss could care less.
> At my Java developer job at Lexis Nexis, I tried to push
> Jini and its too much to explain when everyone knows the
> J2EE model. I tried to push Jini for HPC while in in grad
> school.. "Too slow.. must use MPI" and have no apps to
> show because MPI is so hard to program. Java can be hard
> to sell is my point.

The primary issue for me is that people think the only value added for a piece of software is the speed at which it solves a problem. More efficient software keeps them from having to buy bigger hardware. But, J2EE and .NET both sway you towards single large processors because distributing is harder from the perspective of having to know IP addresses and having to implement leasing and having to do a lot of things that Jini already makes available in the APIs.

What Jini allows is for smaller, existing hardware to be put to use to solve problems in ways that might not be as obvious, nor as tractable as a single processor solution. But, in the end, because you distributed, you get automatic fail over potential as part of your architecture. You can just have a disk and a machine and be back up and running without anyone having to know that you had to switch machines out. They'll see the failure if you're running on a single machine, but if not...

> 9. In the enterprise environment, the data is already in
> a DBMS. Its better just to use stored procs. Good SPs are
> a second or so (sub second at Lexis), and database
> operations are basically constant time O(1). Properly
> indexed tables act like a big hashtable. To run a INSERT
> INTO (SELECT ...) to dump rows in a table takes about the
> same amount of time versus Jini O(n) + all space overhead.
> Space gets much slower as fills. DBs cache the data. You
> have to write your own cache mechanism in JavaSpace.

DO NOT USE A JAVASPACE IF YOU NEED AN RDBMS. This is an important thing to recognize. RDBMS like functionality can not be recreated in javaspaces without some compromise in performance due to the way that javaspaces accesses data. There are, of course, situations where javaspaces can provide the exact behaviour that an RDBMS can provide, but that is a limited set of behaviours.

> 10. "Just buy a faster box" Idiots I work with are not
> distributed computing fans and believe in buying a faster
> dell server.

See my discussion on point 8 above.

> 11. Jini strengths are not to be for database access. It
> was designed for autonomous networking. As Keith Edwards
> said in Core Jini, "plug it in and it just works..like a
> phone"

The concepts of the Jini system are very much usable in a database access application. J2EE was initially targeted at database access mixed with web server applications. Overtime, new functionality in the WS-* standards will drive it to contain more and more support for other things. So, the appserver vendors will be selling these great big, huge complicated systems that will be expensive to develop and support. Their prices will go up. The value added features will further anchor customers with a particular vendor.

Opensource solutions such as JBoss and Apache tools can mediate some of the expense. But, I contend that the adoption of these technologies, blindly, will limit the types of problems that you can solve, and will delay introduction or movement to architectures such as Jini that will trivially solve integration and distributed computing.

> 12. Lack of data. To get to the point where you need that
> much CPU to process something, its hard to find enough raw
> data that is CPU bound, not DB bound, easily parallizible.
> We process some large jobs at umr.com (insurance company)
> so it runs for awhile. I dont have any processes that are
> 100% CPU all the time. If so, I really would use something
> faster such as C based MPI or Java + sockets or J2EE
> clustering.

I'm not sure what issue you are talking about here.

> I could ramble on.. thanks for reading my verbage. YMMV, I
> really tried to get Jini in-house and failed.

Please feel free to follow up here and/or on the Jini-users list. I think this is a great set of issues, so I am going to post this back on Jini-users to see if there are others who will share their experiences and comments.

Guest

Posts: 2
Nickname: cincygamer
Registered: Jan, 2005

Re: How to think like a Jinition Posted: Jan 14, 2005 6:08 PM
Reply to this message Reply
That was an amazing reply. Thanks so much for the time. We are talking about using Jini in different ways. It seems you are using Jini in more of a web services model. Similiar to how Orbitz uses Jini?

I am using Jini to get to JavaSpace to get to the usual master/slave pattern. The entities are Tasks pulled by a worker and result of tasks put back into the Space to consumer(s). THe SETI@Home model.

Why we need distrubuted workers is because each task takes so long to compute if the system is designed correctly. (Seti packets)

How I am using Jini services is as a way to choose which set of tasks to take and as a way for the clients(workers) to dynamically download code and start crunching the tasks.

Should I use another method for this model?

guest

Posts: 9
Nickname: guest
Registered: Dec, 2003

Re: How to think like a Jinition Posted: Jan 24, 2005 6:27 PM
Reply to this message Reply
> What Jini allows is for smaller, existing hardware to be
> put to use to solve problems in ways that might not be as
> obvious, nor as tractable as a single processor solution.

How similar is this to the grid systems that have become popular in the press in recent years?

Thanks,
-james.
--
James Stansell

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: How to think like a Jinition Posted: Jan 31, 2005 11:51 AM
Reply to this message Reply
> > What Jini allows is for smaller, existing hardware to be
> > put to use to solve problems in ways that might not be as
> > obvious, nor as tractable as a single processor solution.
>
> How similar is this to the grid systems that have become
> popular in the press in recent years?

What is still happening today is that people can't imagine how to use Java's ability to move code around as part of the grid solution. Instead, there are many canned types of jobs that you can send data to, or there are specific OS machines that you can send specific programs to.

C and C++ seemed to be used for many of these 'grid problems'. I guess that is because the need for total speed is the driving issue because C and C++ don't have native threading support because they are languages, not platforms. Java, as a platform, specifies both the language and a comprehensive library that is standardized to the spec version applicable everywhere you find something with the Java name on it. This is where Microsoft tried to break the promise, and got sued. Then, they ran off with the MSIL and C# plus restricted C++ etc because they know the benefits of the model. They just don't want to have an even playing field.

I believe that many of the researchers and programmers using grids have been using languages such as C and C++, without threading models and without distributed synchronization support, for so long that they don't/can't think in terms of compartmentalization. They don't see how to break the total solution into many problems that could be solved simulataneously on much smaller hardware using languages such as Java. Instead, they are used to brute force iterative solutions. The SETI@home project shows that there are some people who understand the power of distributed computing power! This is a singular solution which is capitalizing on the largest compute farm in the world.

The Jini tool set provides a distributed transaction service to solve synchronization issues. Jini provides leasing for failure management. Java's mobile code model, as exploited by the RMI programming model and the JERI transport's extensibility, solves many issues with how to keep from restricting a system to only solve problems that were originally perceived as applicable. Finally, the ubiquitous Linda programming model as provided by Javaspaces provides the foundation of a very flexible worker thread management scheme.

Flat View: This topic has 21 replies on 2 pages [ « | 1  2 ]
Topic: The Boost Interfaces Library Arrives! Previous Topic   Next Topic Topic: Agile C++?


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us