The Artima Developer Community
Sponsored Link

Java Community News
Brian Goetz on Fork-Join Parallelism in Java 7

3 replies on 1 page. Most recent reply: Feb 26, 2008 2:49 PM by Brian Goetz

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 3 replies on 1 page
Frank Sommers

Posts: 2642
Nickname: fsommers
Registered: Jan, 2002

Brian Goetz on Fork-Join Parallelism in Java 7 Posted: Nov 15, 2007 6:48 PM
Reply to this message Reply
Summary
Fork-join parallelism offers an elegant programming model to distribute a compute-intensive problem across available CPU cores. In a recent IBM developerWorks article, Brian Goetz describes the fork-join framework slated to be included in Java 7.
Advertisement

Noting that today's multi-core processors require a finer-grained programming model than the relatively course request-level parallelism prevalent in earlier Java applications, Brian Goetz notes in a recent IBM developerWorks article, Stick a fork in it: Learn how to exploit fine-grained parallelism using the fork-join framework coming in Java 7, that:

It is easy to imagine how you can keep a dozen processors busy using a coarse-grained task boundary such as a user request, but this technique will not scale to thousands of processors—traffic may scale exponentially for short periods of time, but eventually the hardware trend wins out.

As we enter the many-core era, we will need to find finer-grained parallelism or risk keeping processors idle even though there is plenty of work to do. As the dominant hardware platform shifts, so too must the software platform if we wish to keep up. To this end, Java 7 will include a framework for representing a certain class of finer-grained parallel algorithms: the fork-join framework.

A popular technique to achieve finer-grained concurrency involves breaking down a larger, compute-intensive task into many sub-tasks, and then merging the results once each subtask completes computation. Goetz shows how the fork-join programming model makes this type of problem decomposition easier.

If the problem is large enough to merit parallel decomposition, [a divide-and-conquer method] divides the problem into two or more sub-problems and recursively invokes itself on the sub-problems in parallel, waits for the results of the sub-problems, and then combines the results...

[A fork-join operation's] behavior is that the current task is suspended, the ... subtasks are executed in parallel, and the current task waits until they complete. Then the results of the ... subtasks can be combined. This kind of parallel decomposition is often called fork-join because executing a task forks (starts) multiple subtasks and then joins (waits for completion) with them.

In the article, Goetz discusses the upcoming fork-join API slated for inclusion in Java 7:

The operation invoke-in-parallel is implemented by the coInvoke() method, which invokes multiple actions simultaneously and waits for them all to complete. A ForkJoinExecutor is like an Executor in that it is designed for running tasks, except that it specifically designed for computationally intensive tasks that do not ever block except to wait for another task being processed by the same ForkJoinExecutor.

What do you think of the fork-join API proposed for Java 7?


Morgan Conrad

Posts: 307
Nickname: miata71
Registered: Mar, 2006

Re: Brian Goetz on Fork-Join Parallelism in Java 7 Posted: Nov 20, 2007 12:33 AM
Reply to this message Reply
Having recently written some similar code, my experience is that the actual forking/joining isn't the biggest issue. Though, I do admit their "work-stealing" concept is cool, and I wasn't creating more than 10s of tasks. IMO, the big issues are

1) event firing / reporting progress / logging / exceptions etc.
2) classic command-liney tasks (and most of the ones I happened to be dealing with) expect files as inputs, not in-memory data. And they write their results to files. Splitting and gathering these files is hard.

I only briefly skimmed Lea's paper, but it didn't seem to deal much with these issues. Not that this is bad, it shows that they tackled the low-hanging fruit first, and probably did a great job. I'm wondering if either I missed something, or there are plans to handle these in later Java libraries.

Gregg Irwin

Posts: 16
Nickname: greggirwin
Registered: Jan, 2004

Re: Brian Goetz on Fork-Join Parallelism in Java 7 Posted: Dec 6, 2007 3:52 PM
Reply to this message Reply
I can't get to the IBM site to read the original article, but this sounds like a spin on map-reduce, or am I mistaken?

Brian Goetz

Posts: 5
Nickname: briangoetz
Registered: Sep, 2005

Re: Brian Goetz on Fork-Join Parallelism in Java 7 Posted: Feb 26, 2008 2:49 PM
Reply to this message Reply
"Sounds like a spin on map-reduce".

Techniques like fork-join, work-stealing, and divide-and-conquer predate map-reduce by many years.

Map-reduce addresses a different space, though -- distributed computation. But the techniques used for both distributed decomposition via map-reduce and parallel decomposition for multicore via fork-join do have some elements in common.

Flat View: This topic has 3 replies on 1 page
Topic: Yahoo Releases YUI 2.5 Previous Topic   Next Topic Topic: Adobe Releases Flex 3, FlexBuilder 3


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us