While inexpensive multicore processors have become widely available in recent years, the clock-speed of individual CPU cores have increased only marginally. As a result, developers are forced to rethink and re-architect applications so that software can take advantage of concurrency as a primary means of scaling performance. Since the number of cores available in commodity processors is expected to increase in the future, the more concurrency an application can take advantage of, the more that application can scale—at least in principle.
In practice, however, each of those processor cores needs to work on some data—and that data needs to somehow be ferried in and out of the cores. In addition, concurrent algorithms require extra communication overhead, consuming more bandwidth. As the number of cores increases, the bandwidth available to carry that data to the processors may limit scalability, according to Sun's David Dice:
If you were to plot throughput on the Y-axis and concurrency on the X-axis the shape of the curve would be convex downward -- performance climbs up to an apex and then falls off...
How can this happen? A common reason is that the communication overheads start to dominate and the benefit of concurrency is overcome by communication and synchronization costs. Under such circumstances it's common to introduce some type of admission control -- say, simple
back-off or more elaborate mechanisms -- to restrict concurrency. Ideally, this yields an asymptotic curve where performance remains constant after reaching the peak, avoiding any fall-off.
To what extent do you think bandwidth required to fetch data in and out of processor cores will be a limiting factor in scaling applications via concurrency?
So long as there are people selling and deploying clustered servers, we'll continue to reap the benefits of processor advances as fewer and fewer boxes are required to satisfy larger and larger already parallelized (via the web server model of requests) work loads. There will be discontinuous speedups when what is currently deployed as a cluster can be deployed on a single machine at reasonable cost. And again when the DB/persistent store can be put on the same physical hardware at reasonable cost.
Fork/join frameworks will fill in the cases where serious parallel computing is needed.
Given that the network and disk are present in the majority of enterprise applications being built, I don't see fetching data in and out of a processor core being a serious problem for most enterprise developers.
It's a little unintuitive but in a lot of the work I do where the key speed metric is average speed over large numbers of transactions, the best approach to improving scalability is to eliminate synchronization (i.e. such that it is not needed) as much as possible. The threads talk, the faster they run in parallel.
The metaphor I use is the limited access highway vs. city streets. We can drive much faster on limited access roads because there are no hard intersections, only oblique on and off ramps. Traffic control signals do not improve speed, they reduce it.
In both cases, it's the high-level structure that makes the most difference. Just as better traffic light technology can only create modest improvements, better thread communication technology only gets you so far.
Oh, let's go back to 4 register pre-64-bit x86 architecture, where you're doing things like xor'ing registers to swap values to relieve register pressure.
C'mon. Just because you don't know what tricks to use in multicore, doesn't make it an impossible problem.
A while back we had a minor disagreement over the prospective performance advances of newer GC algorithms. At the Java Language Summit, Cliff Click of Azul Systems gave a wide-ranging talk about the hardware Azul is putting out. They've got some interesting papers up, and among them is a "pauseless" GC system:
Aside from von Neumann block-structured imperative style, newer memory technologies and memory controllers have to be designed to keep up with the shift in processor architecture. This has meant 3D memory units, and with that the challenges of handling more and more heat dissipation.
I think the biggest problem moving forward will not be the von Neumann bottleneck itself, but instead the operating systems built on it, with their rigid abstractions and lack of programmable kernel abstractions.
Overall, the multicore upheavel will be a good thing, not a bad thing. It will force software engineering to become just that - engineering.
I know Cliff (one scarily-brilliant dude), but with a 10-line program (written by Gil Tene, Azul CTO) you can get the Azul GC (which has hardware assist from the memory controller) to hard-pause for minutes. In other words, even with all that foresight and brilliance, building their own hardware and OS and JVM (all in one), they still [can] suffer from excruciatingly long full GC pauses.
> I know Cliff (one scarily-brilliant dude), but with a > 10-line program (written by Gil Tene, Azul CTO) you can > get the Azul GC (which has hardware assist from the memory > controller) to hard-pause for minutes. In other words, > even with all that foresight and brilliance, building > their own hardware and OS and JVM (all in one), they still > [can] suffer from excruciatingly long full GC pauses.
Interesting. Is it a realistic (from an enterprise perspective)? The Azul whitepaper doesn't metion it (or I didn't catch it when I read it.) Is there a paper anywhere on the problem?
Regarding your blog post, as is obvious from my other posts, I fall into the "things are fast enough" camp, the "there aren't as many parallelizable problems as people think" camp and the "developer productivity uber alles" camp. What do you think of my assertion that, as long as we are halving the number of nodes in our app server clusters with each generation, we'll still see plenty of speedup as caching becomes a chip problem rather than a network problem?
Regardless, the problem domain expands over time in such a way that managed "heap" sizes will grow, reaching a point where automated memory management will begin to work against itself, i.e. as we know it today, it will become largely unusable as memory sizes grow.
Previous Topic
