This article is sponsored by the Java Community Process.
java.io.File, the new
Watchableinterface, and asynchronous I/O programming. Bateman is an engineer in the SE Core Technologies team at Sun.
Frank Sommers: Please give us a brief overview of JSR 203, More New I/O APIs for the Java Platform?
Alan Bateman: The first NIO specification was JSR 51, New I/O APIs for the Java Platform, led by Mark Reinhold. It was delivered into JDK 1.4, and that API put in the basis for NIO in Java, focusing on buffers, channels, and charsets. JSR 51 delivered the first piece of the scalable socket I/Os into the platform, providing a non-blocking, multiplexed I/O API, thus allowing the development of highly scalable servers without having to resort to native code.
JSR 203 takes over from JSR 51. For many developers, the most
significant goal of JSR 203 is to address issues with
java.io.File by developing a new file system
interface. We've been living with
ten years now. It has a lot of problems, and is missing a lot of
functionality. We're trying to address in JSR 203 all those
problems, and that's the largest part of this JSR.
We're also completing the socket channel functionality that was started in JSR 51. That means updating the existing socket channel API, and adding things like multitasking. The third part of JSR 203 is to develop an asynchronous I/O API for both sockets and files.
A relatively small piece of JSR 203 is that we revisited the
buffers API in Java NIO. The original buffers were indexed by an
int, and we defined a new set of buffer classes to
be used by very large containers. If you want a contiguous
mapping of a very large file, larger than 2GB, you can do that
with this new API.
Our goal is to eventually put this API forward as a component that would go into Java SE 7.
Frank Sommers: What made it necessary to
Alan Bateman: For starters,
java.io.File defines a lot of methods that return a
boolean value, rather than throwing exceptions when they fail. If
you try to delete a file, for example, and that method returns
false, you have no idea why that operation failed.
That's a big problem that comes up very often, and we solved that
in JSR 203.
java.io.File doesn't work well with symbolic
links on the file system—we're providing an API for that. In
java.io.File only provides access to a
very limited set of file attributes. We continuously get requests
from people who want to access file permissions and access
control lists, and those are some of the attributes we need to
support in JSR 203.
A big issue with file attributes in
is performance when querying the file system. When you ask a file
its modification time or what type of file it is, every one of
those methods goes to the file system. It's very common to see
multiple methods to
java.io.File called together.
Because each one of those goes to the file system, that causes
performance problems. One of our goals for the new API was bulk
access to file attributes, allowing you to get access to all the
file attributes at the same time.
java.io.File works for directories just
doesn't scale to large directories. There are also a lot of
problems with how it handles paths. We get requests from people
wanting file system access to other types of files, such as
memory files, and
java.io.File is missing a lot of
basic functionality there, too.
You won't find a method in
copies a file or one that moves a file. We have a
rename() in there, and that drives people crazy
because it doesn't work the way you'd expect it to.
Frank Sommers: Do you envision this API to
Alan Bateman: We don't propose to deprecate
java.io.File. We have instead added a method in
java.io.File that lets you get a reference to the
new API and allows for interoperability. That solves some of the
java.io.File in a very localized
For example, to solve the problem of trying to delete a file
and getting back a boolean
false if the delete
fails, you can just go to that piece of the code now and invoke
toPathReference() method to get access to the
remove() method in the new API. That will give you
more exceptions so you'll know why the operation actually
Frank Sommers: What does the
Watchable interface provide in the new API?
Alan Bateman: The
interface is implemented by objects that can be watched. There is
WatchService class that watches objects in the
file system for changes. Many developers would relate to the
example when they work with an editor, and some file open in one
of the editor buffers is changed outside the editor. The editor
pops up a window to tell you that that file changed, and asks if
you wanted to reload the file. That's a simple example for when
you would want to get a notification.
A lot of applications, particularly editors and IDEs, are currently forced to poll the file system, and that creates big performance issues. We also see this requirement with servers that have a directory where you deploy JAR files to. The server wants to be able to recognize when a JAR file is being added so that it can load, or reload, that JAR. The server doesn't want to have to have a thread in the background polling a directory and hitting the file system all the time, but would want to be notified instead of file system or file changes. Watching configuration files is another thing that comes up periodically.
WatchService plugs into the native file
system notification mechanism, if one is available. Linux people
will recognize this as an
There are equivalent interfaces on Windows, and the Solaris folks
have been working on a similar API for the next major release of
Solaris. When there is no native file system support for
notifications on file changes, the JSR 203 implementation will
end up having to poll like the file I/O implementations do
Frank Sommers: What sorts of native files and file systems will the JSR 203 API work with?
Alan Bateman: Out of the box, this API works with the native, or platform, file system. We also define a service-provider interface that allows people to develop their own file systems. A common example is to be able to develop memory file systems. You can reserve a chunk of memory in order to emulate a file system in memory. To do that, you can develop a file system provider that provides a file system interface, and that can just plug into the JSR 203 API implementation. The JSR 203 specification shows how to deploy these file system implementations so that everything just works.
There are other interesting ideas for file systems out there, and many of them are being pursued in other projects. There are file system interfaces into archive files, for example, such as JAR and zip files that are simple file systems, but are interesting in some cases. You'll be able to deploy those types of file system providers as well. In general, anything that you can develop a provider for, you'll be able to plug in, including distributed file systems.
Frank Sommers: What are I/O channels, and what new channel features do you provide in JSR 203?
Alan Bateman: The most important difference
between channels and
java.net.Sockets is that in the
case of channels, the I/O is done with byte buffers. Byte buffers
offer the potential for performance gain as the bytes may not
need to be copied into a native buffer to perform the I/O.
The other big difference between sockets and channels is that the latter can be configured to be non-blocking and thus allowing for event-driven designs. That provides a much more scalable way to develop servers than the old, one thread-per-socket model does.
JSR 51 introduced several network-oriented channels:
DatagramChannel. Those did not provide complete
abstractions of a network socket: They didn't have methods that
allow you to bind into the channel socket, set socket options,
and so on. Instead, they use a socket adapter: A
socket() method defined in each of those channels
that provides a
java.net.Socket that you actually
use to bind and set options and so on. The reason for that was
interoperability and, mostly, because there was no time to define
a full socket channel API in JSR 51.
We've completed that work in JSR 203. If you look at the
channels package in JSR 203, you'll see new interfaces such as
Those let you actually bind and set options directly in the
channels package. You don't have to mix the socket and the socket
channel APIs to do those things, which has been very
counter-intuitive for most people.
The second part of completing work on channels is about multicasting support: We didn't have multicast support in the channels package before, and now we've added that.
Frank Sommers: JSR 203 defines an asynchronous I/O API, in addition to the non-blocking I/O model introduced in JSR 51. What's the difference between asynchronous and non-blocking I/O?
Alan Bateman: With non-blocking I/O, you're getting events through a selector when the channel is ready to do I/O. The asynchronous API gives you a notification when the I/O is completed.
For example, with a socket channel in a non-blocking mode, you register with a selector, and the selector will give you a notification when there is data on that socket to read. With the asynchronous I/O, you actually start the read, and the I/O will complete sometime later when the read has happened and there is data in your buffer.
In the asynchronous API we have what we call a completion handler. You specify a completion handler when you do your read, and the completion handler is invoked to tell you that the I/O operation has completed.
We put a lot of thought into how the asynchronous API would be used in server applications. You'll see in the asynchronous I/O package that we allow high-end servers to be able to plug in their own thread pools, have their own configuration of thread pools, and so on.
Frank Sommers: How does a developer decide when to use the asynchronous API versus the non-blocking API?
Alan Bateman: When developing a highly scalable server, you have to take a lot of things into consideration. That includes the operating systems and hardware that the server will be deployed on.
Selector in the current API can deliver great
performance and scalability when mapped to an operating system
that has a highly scalable polling interface—both Solaris and
Linux have such interfaces for example. Asynchronous I/O delivers
great performance when the underlying operating system has a high
performance and scalable asynchronous I/O facility. The
application and server design is also critical.
In terms of design patterns, the
Selector in the
existing API is the basis for the Reactor role in the Reactor
pattern. The asynchronous I/O API, by contrast, enables the use
of the Proactor pattern. When to use one or the other can be
difficult to decide. In terms of complexity, they are both
advanced APIs, but some developers may find the asynchronous I/O
JSR 203, More New I/O APIs for the Java Platform
JSR 51, New I/O APIs for the Java Platform
Comparing Two High-Performance I/O Design Patterns
by Alexander Libman with Vladimir Gilbourd
Frank Sommers is Editor-in-Chief of Artima Developer. He also serves as chief editor of the IEEE Technical Committee on Scalable Computing's newsletter, and is an elected member of the Jini Community's Technical Advisory Committee. Prior to joining Artima, Frank wrote the Jiniology and Web services columns for JavaWorld.
This article is sponsored by the Java Community Process.