The Artima Developer Community
Sponsored Link

Weblogs Forum
Myths of Memory Management

81 replies on 6 pages. Most recent reply: Sep 12, 2005 6:10 PM by Max Lybbert

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 81 replies on 6 pages [ « | 1 ... 2 3 4 5 6 | » ]
Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: GC -> Non-GC transitions Posted: Sep 7, 2005 2:17 PM
Reply to this message Reply
Advertisement
> The only common case I see for that these days is Mac OS X
> development where, despite Apple's marketing to the
> contrary, Java is very much a second-class citizen
> compared to Obj-C.
>
> Also, of the fresh C/C++ development that I've seen in
> complex applications, more are using GC solutions.

Seems to call for another blog: What is C++ for?

I'm not sure I'm up to writing that one yet. :)

John D. Mitchell

Posts: 244
Nickname: johnm
Registered: Apr, 2003

Essentials vs. Wastes of Time Posted: Sep 7, 2005 2:26 PM
Reply to this message Reply
> > Also, of the fresh C/C++ development that I've seen in
> > complex applications, more are using GC solutions.
>
> Seems to call for another blog: What is C++ for?
>
> I'm not sure I'm up to writing that one yet. :)

:-)

In terms of new, fresh development? Nothing.

In terms of extending legacy C/C++ systems, a lot. However, even here, given the amount of work switching to the latest UI toolkits, web-based development, etc. even this should, IMHO, be shrinking faster and faster as people silo that code off from the new code.

Vesa Karvonen

Posts: 116
Nickname: vkarvone
Registered: Jun, 2004

Re: Myths of Memory Management Posted: Sep 7, 2005 2:36 PM
Reply to this message Reply
> I keep trying to learn ML

I recommend that you get a good book on ML.

> That is, as an automatic variable, ~lock_file will be
> called at the end of its scope -- I don't have to
> remember, AND I know WHEN it will happen.

In ML you could do it like this:

withFileLocked (file, fn () => (* do whatever with file *))

The idea is simple: the function withFileLocked locks the given file before evaluating the given thunk and then unlocks the file after the thunk has been evaluated.

> (Yes, I know many readers know this, but a few comments
> convinced me that many do not).

I have written quite a bit of C++ code since 1997.

...

What is wrong with the formatting tags? At any rate, this isn't in bold in preview.

Todd Blanchard

Posts: 316
Nickname: tblanchard
Registered: May, 2003

Re: Myths of Memory Management Posted: Sep 7, 2005 3:37 PM
Reply to this message Reply
> When pop() is called, without the explicit nulling
> in the example, the class could cause a memory leak (more
> properly called "unintentional object retention," or
> sometimes called "object loitering") because the reference
> stored in stack[top+1] is no longer reachable by the
> program, but still considered reachable by the garbage
> collector.

> public class SimpleBoundedStack {
> public Object pop() {
> Object p = stack [top];
> stack [top--] = null; // explicit null
> return p;
> }
> }

Only a C or C++ programmer would find this requirement odd. The array is also a set of references to objects. Just because it is a hidden implementation detail doesn't mean the GC system shouldn't "see" it.

If putting the object into the container is considered to be sufficient to prevent GC (the container reference is "strong") then it is necessary to break the reference from the array to the object. What's the big deal?

Most GC systems also provide support for weak references. Perhaps you want one of these. If the container implements weak reference semantices, explicit nulling is not required and GC can proceed. This pretty well implies that you need to have some other reference that is strong if you mean to keep the object around.

GC doesn't eliminate the need to program with memory in mind, but it significantly reduces the need to micro-manage memory. In particular, it allows the developer to ignore cleaning up locally allocated temporaries. Longer lived object lifetimes still require a bit of planning.

So what. The GC system will still probably leak less than the manually managed system.

Todd Blanchard

Posts: 316
Nickname: tblanchard
Registered: May, 2003

Re: Memory management vs. Resource management Posted: Sep 7, 2005 3:46 PM
Reply to this message Reply
> This is precisely the point of my blog, a GC lulls you
> into thinking you can ignore memory management, though in
> reality it just leads to subtle design flaws which slip by
> unnoticed.

OK, so the fact is that you can ignore memory management 95% of the time. That should eliminate 95% of the bugs. The rest was going to leak anyhow because you'll get it wrong (or right) either way.

Competent users of GC systems are not lulled into any false sense of security. Incompetent ones may be, but they write leaky buggy code anyhow. At least with GC, their errors are reduced.

As for your overall point - memory management is book keeping. Computers are much better at book keeping than people are. Might as well let them to do it. Requiring developers to continue to focus on these micro issues prevents us from moving up to the next level.

You're advocating buggy whips in the 21st century. They're not completely useless, but they're getting to be boutique needs. We have bigger problems to solve than this. Time to call it done and move onwards (and upwards). Does it not disgust you that we're still fiddling with the same stupid problems after 20 years? It completely disgusts me.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Myths of Memory Management Posted: Sep 7, 2005 5:29 PM
Reply to this message Reply
> My reference to "finally" was actually not a reference to
> finalizers, but a reference to the strange attempt Java's
> designers made at making exception safe code (try ...
> catch ... finally). The concept is that the GC will take
> care of the memory leaks, but the finally block must be
> there to take care of any other necessary cleanup.

finally blocks don't really have anything to do with GC, so I'm not sure where you are going with this.

I also don't see what's strange about them. try/finally and try/catch/finally ae indespensible in a language without gotos. I would actually make the argument that they pretty much make goto unecessary.

I see where you are going with the autopointers. However, I prefer the delayed cleanup method. I prefer it over a sudden autopointer tree collapse that crushes system peformance under heavy load. In practice, I've found that using GC and Weak/Soft/Phantom/References are usually fine for resource cleanup and when they are not there are other perfectly acceptable approaches. One of the things I like about Java is that I am often challenged to find a different approach to solving an issue. A lot of times the new approach is better anyway.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Phantom References? Posted: Sep 7, 2005 5:40 PM
Reply to this message Reply
> Can somebody direct me to a Phantom References link? All
> I can find is at
> http://forum.java.sun.com/thread.jspa?threadID=609458&messa
> geID=3357928 which includes this gem "The running app
> closes the stream, and nullifies the object's reference
> just to be safe.
>
> Unfortunately, the JVM has other plans in mind. It throws
> the old Stream object into the garbage collector's work
> heap, then forgets it ever existed without bothering to
> stick around and personally make sure the file lock is
> released RIGHT AWAY" (third message).

http://java.sun.com/developer/technicalArticles/ALT/RefObj/index.html

I didn't mean to suggest that it would help with immediate cleanup. This is something that cannot be guaranteed in Java.

There is a simple approach to dealing with this kind of issue in Java. You create a Handler interface. When someone want's to use a resource, they pass you a handler. Your code provides the resource through their handler method. When the task is compelete, you return the resource to it's pool or whatever. This has some other benefits that can be very useful but aren't pertinent.

Paul Brown

Posts: 284
Nickname: paulrbrown
Registered: Dec, 2003

Re: Myths of Memory Management Posted: Sep 8, 2005 11:26 AM
Reply to this message Reply
Pointers to
null
in Java? What pointers would those be? The best practice for Java is to work with the garbage collector and not try to second guess it...

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Myths of Memory Management Posted: Sep 8, 2005 11:36 AM
Reply to this message Reply
The value of a refence is a pointer in Java. That's why we have NullPointerExceptions

From the JLS (version 2)

"An object is a class instance or an array.

The reference values (often just references) are pointers to these objects, and a special null reference, which refers to no object."

http://java.sun.com/docs/books/jls/second_edition/html/typesValues.doc.html#106237

Max Lybbert

Posts: 314
Nickname: mlybbert
Registered: Apr, 2005

Phantomly Held References Posted: Sep 8, 2005 12:12 PM
Reply to this message Reply
Thanks James. It's an interesting read. Kind of reminds me of the Perl WeakRef modules, but that's another story.

OTOH, C++ templates remind me of Ada Generics, C++ classes remind me of Simula, ... (i.e., not that there's anything wrong with that. ...)

So, thanks again.

Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: Phantom References? Posted: Sep 8, 2005 12:40 PM
Reply to this message Reply
> There is a simple approach to dealing with this kind of
> issue in Java. You create a Handler interface. When
> someone want's to use a resource, they pass you a handler.
> Your code provides the resource through their handler
> r method. When the task is compelete, you return the
> resource to it's pool or whatever. This has some other
> benefits that can be very useful but aren't pertinent.

More and more, I think that divide between languages that provide closures and ones that don't is as significant as the divide between static and dynamic languages.

Max Lybbert

Posts: 314
Nickname: mlybbert
Registered: Apr, 2005

Re: Myths of Memory Management Posted: Sep 8, 2005 2:53 PM
Reply to this message Reply
> finally blocks don't really have anything to do with GC,
> so I'm not sure where you are going with this.

I have to say that my gripes w/ the finally clause are probably based on my C++ background.

In C++, "exception safe code" falls under various requirements. The basic guarantee is that if an exception is thrown, the programmer knows (a) data structures are left in a valid state, and (b) the algorithm doesn't leak resources (such as memory, but can include file locks, sockets [which come from a finite pool], disk files, etc.). The strong guarantee is that (a) the algorithm either completes successfully or has no effect, and (b) the basic guarantee also holds. The nothrow guarantee is that (a) no exception can be thrown, and (b) the strong guarantee holds.

In C++ you have to go through various acrobatics to avoid leaking resources. The basic heuristic is that you (a) allocate all resources through automatic variables (so that throwing an exception automatically calls the destructor), and (b) watch when you call functions/methods that can throw exceptions so that you know when things might get interupted, and order your calls accordingly. No, it's not fun, and yes, there's got to be a way to offload much of that to the computer. I've got a few ideas, but they're outside of the compiler (something like an exception checker, similar to Spin [http://www.cs.bell-labs.com/sys/doc/spin.html ], the thread-safe checker).

My point is that Java's implementation does NOT properly offload that work to the computer. In particular, the finally clause actually causes more tedious work than the C++ acrobatics. Simply put, the Java GC handles the leaking memory part, but since there's no way to know WHEN finalizers are called, you can't use them to handle the leaking of other resources (remember the lock example?). That must be done by hand, and must be done in a finally block. This means you end up typing the same clean up code in several finally blocks (and you must make sure those finally blocks are up to date), instead of once in the destructor.

> I see where you are going with the autopointers. However,
> I prefer the delayed cleanup method.

To be honest, I'm a "whatever floats your boat" kinda guy. I've been looking into linking things to incremental garbage collectors (such as the Hans Boehm one included by default in GCC) for "litter collection" (i.e., to catch my mistakes). My main point is that I'm not all that find of finally blocks.

Max Lybbert

Posts: 314
Nickname: mlybbert
Registered: Apr, 2005

Maybe there's a Java workaround Posted: Sep 8, 2005 3:51 PM
Reply to this message Reply
OK, thinking things over a little, perhaps there's a way to offload some cleanup in Java.

Before starting a try ... catch ... finally block, declare a few arrays that will hold references to things that must be cleaned up (eg., a locked_files array). Push references to things that need cleanup into those arrays as they are used in the try block, and then iterate through them in the finally block while doing the cleanup.

This at least reduces the burden of keeping track which variables to clean up, although it's still hand-rolled clean up.

Vesa Karvonen

Posts: 116
Nickname: vkarvone
Registered: Jun, 2004

Re: Maybe there's a Java workaround Posted: Sep 8, 2005 7:17 PM
Reply to this message Reply
Are you thinking of something like this?

import java.io.FileWriter;
import java.io.Writer;
import java.util.LinkedList;
 
class Cleaner {
  public abstract static class Action {
    public abstract void execute() throws Exception;
  }
 
  public final void add(Action action) {
    assert null != action;
    actions.addLast(action);
  }
 
  public final <W extends Writer> W close(final W writer) {
    assert null != writer;
    add(new Action() {
        public void execute() throws Exception {
          writer.close();
        }});
    return writer;
  }
  /* ...and for other common kinds of resources... */
 
  public final void clean() throws Exception {
    while (!actions.isEmpty())
      actions.removeLast().execute();
  }
 
  private final LinkedList<Action> actions = new LinkedList<Action>();
}
 
public class CleanerExample {
  public static void main(String[] args) throws Exception {
    Cleaner cleaner = new Cleaner();
    try {
        FileWriter writer1 = cleaner.close(new FileWriter("output1.file"));
        writer1.write("The file will be closed at the end of block.\n");
 
        FileWriter writer2 = cleaner.close(new FileWriter("output2.file"));
        writer2.write("The other file will also be closed.\n");
    } finally {
      cleaner.clean();
    }
  }
}

Keith Ray

Posts: 658
Nickname: keithray
Registered: May, 2003

Re: Myths of Memory Management Posted: Sep 8, 2005 9:20 PM
Reply to this message Reply
"This means you end up typing the same clean up code in several finally blocks (and you must make sure those finally blocks are up to date), instead of once in the destructor."

This becomes much less necessary in languages that support blocks or closures. Those eliminate a lot of what would be duplicate code in Java. For example, Instead of [pseudocode]


var file;
try
{
file = open("name");
data = file.read();
}
catch( ex )
{
print("open or read threw an exception", ex );
}
finally
{
file.close();
}



you use (and re-use) blocks of code [pseudo-smalltalk syntax]



// no error handling:
file read: "blah" into: [ :x | data = x ] onError: nil ];

// with error handle
file read: "blah" into: [ :x | data = x ] onError: [ :ex | print("open or read had an error", ex") ];



where file's "read:into:onError:" method is given one or two blocks of code, and invokes the first if the open and read are successful, and the second if the open or read fails, and of course closes the file as well.

Flat View: This topic has 81 replies on 6 pages [ « | 2  3  4  5  6 | » ]
Topic: How Much Profit is Enough? Previous Topic   Next Topic Topic: QA and TDD


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2017 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us