This post originated from an RSS feed registered with Ruby Buzz
by Christian Neukirchen.
Original Post: On Mail
Feed Title: chris blogs: Ruby stuff
Feed URL: http://chneukirchen.org/blog/category/ruby.atom
Feed Description: a weblog by christian neukirchen - Ruby stuff
I have just aborted a week-long oddisey of finding a good mail client
that can deal with pretty big IMAP mail boxes. (Of course, this is
because Gmail now does IMAP, which is a really nice idea.)
They all suck, are too slow, or both.
Before everyone now comes shouting, “Bah, just use Mail.app”, these
are my requirements for this use case (I’ll try Mail.app again as
soon as I get Leopard, and will tell you how it breaks down then…):
Must work over SSH without X, which means console/line based
Must support IMAP well (bye mutt)
Must support big mailboxes (100k messages and more)
Must be open-source
Should be unixish in some way
One more word about the big mailboxes: I do not need to see all
headers at all times, and if the mailer can operate fast on a subset
(say, the last 2000 mails), this is fine enough. I’d rather use a
quick program with a smaller working set than a slow program which
shows all messages of this century. In fact, this is recommended by
the IMAP client coding
HOWTO. (I
think only Pine does it, and that didn’t work too well either.)
I tried all of these: Mutt (the only program
with nicer source than interface),
Pine (works well with smallish IMAP
boxes, but breaks down with my 172k-ruby-talk box),
Nail (same, and the UI
sucks), and Cone (I gave up
compiling).
As you will notice, most of these clients are written in C. It’s
probably because of their age, but really it is a waste of time.
Mailers written in C are the worst you can imagine, except for mailers
written in C++, which share the issues but take ten times as long to
compile.
Mail clients written in C usually means that they are
noncustomizable/untweakable (Pine) or they use hacky configuration
language (Mutt). Scripting languages exist, and given proper
algorithms and datastructures (which you need anyway, if you want to
make a scalable mail reader), they are fast enough to do anything mail
related, while still being proper, portable languages for extensions and
plugins. (There are a few mail programs written in Perl, but they
don’t seem successful.)
I’m back to Gnus/fetchmail/Maildir now. It can support IMAP, and
works relatively fast (see below for reason), but searching for new
mail is slow, and blocks my complete Emacs (I may end up just starting
two instances…).
And Gnus, I now realize, is a fine mail reader.
This probably is because it was made as a news reader, and the virtues
of a news reader are what counts in my case: Gnus asks you how many
messages to load (usually just the unread ones), and works zippy with
them then, but even with 10k of messages, it still *is* usable.
Furthermore, it has a seriously cool feature: expiring. You don’t
delete mail (I never do that, anyway, which is why my mailboxes are
getting so big), but you expire it, and if the mail is expired and
older than a week or so (configurable), it either deletes it, or moves
it into a different mail box, or does anything you tell it with elisp.
I now use this to make monthly mbox-archives of ruby-core and
ruby-talk, since HFS+ doesn’t really like 150k+ files in a directory.
Which gets me to a side note… one of my first
posts
on this blog was about mail storage formats, and Maildir essentially
was the winner. Maildir is rock-solid, but tools like rsync or
rsnapshot really have to work hard to back them up, if they reach a
decent size. And many file systems (still no ZFS in OS X) slow down a
lot. I’d like to propose a Multimaildir format that stores mails like
Git in Maildir/000/999 and the next one in Maildir/001/000 instead of
stuffing them all into one directory. Should be pretty easy to do,
and makes everything faster. (You also could move your old mail
easily, just move Maildir/000 somewhere else (O(1)), instead of
globbing like hell (O(n)).) End of sidenote.
I also had a closer look at MH, which always fascinated me. There is
a good O’Reilly book on it available online, and I like how it was
designed. Very unixish. I can’t really imagine using it, though. (I
read/skim lots of mailing lists, and apparently MH doesn’t thread, and
I guess it’s just too slow to vgrep a summary and read the few
interesting posts. Maybe with a really well-tuned Zsh
setup, with keybindings and
everything.)
For half a day I glimpsed the idea of writing an IMAP-based MH. In
fact, such a thing, written in Python, exists as
MHI. But I’m not sure I really want
to use it, and it would suck to spend a lot of time to reinvent the
wheel and not even drive with it.
So, I thought about my needs, did a bit of research with antique mail
clients. (Did you know jwz used Netscape 3.02 for a looong
time to read his mail? Not
console based, unfortunately.)
Now, I’m sucking it down and will write my own client, and I’ll use
something one rarely sees on Unix: lets call them “interactive
non-screen based interfaces”. Actually, that’s wrong, because you
probably use the shell every day. Mutt and Pine take all your screen
and are nontrivial to implement (you need to do a pager, and all the
curses stuff, yeech), while mail/mailx/nail are totally-line based and
you need to end each command by pressing return (which is one key too
many for lots of mails).
I think we should do it like this: make a small library that provides
a few widgets, like “line picker”, “item picker”, “line reader”, add
an Emacs-style (or, almost easier, vi-style) keyboard map system to
dispatch between these and make all input interactive (cbreak). Many
apps on ITS worked like that, and it’s pretty comfortable to use,
while still fast, flexible and text-based.
I’m pondering making a Gnus feelalike in Ruby based on this scheme,
with the following very limited function set: just IMAP (but that
well), and all configuration by editing/adding Ruby, threading like
jwz does it, and just the
stuff I need. One should have usable results within a week, who
knows.
I could end up in the history of men by writing the first mail reader
that doesn’t suck. :-P
Previous Topic
