There’s a really solid, tried and true, Internet wide messaging queue. It’s been around for decades and has first class support in Ruby. What is it? Email. That’s right, email.
Before you run screaming from the room, think about it. Email works. It’s robust. It has fallback and retry strategies. It’s standardized and well understood.
In this two-parter, we’ll look at how you might use email to get data into your app. Let’s start with a story:
A million years ago, a friend on mine created a better version of the UNIX “from” command. It added a few useful features:
- Show only unread messages.
- Filter based on From.
- Filter message based on To/Cc (Handy for showing only messages to me).
Handy, if not earth shattering. I used it forever, I liked to quickly see what new mail I had, without getting sucked inside a mail client. I especially liked using it to so only new mail to me. I had an alias:
However, subj only knows about mbox format and can only do it’s thing if you are sitting on the server where your mail is spooled. In the Modern Era, my mail, like yours, live on a distant IMAP server I rarely log into. “subj” time has passed.
But say we wanted to reproduce it’s functionality (~500 lines of C) with Ruby? What would it take?
Not much it turns out, Ruby ships with a full-featured IMAP library. It’s straight forward.
First, we need to connect to an IMAP server:
ssl turns on TLS encryption. Don’t think, just use
Next, we need to log in:
1 2 3 4 5
authenticate method raise an error if it fails, hence the
rescue. There are a handful of IMAP authentication methods, but
typically PLAIN or LOGIN is what you want.
Once we are connected, we need to select the IMAP folder we are interested in:
#examine opens the folder read-only, which is good for our
purposes. If you are going to make changes, use
Next we need a list of message, but first an aside about what it means to delete a message in IMAP. There are three possibilities:
- Remove the message immediately.
- Move it to “Trash” folder.
- Flag it as deleted, hide it in the current folder, and show it in a virtual “Trash” folder.
The first two options are relatively expensive on the server side. And the first is user unfriendly, there’s no undo.
As a result, most implement the third option. This means that deleted messages remain in the folder until “the trash is emptied” which, under the hood, means that the IMAP server is told to permanently delete them.
Given that our INBOX almost certainly contains deleted messages, we will need to filter them out. The simplest way is to get our list via a search.
['ALL'] would get everything, including the deleted messages.)
Another option for our purposes is
which effectively means “return any message that has not been marked as read”. (Note, if you read up on IMAP, you might be tempted to try “RECENT”, which means the message has not yet been access by any client, this is not what you want.)
Now we have a list of message ids, let’s get the messages or, more precisely their headers. We ask the server for all of the “envelopes” for our list of messages.
The response is an array of
Net::IMAP::FetchData, which is just a
wrapper around a Hash containing the envelope we ask for:
The envelope it self is an
Net::IMAP::Envelope, a Struct that allows
us to get at standard headers like To:, From:, and
Headers that contain email address are always arrays, even when
there can be only one,
like From, and are another, rather unhelpful type, a
Net::IMAP::Address. All other headers are simply Strings.
So, the subject is easy, it’s just
envelope.subject. The from is
more of a pain.
Net::IMAP::Address doesn’t actually expose the full
email address, we have to compose it:
1 2 3
For the purposes of
subj I’m just going to display the name or the
email if there is no name:
Let’s put it all together:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
Note that I’m displaying only new, UNSEEN, message. Display all messages or, better still adding a command line flag to toggle between all and new, is left as an exercise for the reader.
Next time we’ll take this new found knowledge and build a job queue.