Introduction

In the previous article we discussed the idea of a dynamic email drop box. We took a look at some existing services that are offered that piggy back a feature like this. I then defined a quick list of all the tasks that we would need to perform to create an email drop box.

In this article we will take a look at the first half of our task list. We will discuss how to set up your pop3 server so that it can support the drop box feature. We will then take a look at what is required for your application to communicate with your pop3 server. And finally we will go over some of the basic email parsing that is required to get us to the making the actual features.

Let’s get started!

POP3 server configuration

In order for our entire application to work we must first set up an email address that can collect all of the email for a specific domain on behalf of our users. This address might look like dropbox@andrewsiemer.com. The key here is that the users won’t be sending their emails to this address directly (although they could if they so chose). They would instead be sending their communications to VeryPrettyGirl123@andrewsiemer.com or ScottsWidgets@andrewsiemer.com. These email addresses won’t physically exist on our pop3 server and so it will be the duty of dropbox@andrewsiemer.com to accept them up for later distribution.

To set this email address up appropriately we need to take a look at setting up an email alias. An alias is a recipient mapping rule. You might think of this as a regular expression for recipient email addresses. Any email that comes into the pop server that matches the specified rule will get routed according to that rule. In our case we need to create a dropbox@andrewsiemer.com email account. Then go to the alias’ configuration for that newly created account and add a rule for *@andrewsiemer.com (or whatever is appropriate for your type of pop server to collect “ALL” unknown emails delivered to that domain). This means any email that comes to the andrewsiemer.com domain that doesn’t match a physical account on the server will get sent to dropbox@andrewsiemer.com.

Once you have your dropbox@YourDomain.com email address created, test it by sending an email to someaccount@YourDomain.com. Verify that this email was indeed delivered to your dropbox account. If it worked, send 4 or 5 test emails to various accounts so that we have something to code against in the next section.

POP3 client

I am not sure why tools don’t exist in the .NET framework to interact with a pop3 server in a clean fashion (as they do for sending mail through an SMTP server). While you can buy many widgets that interact with a pop3 server I figured we may as well address the need ourselves directly. A quick Google search of “C# pop3” will give you loads of tutorials on how to connect to and interact with a pop3 server as well as discussing the basics of pop3 interactions using telnet. However, none of them appear to really provide you with a good framework or wrapper to hide the complexities of communicating with your pop3 server. These communications after all rely directly on RFC 1725 – or simple telnet commands via TCP with your pop3 server. Talk about old school! Keep the RFC specification in front of you to help you decipher the responses you get back from your server while coding – it will help you understand things better.

Let’s first get started by creating a directory structure to work in. I like to work with all of my solutions resources (not just code) under a “trunk” directory. Next I would have my “src” (or source) directory where all of my projects go. If I had documents, a database, artwork, or other items those would go under trunk as well.

Then in Visual Studio we will create a C# Class Library project which will live inside the “src” directory. I named my project PopClient and set the assembly name and namespace to be “AndrewSiemer.Email”.

In this project I have created the following class files in the root of the project: ConfigurationService.cs, Extensions.cs, Message.cs, MessageSummary.cs, Pop3Exception.cs, and PopClient.cs. I also added a reference to System.Configuration for future use.

There are some configuration aspects to our program so I am going to start off by introducing our ConfigurationService class. While we could just hard wire some values for now I know that we will need the ability to play with these values down the road. For that reason it makes sense to set up a utility to make configuration quick and painless.

This will be a no frills class that provides two private methods GetNumberFromConfig(string Key) and GetStringFromConfig(string Key). This class allows me a way to centralize all communications with my configuration files in case I need to swap my configuration plumbing out down the road (a wrapper if you will). From this class I expose public methods for each of my configuration keys. The ConfigurationService class is shown in the listing below.

Listing 1: ConfigurationService.cs – ConfigurationService

Once we have this class built we then need to put a configuration file under it. In this case I didn’t simply add an app.config file to my existing project. I added the app.config to my src directory above my project directories. This way all of my projects can have access to the same config file as a linked resource. This will make sense when we build our window service.

Here is what my configuration looks like at the moment:

Listing 2: app.config

Next I have a couple of structures that I defined to help us with moving data around. These are basically data transfer objects (or DTO’s for short). We have a Message and a MessageSummary.

Listing 3: Message.cs – Message

The Message object is used in our application as a container to hold the data that we will parse out of an email message. An email message is basically a large blob of text (for the most part). It has a lot more data in it than we care about so we will eventually get to a point where we will extract what we need from that blob and push it into this message object where we can then more easily interact with the items that we care about.

Listing 4: MessageSummary.cs – MessageSummary

The MessageSummary is a similar DTO style object in that it will carry some parsed data. In this case though we won’t get the data from an email message. Instead we will be getting it from a response from our pop3 server.

Next we will discuss our Pop3Exception class. This is a custom ApplicationException based class that we will use to express possible errors or response messages in our application. While we could have chosen to inherit directly from the base class Exception I chose ApplicationException as it was created specifically for user’s applications to create their own custom exceptions (ApplicationException after all inherits from Exception). One rule of a top notch programmer is that you never try to handle Exception directly and so one should use ApplicationException to build out custom exceptions which can then themselves be trapped in your error handling logic. You will see that I handle the base class exceptions directly throughout my code though – sorry! As it turns out this is a huge battle between purists and everyone else. Take a look here for more information on this subject: http://blogs.msdn.com/brada/archive/2004/03/25/96251.aspx I prefer to minimize flames on silly subjects such as this though. (although the word choice SILLY may cause flames!)

Listing 5: Pop3Exception.cs – Pop3Exception

This is a simple exception that pushes the error message through to the base ApplicationException class.

Finally we will look at our Extensions utility. In this class I defined a PrintMessage method that our Message object can use. This will help us when debugging later so that we can easily print a Message in the command window. Nothing fancy here. This is just a simple extension method that builds a formatted string of all the data in the Message object.

Listing 6: Extensions.cs – Extensions.PrintMessage()

With all of the set up stuff out of the way we can now take a look at the PopClient code. To interact with our pop3 server we are going to be using TcpClient, NetworkStream, StreamReader, byte arrays, and strings. Some familiarity with these would be good but is not required. We will define methods to connect, disconnect, list messages, delete messages, get message counts, etc.

The best place to start with our PopClient is with the general use variables. From top to bottom (in the listing below) you will see the declaration of the TcpClient which is used to connect to the pop3 server. Next you will see our NetworkStream which is used to send commands to the server. Then our StreamReader which we use to read the response from the server. Then we have the _data string which we format our commands in. We then break the commands (_data) into a byte array and place it into _arrData (which is what is sent to the server). Then we have _crlf which is used for formatting our commands and messages. The remaining _popServer, _username, _password, and _port variables are used for getting to and authenticating with the pop server.

Listing 7: PopClient.cs

Then we have the constructor for PopClient. Here we are reading our configuration values from our ConfigurationService (discussed above).

Listing 8: PopClient.cs – PopClient

Next we get to our most important method – the Connect method. As the name suggests we will use this method to connect to our pop server. The first part is probably the most important part. We are instantiating a connection via our TcpClient with our pop server. We do this by specifying our pop servers address and port. We then open up our NetworkStream by calling _server.GetStream(). Next we set up our StreamReader in the same manner as the NetworkStream so that we can receive the responses from the server. We are now set up to send commands to the server and receive data (responses) back from the server. Keep in mind that once you connect, this socket is open until you disconnect or dispose of the PopClient, or the session times out.

Listing 9: PopClient.cs – PopClient.Connect()

Once connected to the server we are put into the authentication state. In this state we must issue the username (USER) and password (PASS) commands to the server so that it can authenticate us and locate our email account.

Listing 10: PopClient.cs – PopClient.Connect()

Once connected and authenticated remember that all your resources are open and waiting for use! You are next put into the transaction state where you can issue any of the other commands such as listing your messages, deleting messages, reading a message, etc. All of these interactions with your pop server are pretty much identical to issuing your username command or password command. For this reason we will not delve into them specifically. Take a look at the attached source code for all of these listings.

Next let’s take a look at working with the server responses and specifically how to parse the data that we are most interested in.

Basic email parsing

Email parsing is not really a terribly tricky business. In reality it is digging into a text response and isolating the information that you are most interested in. In our first example we will look at the data that comes back from the LIST command. The LIST command returns to us a list of all the emails in the system and their size. It doesn’t return the actual email but instead returns the index of each email and the size of that email. The data returned from the LIST command might look something like this:

Listing 11: LIST command’s response

Notice that our response starts off with a “+OK”. You can always check for this to ensure that your response will contain what you think it will. Then notice that the last line of the response is a “.”. You can also test for this to determine the end of your response message (though it is not guaranteed to be at the end of EVERY message!).

The data that we are actually interested in here is the index (the number on the left) and the message size (the number on the right). In order to get to each of these values we need to move past the first response and pass each consecutive response to our MessageSummary parser until we hit the final period. The code for that process looks like this:

Listing 12: PopClient.cs - PopClient.ListMessagesSummary()

With each iteration over our LIST responses we make a call to ParseMessageSummary(readerOutput). This is where we get our first parsing done. In the case of the LIST data this is very easy to dig into. We take the string that is passed in, split it on a space, and read the appropriate index for the value that we would like to get. We then pass that value into an instance of a MessageSummary and then return it to the caller.

Listing 13: PopClient.cs – PopClient.ParseMessageSummary()

The last form of parsing is fairly easy compared to our next set of tasks. It is all pretty straight forward as you will see shortly. Now we can take a look at parsing an actual mail message. Once we have a listing of emails we can then choose to download an actual message. Downloading a message might come in two forms. We might know a specific email message that we are interested in, in which case we would call the GetMessage method. With this method we would pass in the index of the email message we are interested in. Or we might execute the LIST command and iterate through all the email messages on the server. Either way, we have to parse the email messages that are passed down.

Before we discuss parsing in depth let’s take a look at what a raw email message might look like. Notice that it starts with a “+OK” and after that…well that is a whole different article!

Listing 14: Raw email message example

WOW! I don’t know if you noticed but all of the above text was simply to send the following line of text:

Yep.

You may have also noticed that there are some bits of information in there that are fairly easy to get to. The Message-ID, From, To, Subject, and Date are tags that we can test for at the beginning of each new line. Do notice that when there are multiple email addresses in the To field that we have a different type of format to work with! If we had CC’d someone here they would show up too.

The other bit that we will be interested in gathering for the purpose of our drop box application is this:

And this bit:

These are the actual body of the email that was sent. The important parts to watch for are the bracketing pieces of that text. The chunks that starts with Content-Type: text/plain; and Content-Type: text/html;. Both of them end with ------=_....

With that quick text analysis let’s take a look at the actual ParseMessage method. You will notice that the majority of the work is done by iterating through each line of the email message. We look at each line once and only once. We run each line through various tests to find out what we are dealing with and process it accordingly.

Most of the tests are easy in that they sample the first part of the line to look for things such as “Date:” or “Subject:”. There are however other cases where things are a bit more complex. When we locate the “To:” field for example there is possibly the case where we may have more than one email address. Each additional email address comes in on a new line. For this reason when we locate a “To:” tag I set a ReadingTo flag to true to indicate that each following iteration may be an additional email address. Once we get to the next field (I only test for “Subject:” but there may be other insertions available that would need to be tested for) we can turn off the ReadingTo flag and continue parsing for other fields.

We play similar games when it comes to reading the text and html versions of the email’s message.

Listing 15: PopClient.cs – PopClient.ParseMessage()

As we parse the bits of data out of the message that we need we can feed them into our Message object. As there is tons of data in an email message there is much room for improvement for this parser. You can expand the Message object to hold as much of if not all of this data as you see fit. Once the message is parsed appropriately we can then return what we need as a simple to use Message object.

Test harness

Once you have had a chance to dig through the PopClient’s full code listing (in the source code download) and understand all of the functionality that it offers then the next logical step is to toss together a test harness. I generally use a console application to do this. Let’s add a new project to our solution. We will add a C# console application to the “src” directory and name it TestHarness.

Next we need to add a project reference to our new project. Add a reference to the PopClient project. Then open up your program.cs file. In there you can play with your new PopClient till your heart’s content. In my test harness I have the following:

Listing 16: Program.cs – Program.Main()

Notice that in my test harness project I am running through some of the basic commands of the PopClient. I start by instantiating a new PopClient. I then ask it to Connect to my pop server. Then we call ListMessagesSummary and then GetMessageCount. Next I make a list of all the messages on the pop server and print them to the console using the extension method we looked at: PrintMessage. Then I ask for a specific message using the GetMessage method. I have some code there to delete a message but don’t run that over and over again as it kills all of my test messages! Finally I call Disconnect and close the connection to the pop server.

Summary

In this article we took a look at setting up the base framework for our application. We configured our pop3 server to accept email on behalf of our users. We then created a pop3 client framework to allow our application to communicate with our pop3 server. Then we analyzed the incoming email packets and set up some basic email parsing to make working with the received emails a little less painful. With the PopClient completed we created a quick little test harness to make sure that it works as expected.

In the next article we will finish up our email drop box application. We will start by creating some account matching tools. Then we will take a look at how we can support tags in our emails. Then we will take a closer look at how to use those tags to support specific features such as statuses, blog posts, and anonymous user messaging. Finally we will finish up our application by sticking all of this functionality into a window service. See you then!

Resources

• RFC 1725 – Post Office Protocol – Version 3: http://www.faqs.org/rfcs/rfc1725.html
• A good understanding of the pop commands and telnet to interact you’re your pop3 server: http://techhelp.santovec.us/pop3telnet.htm

About Andrew Siemer

I am a 33 year old, ex-Army Ranger, father of 6, geeky software engineer that loves to code, teach, and write. In my spare time (ha!) I like playing with my 6 kids, horses, and various other animals. This author has published 25 articles on DotNetSlackers. View other articles or the complete profile here.

---

Discussion

Subject	Author	Date
Final Installment	Khuzema Kanore	8/17/2009 1:40 AM
Hi, When the final installment is coming up, nice work though. regards Reply | Permanent link
Directory Structure	Eduard Gopp	8/26/2009 9:59 AM
Hello Andrew, i dont look through your description of the directory structure, can you please explain this more precisely? At the moment my structure looks like this: I have a directory, called "PopClientProjekt" in this directory there are two subdirectories (PopClient and PopClientTest) "PopClient" contains the class library and "PopClientTest" is the TestProjekt with reference to the class library Where i now have to put the App.config, so i have no problems to access from each of the two projects? Or how should the directory structure looks like, if the one i use is faulty? Sry for my bad english, i dont use it often ;) Sincerly Reply | Permanent link
RE: Directory Structure	Andrew Siemer	8/26/2009 11:30 AM
I would suggest putting the app.config in the PopClientProjekt directory.  The root of your project and at the same level where your actual class library projects live. PopClientProjekt -->PopClient -->PopClientTest -->app.config Then you can add the app.config as a linked file (in the add window you can click to the right of add button which then becomes a drop down button?? to add a linked file).  Do this in both projects.  This way when you update one file you are updating both projects. - Andrew Siemer asiemer@hotmail.com www.andrewsiemer.com blog.andrewsiemer.com www.dotnetradio.com twitter.com/asiemer twitter.com/dnetradio Reply | Permanent link
RE: RE: Directory Structure	Eduard Gopp	8/27/2009 3:26 AM
Thx, works ;) Reply | Permanent link

---

Please login to rate or to leave a comment.