I've been working on a newsletter project for my company.  The newsletter software is a product I've built on top of SilvaNews, built on top of Silva, built on top of Zope.  It has the capability of publishing to the web and publishing customized versions of the newsletter, in multiple formats, to a list of email addresses (link to online version, text-only, html or text, pdf.  The addition of a new newsletter required some updates to the code.  The update I'm talking about here is including images in the html email, with the images embedded in the email.

There are many pros and cons to embedding images, as opposed to linking to images on a website.  I'll not go into them here.  There are a number of things you need to understand before embedded images will actually work.  I was unable to find even one site containing all of the necessary information, so I'm outlining it here.

A bit more background: the product is developed in python, and uses python's standard email package to generate the MIME html email.

MIME format

The first important item is the ordering of the MIME parts.  The main MIME container needs to have a content-type of: multipart/related (see rfc 2387).  Included in the main container's content-type header should be a 'type' parameter containing the content type of the display message.  In our case it is the multipart/alternate container which holds the text and html versions of the newsletter.

The newsletter part should be composed of a text/html part and a text/plain part (the html and plain versions of the newsletter).  Thse should be in a multipart/alternative container. After the newsletter part, include the images.

The final message outline should look something like this:

• content-type: multipart/related; type="multipart/alternative"
  start: "<newsletterpart>"
  

  • content-type: multipart/alternative
    content-id: <newsletterpart>
    

    • content-type: text/plain
    • content-type: text/html
  • content-type: image/jpeg
    content-disposition: inline
    content-description: image title 1
    content-id: <image1>
    
  • content-type: image/jpeg
    content-disposition: inline
    content-description: image title 2
    content-id: <image2>

The start header lists the content-id of the display message.  This is optional, as the rfc specifies clients must use the first contained message if start isn't present.

About content-id

The content-id header is a unique identifier wrapped in angle-brackets.  The rule of thumb is that is should be a 'random' number @ domain (e.g. <a234bczWq0@bethel.edu>).  It should be very random, because it needs to unique even when the message is forwarded as part of a larger message.

Embedding images in html part

To reference embedded images in the html part, you use the content-id of the embedded image.  Like this:
<img src="cid:image2" />

Beware though, not all html email clients display embedded images properly.

Email Client issues

The email clients I tested this format in were MS Outlook Express, Thunderbird 1.0.6/Win, GMail, and Kmail 1.7.1/ KDE 3.3.0, and Mail.App/OSX.  The results were mixed:

MS Outlook Express

It doesn't allow you to choose between the alternative parts.  I had much difficulty getting the images to display, but once I had the cid thing figured out, the html displayed properly.  It appears the main container can be either multipart/related or mulipart/mixed.

Thunderbird 1.0.6/Win

Like Express, it doesn't allow you to choose between the alternative parts.  In fact, the html part will not display if you change the order from plain, html to html, plain.  This appears to be a bug that has been around for awhile.  But, given the format I've outlined, it correctly displays embedded images.

GMail

Prefers the text/plain part if it exists.  It doesn't allow you to display the html part.  That is slightly understandable, given the web-based nature of the program.  It does display the embedded images though. It will do its best to render a text/html email, if it doesn't have a text/plain alternative.  That really bothers me, since it has demonstrated it can display html.  Why not give the user more control over the display?  Grrr!  (it seems nearly all of the email clients I tested have this problem)

Kmail 1.7.1/KDE3.3.0

This is my preferred email client, however there is no support for embedded images.  It will display images that reference images located on the web.  It dislays the embedded images at the bottom of the display.  I really like this client, because out of all of the ones I tested, it was the ONLY one that has a tree-view of the email, allowing you to select any of the included messages for display.  A VERY nice feature.  Kmail doesn't recognize the multipart/related content type.

Mail.App/OSX

Finally, mystery solved!  While working on the previous newsletter (e-announcements) I was never able to get the html part to display.  This client appears to have the same issue with ordering that Thundrebird does.  Other than that issue, it displays the html version correctly, with the embedded images in the html.  Like all other clients tested, except kmail, it doesn't allow you to navigate/choose the mime part to display.

Python/Zope-specific notes

Setting the 'type' parameter:

msg.set_param('Type','multipart/alternative')

Unicode issues:

This one took me awhile to figure out.  In Silva, the content is returned as a unicode string.  I had to find a couple key spots in the rendering chain and make sure the content was actually unicode.  Apparrently, it isn't always the case if you're using page templates.  When you call msg.as_string() to render the entire email message, it has problems with unicode (utf-8) strings that don't map to the default character set (in our case, ascii).  To solve this issue, you need to convert the unicode string back to a properly-encoded bytestring prior to setting it as the payload:

data = data.encode('utf-8')
msg.attach(MIMEText(data,'html','utf-8'))

Conclusion

This was a battle hard won.  The key points I learned were using/supplying a content-id header on each part, using multipart/related, and the cid: URI.