Weekly Release Notes: More Consistent Performance, Testing Templates, and More

Friday was a little crazy, but we still have plenty of things that went live this week. Let's get on with it.

More consistent and better sending performance

As part of the effort to build our new status page, we created a large international set of benchmarks (expect another blog post on this later). These tests send an email from every continent through our servers once per minute and measure how long the different parts of delivery take. We've always monitored sending performance through Mandrill in the effort to make email delivery fast, but these benchmarks showed how spikey our delivery times could be during periods of high load and contention. So this week we strapped on our profilers and went to war to trim some of those edge cases. A few days later, battered and exhausted, we have some results to share.

You can see the dramatic change in the graph above, but here are some hard numbers. On average, emails will be sent with about the same speed before and after (the median email delivery time was about 750ms before the changes and is about 650ms after). The big changes were in the distribution of the spikes. Before you could see spikes of up to three or four seconds, which now are being flattened to a much more reasonable 1.2 or 1.3 seconds. For higher volume and bulk senders, this can have a dramatic impact on your total delivery time, even if the median didn't move much. There's always more that can be done here, so expect some more movement on this in the coming weeks.

Template testing

Mandrill templates continually maintain two versions of themselves - a draft version and a published version. This way you can iterate and collaborate on a template without impacting the emails being sent. This was a good idea, but it was missing an obvious feature - you could preview the template content on a web page, but you couldn't preview the template in an actual email. Starting this week, we fixed that by adding a "send test" feature to templates. It's a small change, but it should make the experience of editing templates vastly easier.

Webhook batch retrying

It can be really frustrating when a webhook request is failing and you don't know why. This is particularly true if you have inadequate logging on your server or you can't get access to the logs to see what's really going on. Previously, Mandrill didn't help this process, since we would retry failed webhook calls at exponentially increasing intervals. That's good for not hammering a failing server, but bad for debugging. Starting now, you can tell Mandrill to retry a failed webhook call immediately. This way you can activate a debugging mode or more actively monitor requests going through the system. You can do this as many times as you like until the problem is fixed, though failing batches will still expire and be deleted after about three days.

More information about rejections

When an email address hard bounces or unsubscribes or marks a message as spam, we add it to the rejection list so we know to stop sending to the address. You can access your rejection list through the web interface and the API, but sometimes it was difficult to trace exactly what message caused the problem - particularly if the original message was sent a long time ago. Starting this week, we are adding more data to rejection listings in the interface and the API, giving more details on the exact cause of the rejection.

Custom rejection blacklist entries

Mandrill will automatically add rejection blacklist entries for bouncing or unsubscribing users, but sometimes you want more direct control over the blacklist. Starting this week, you can add your own entries into the rejection blacklist that will never expire. This is perfect for those recipients that contact you directly asking to be removed, or even if you want to test rejections with your own email addresses. Don't worry - adding and removing custom rejections like this will have no effect on your account's reputation.