Measuring E-Mail Campaigns in the New Environment

Open rates, click rates, conversion rates and $/1,000 mailed are the traditional metrics upon which the success of e-mail marketing programs is measured. Are they enough in today's spam-dominated e-mail world?

 Using the traditional metrics as a guide, you may consider testing new subject lines, improving your targeting or providing a more compelling offer to optimize your results. However, the best database management, the best creative and the best one-to-one targeting will not have a lick of impact on your results if your message simply never reaches your customer.

As the volume of spam skyrockets, ISPs, enterprises and individuals are deploying a plethora of filters that block, delete and redirect e-mails to junk/bulk folders. None of these methods is perfect, and an increasing percentage of opt-in e-mails is being mistaken as and treated as spam ("false positive").

Today, 20 percent of legitimate permission-based e-mails are not being delivered to the recipient's inbox. Deliverability, and specifically getting your e-mails into your customer's inbox, is the most important factor driving campaign results today.

Campaign-Related Delivery Metrics

Bounce rates, hard and soft. Your e-mail deployment vendor or your e-mail software typically provides "mailed," "bounced" and "delivered" statistics. Ensure you track the specific reason for the bounce by campaign and by ISP for "hard bounces" (delivery status notifications beginning with a "5" and indicating a permanent error such as an invalid e-mail address) vs. "soft bounce codes" (delivery status notifications beginning with a "4" and indicating a transient error such as the recipient's server is down).

The reason? The incidence of "false bounce reporting" is on the rise as ISPs send delivery error messages indicating a permanent failure when it is not. They do it to thwart spammers who use the delivery status notifications (or lack thereof) to confirm they have a valid e-mail address. For more information on bounce codes, visit: http://rfc.net/rfc1893.html

True delivery or "% received." Standard e-mail campaign reporting provides a "delivered" statistic. However, this is not the true delivery rate as standard SMTP-based reporting subtracts only bounced e-mail to arrive at the delivery rate. However, this figure represents just what was delivered to the ISP. The ISP may now elect via its spam filters to delete some of this "delivered" mail without providing notice to the sender. On average, 10 percent of legitimate mail is being "deleted" by the ISP. There is only one way to track the "% Received": You will need to set up e-mail accounts at all the major ISPs and record what percentage of mail is received or hire a service to do this for you.

% bulkbox and % inbox. The true delivery quantity or percentage of mail actually received by your recipients (described in item No. 2) may be further broken down into "bulkbox" and "inbox." Bulkbox is the mail that has been redirected by the ISP to the recipient's junk, spam or bulk folder. Inbox is the mail that has been allowed to flow directly to recipients' regular inbox folder. These are important metrics, as many recipients do not bother to view e-mails in their bulkbox, and so many marketers consider bulkbox placement the equivalent of non-delivered mail.

Another consideration of bulkbox placement is the negative implications for your brand when your mail is placed in the same folder as spam. Thus, you want to minimize bulkbox placement and maximize inbox placement.

Measuring inbox vs. bulkbox placement cannot be done with the standard SMTP-based reporting provided by most e-mail deployment technology providers. Similar to tracking "true delivery or % received," you will need to hire a seeding service that specifically tracks these metrics for you.

Campaign-Related Marketing Metrics

Open, click and conversion rates. Open rates (the percentage of HTML-enabled recipients who open your e-mails), click rates (the percentage of recipients who click on one or more links within the e-mail) and conversion rates (the percentage of recipients who take the prescribed action -- e.g., register or place an order) form the basis of most marketing analysis. However, marketers historically have calculated these statistics using the "delivered" quantity provided by their standard SMTP-based reporting as the denominator.

This does not let you accurately compare different marketing efforts or performance by ISP, as it does not account for the amount of mail deleted by the ISP or redirected to the bulkbox. When calculating these statistics, you also will want to consider measurement and evaluation on three levels:

· Use "delivered" as the denominator.

· Use "true delivery" or "% received" as the denominator.

· Use "inbox" as the denominator.

Spam complaints, by campaign and ISP. Spam complaints are the number or percentage of individuals who complain that the e-mail you are sending them is spam. Typically, recipients will reply to the e-mail you send them. Most marketers today have what is called an "inbound mail handler." This is the system that processes inbound e-mails to your reply-to e-mail address.

More sophisticated systems automatically segregate these e-mails by content (e.g., unsubscribe requests, spam complaints, customer service requests) while less-automated systems direct the e-mails to a person or group in your company. Either way, it is important to record the number of individuals complaining that your e-mail is spam on a campaign and ideally on an ISP-within-campaign basis.