Split Testing for Ecommerce Email Campaigns

What Ecommerce Emails Should Test

Ecommerce email campaigns come in several distinct types, and each has different variables worth testing. Promotional emails, abandoned cart sequences, post-purchase follow-ups, and re-engagement campaigns all have unique characteristics that affect what you should test and how you measure success.

Promotional Campaign Tests

For promotional emails announcing sales, new products, or seasonal offers, the most impactful tests are subject line format (urgency vs. curiosity vs. direct offer), discount presentation (percentage off vs. dollar amount vs. "save up to"), and product selection (featuring one hero product vs. a curated collection). The success metric should be revenue per email sent, not just open rate or click rate, because a high open rate means nothing if those opens do not convert to purchases.

Abandoned Cart Tests

Abandoned cart emails are among the highest-converting email types, making them prime testing candidates. Test the timing of your first cart reminder (1 hour vs. 4 hours vs. 24 hours after abandonment). Test whether including the abandoned product image versus just the product name affects recovery rates. Test whether adding social proof like reviews or popularity indicators increases the completion rate.

Post-Purchase Follow-Up Tests

After a customer buys, test what drives the next purchase most effectively. Test whether cross-sell recommendations based on the purchased product outperform general bestseller recommendations. Test the timing of your first post-purchase email. Test whether a review request or a product recommendation generates more long-term engagement.

The Revenue Per Email Metric

Many ecommerce teams make the mistake of optimizing for open rates or click rates rather than revenue. An email variation might get fewer opens but higher revenue per open because it attracts more purchase-ready clicks. Always measure revenue per email sent as your primary success metric for ecommerce tests.

Revenue per email sent is calculated by dividing the total revenue attributed to each email variation by the number of emails sent for that variation. If variation A was sent to 5,000 people and generated $2,500 in revenue, the revenue per email is $0.50. If variation B was sent to 5,000 people and generated $3,200, the revenue per email is $0.64. Variation B wins even if it had a lower open rate, because it produced more money.

Seasonal Testing Strategies

Ecommerce businesses have natural testing rhythms tied to their sales calendar. Use lower-volume periods to run tests that inform your high-volume campaign strategy. If you learn in September that urgency-based subject lines outperform curiosity-based ones by 15%, apply that learning to your Black Friday campaigns where the stakes are highest.

During peak seasons like holiday shopping, your list is at its most active and your traffic is highest. This means tests reach statistical significance faster. Take advantage of this by running quick, focused tests on the elements that matter most: subject lines for your promotional sends, product layouts for your campaign emails, and discount framing for your offers.

Testing Across the Customer Lifecycle

New customers and repeat customers respond to different messaging. Test whether new customer emails perform better with educational content (how to use the product, care instructions, styling guides) versus immediate cross-sell offers. Test whether repeat customers respond better to loyalty-based messaging (exclusive access, early sale entry) versus standard promotional content.

Segment your tests by customer stage whenever possible. A test result that says "urgency works better than curiosity" might actually mean "urgency works better for repeat customers but curiosity works better for new ones." Aggregate results can mask important differences between customer segments.