How to A/B Test Your Ecommerce Website: An Infographic Guide

When it comes to A/B testing your ecommerce website, I’m sure you’ve heard enough about sample sizes and statistical confidence for a lifetime.

If you’re looking for a comprehensive resource that you can use on your site today, then you’ll love this infographic.

It’s a “best practices” guide that will guide you to producing the A/B tests you need in order to increase conversions:

[share-image link=”http://bluestout.com/resources/infographic/” image_src=”http://bluestout.com/wp-content/uploads/2016/07/AB-testing-infographic-for-ecommerce-Blue-Stout.png” height=”160″]AB testing infographic for ecommerce BlueStout[/share-image]

Here’s my take on the A/B testing insights from the infographic:

1. Determine Your Conversion Goal

Before doing any type of testing you need to determine what you’re going to call a ‘conversion’.

Since the ultimate end goal of an A/B test is determining what converts better, you have to establish what a conversion means to you before beginning.

Of course you want your A/B test to be as accurate as possible, and clarifying this before you start testing can save you time and money.

2. Determine the Significance of Your Results

Most significance calculators are built on the assumption that sites operate at just 1 price point, and the number of conversions and visitors are both in the same ‘unit’ of measurement.

For the majority of you: that’s not the case. The number of your visitors probably never aligns 1:1 with conversions.

So you need to use a metric that is inclusive of price variations. Use this formula:

Annual Incremental Revenue Added = (Monthly Traffic x New Conversion Rate x New AOV x 12) – (Monthly Traffic x Old Conversion Rate x Old AOV x 12)

This will tell you how much your conversion has contributed to increasing revenue, giving more meaning to what a conversion really is to your business.

3. Determine the Sample Size

Lots of clients have questions surrounding sample size: how large should your sample size be to arrive at a significant result?

The short answer is don’t test on a small sample size.

You need to estimate your size based on your current/baseline conversion rate, your desired detectable effect (what you want to see happen in your conversions – usually a % value), and your desired statistical confidence.

I’d recommend using this calculator from Optimizely.

4. Determine Test Duration

To eliminate any external factors like a spike in traffic from a holiday (and similar outliers), you should run your tests for a minimum of 1-2 business cycles.

Your test duration should factor in what’s happening each day of the week on your website, account for visitors who may return to your site at a future date, and account for any other various different traffic sources.

If you need help figuring this out, I’d suggest using this calculator from VWO.

5. Determine Statistical Confidence

Many clients have expressed doubt in the accuracy of the statistical confidence given to them from the software they’re using (which usually is around 90% by the way).

A good rule of thumb is to run your tests for at least 4 weeks (7 days a week) to achieve confidence.

And always conduct follow up tests if you’re in doubt.

You can only be sure about the permanence of your results by continuing to test, and then comparing the results with the original test outcome. If the result is still the same, you’ll have more confidence in the stability of your results.

By ensuring you’ve tested for a long enough time period, you reduce the margin of error in your A/B tests.

So don’t be in a rush. It’s tempting to turn off the underperforming variation, but you have to hold out to make the best-informed decisions for your business!

6. Re-Test Against Original Control After Picking Winner

After you’ve completed your initial A/B test, retest the results after 6-12 months.

This will ensure that the results you obtained are valid and not a result of any external factors (like spiked holiday traffic, for example).

For this re-test, it’s important that you use your same A/B testing software to re-test against the control.

Use copies of the original control version and the winner for the retest, so you benchmark the results against the right base and minimize any margin of error.

Start Diagnosing Areas To Test On Your Ecommerce Website

Before you begin testing something on your ecommerce store, you need to identify the gaps in your conversion funnel.

Where are customers dropping off?

Why are they leaving the product page?

Why are they abandoning their carts?

I’m going to help you focus in on your gaps.

We’ve got a FREE conversion training: “How to Grow Ecommerce Sales, Conversions and Profits WITHOUT Spending More on Marketing” that can help you identify how you can use your existing website to improve your conversions. You’ll learn:

• How to get more conversions from your traffic, without having to overhaul your website.
• A simple system for increasing your sales AND profit margins, without spending more money on ads or SEO.
• A breakdown of the exact conversion mistakes you are making and how to fix them.
• Our strategy for PREDICTABLY growing your business WITHOUT investing outside capital.
• BONUS: A map of our top-performing “pre-purchase” funnel and an ROI Calculator from Referral Candy!