A Beginners Guide to A/B Testing

One of the great advances to come from marketing online is the easy availability of data to base your decisions on.

The wealth of trackable information on everything you do makes continual optimisation a reality for even the smallest businesses. One of the simplest and most effective ways of doing this is A/B testing or split testing as it’s sometimes known.

What is A/B testing?

A/B testing is the process of testing two variations to see which performs better. For example, you might want to see if people are more likely to click through from your mail-out to your site with a picture of a cat or a dog (I bet dog).

How do I test?

The most important thing is to know what change you’ve made, and what behaviour you’re tracking to measure the effect.

Continuing with the dog/cat mailout, you would simply split your mailing list in 2 (randomly) and send your mail out with the picture of the dog to one half and the cat to the other. Then sit back and watch your click rates.

On-site changes are slightly more complicated, requiring different versions of the same page to be made and the appropriate tracking set up. This will either require the appropriate funnels set up in your analytics, or an A/B testing program that will track it for you. These tools (such as Google Website Optimizer) will also automatically serve each variation on the page randomly for you.

It’s important not to introduce any other variables as these could affect the outcomes. Running A for two weeks followed by B for two weeks won’t give reliable results. Similarly, running multiple variations, although possible with the right tools, is more complicated to track. It’s better to run consecutive A/B tests.

What should I test?

Anything and everything. People’s behaviour can be influenced by everything from the colour of the links to how many times a word repeats on the page.

Have a look at your analytics. Identify anything you think is underperforming. Evaluate what could be putting people off. If you really have no idea, start running some tests on the simplest changes to make.

The main things to consider are:

  • Headlines
  • Calls to action
  • Images
  • Body copy
  • Layout

When should I test?

Continually. Your marketing material can always be optimised further. Make sure you always run tests for a reasonable amount of time to collect enough data to be useful. A/B testing doesn’t happen in a day.

Have you heard of A/B testing before? Do you think it could be put to good use on your website? Please leave your comments below.