One of the great advances to come from marketing online is the easy availability of data to base your decisions on.
The wealth of trackable information on everything you do makes continual optimisation a reality for even the smallest businesses. One of the simplest and most effective ways of doing this is A/B testing or split testing as it’s sometimes known.
What is A/B testing?
A/B testing is the process of testing two variations to see which performs better. For example, you might want to see if people are more likely to click through from your mail-out to your site with a picture of a cat or a dog (I bet dog).
How do I test?
The most important thing is to know what change you’ve made, and what behaviour you’re tracking to measure the effect.
Continuing with the dog/cat mailout, you would simply split your mailing list in 2 (randomly) and send your mailout with the picture of the dog to one half and the cat to the other. Then sit back and watch your click rates.
On-site changes are slightly more complicated, requiring different versions of the same page to be made and the appropriate tracking set up. This will either require the appropriate funnels set up in your analytics, or an A/B testing program that will track it for you. These tools (such as Google Website Optimizer) will also automatically serve each variation on the page randomly for you.
It’s important not to introduce any other variables as these could affect the outcomes. Running A for two weeks followed by B for two weeks won’t give reliable results. Similarly, running multiple variations, although possible with the right tools, is more complicated to track. It’s better to run consecutive A/B tests.
What should I test?
Anything and everything. People’s behaviour can be influenced by everything from the colour of the links to how many times a word repeats on the page.
Have a look at your analytics. Identify anything you think is underperforming. Evaluate what could be putting people off. If you really have no idea, start running some tests on the simplest changes to make.
The main things to consider are:
- Headlines
- Calls to action
- Images
- Body copy
- Layout
When should I test?
Continually. Your marketing material can always be optimised further. Make sure you always run tests for a reasonable amount of time to collect enough data to be useful. A/B testing doesn’t happen in a day.
Have you heard of A/B testing before? Do you think it could be put to good use on your website? Please leave your comments below.
Image Credit: n_yfe
Hi Jamie, thanks for sharing your introduction on A/B testing a good write-up. I want to suggest two things.
1) Try significant changes in A/B tests not just change a headline change the whole concept first and make it dramatic. This will give you the best insights into what works and not. Later on focus on the small changes.
2) Give our tool a spin. One code to add to your entire site and you will test every week something new. Its actually fun. http://www.reedge.com
Dennis van der Heijden recently posted..Cross Domain A/B Testing
Hi Dennis, nice to hear from you. Thanks for your comments and the link to your site – I will certainly take a look.
Hi Jamie, here another example of a more dramatic change with >100% increase. http://37signals.com/svn/posts/2991-behind-the-scenes-ab-testing-part-3-final
Lovely test with a good writeup.
Dennis
Dennis van der Heijden recently posted..Cross Domain A/B Testing
Thanks for sharing that Dennis, it’s a very interesting case study; certainly food for thought.
thanks, plain english and easy to understand. i have been hearing bout this for a long time. what amount of traffic(daily hits a day) do you need to be getting a day to consider the data more acurate?
Tim
Hi Tim, it’s hard to put an exact figure on it. If your site gets low volumes of traffic then you’ll want to run the A/B test(s) for longer to build up a more accurate picture. The more visitors you have the more accurate the results will be.