When it’s time to improve a user’s digital experience, many things can be done. Among these things, one can do what is known as A/B testing. In the following article I will explain what is A/B testing exactly and what are the frequent A/B testing mistakes.
What is A/B testing?
First, A/B testing is a way of comparing the performance of different web pages between them. For each page or content, two versions will be created: version A and version B. There will be only one thing different on each version. For example, the Contact us button will be red on page A and blue in page B. Users will then land on either version A or version B. Success of a version is determined by the purpose of the website. For instance, a foundation will track how many people actually click on the button to donate.
A/B mistakes
Nobody wants to waste time working on an experiment that turns out full of weaknesses. So, let’s dive in and learn more about 3 common A/B testing mistakes!
Testing more than one aspect
Testing more than one element at the same time is a big mistake! By doing so, it becomes impossible to pinpoint exactly what change worked. Is it the red color or the fact that the button is now at the top of the page? Hard to tell… Moreover, A/B testing is designed to test simple elements such as new headlines, new images, new layout, new pricing, etc. Testing one change at a time is simple and avoids you a classic A/B testing mistake.
Not letting enough time
With this kind of test, it is really important to let it run for at least a couple of weeks. Ideally, to steer clear of this A/B testing mistake, the experience should last between 2 to 4 weeks. Less than that, the results won’t be accurate. In fact, in early beginnings, results tend to show high variability and small traffic volume. In short, let the experience run long enough and you can easily skip this A/B testing mistake!
Unrepresentative traffic volume
Another big A/B testing mistake is having unrepresentative traffic. In other words, this means either having unequal traffic volume directed to the different versions of the web pages or content or having a traffic that is not determined randomly. This would be particularly hurtful as it is randomness that allows us to say we have a cause-effect relation.
In brief, A/B testing is a perfect way to test one element at a time. It is easy, cheap and simple to implement. However, in order to have accurate results, it needs to be done in a certain way to avoid mistakes.
Sources:
https://www.convert.com/blog/a-b-testing/ab-testing-mistakes/
Digital marketing Powerpoint, Theme 1
