How to do an A/B Test Right?

How to do an A/B Test Right?

A good refresher for the marketing department

What is A/B Testing?

The intuition is to test two different methods or elements before landing on one choice. This is done in many industries, scientific analysis, medicinal studies, marketing departments, etc. The focus of this post is on the marketing department.

A/B testing is also like a pilot study. Before mass marketing a campaign and spending thousands, you want to determine whether the approach is right or not. Say you are launching a year-end marketing campaign and have to decide whether to offer a buy one get one free promotion or a free shipping promotion. How do you decide on one?

That’s where A/B tasting comes into play. You test both these methods on a small number of customers to observe their reactions and the return of the campaign. The one with the higher return is the one you stick with for mass marketing. This prevents a potential failure for a campaign and saves a lot of money.

The Right & Wrong

A/B test is a very simple calculation of which campaign promotion generated more revenue and performed better. You would have to try very hard to get the calculation itself wrong. So, How can this go wrong?

A campaign might fail despite showing better results compared to the alternative method during the A/B testing. The real reason for that is poor target audience selection. More often than not, campaigns fail to produce significant returns for this reason alone.

Here is an analogy;

Let’s say you are conducting an A/B test between free shipping and buy one get one free for clothing sections. For argument’s sake, let’s say the buy one get one free would have generated much better results for the year-end campaign. However, the audience selection for this campaign during the A/B test was very irrelevant. This would generate poor results for this campaign selection, and you are forced to choose the free shipping promotion. As a result, the actual marketing campaign failed to generate a significant return, which could have been a completely different story if the target selection was better for the buy one to get one free choice.

The point is, just because the A/B test shows results, you should not blindly accept it. A/B test would always be right in terms of calculations; what you should be asking is whether the target audience selection was proper or not. This way, you evaluate your options more accurately, the results would portrait much closer to the actual picture. You would be spending the campaign money on something more significant and relevant.

Next week I will be explaining how the Target audience for the A/B test can be chosen using the help of data science and software called Enhencer.




Your Friendly Neighbourhood Data Scientist @Enhencer

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

You Still Play The Games. Sports, Math, and the Politics of Machine Learning.

10 Interesting Big Data project Ideas

Forecasting COVID-19 cases in Realtime for different countries

Using K-Means Approach to Cluster Boroughs in London.

How to connect Jupyter Notebook with a database using MySQL Connector

This Week on Nightingale: What’s the Difference Between a Report and a Visualization?

Time Series Forecasting Forecast Bitcoin

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
M Ahmed Tayib

M Ahmed Tayib

Your Friendly Neighbourhood Data Scientist @Enhencer

More from Medium

Why Marketing Operations Matters

The marketing operations maturity model, from Dr. Debbie Qaqish. It has five stages: Unaware, Efficient/Effective, Get Revenue, Customer Centric, and Next Generation.

The Real Cost of Failing to Report the Customer Franchise in Corporate Risk Registers

Black Friday ’21 and the art of tackling impulse purchases!

🚙 5 ways Uber can increase ridership and retention in 2022