Let’s talk about success. That glorious feeling when a marketing strategy actually seems to be working– customers are coming into your store, cars are being sold, and you’re just about ready to tell your marketing team to pour every one your resources into this tactic.
Wait. Don’t call them into your office just yet.
Why? Because– maybe you could be doing better.
That might be a frustrating thing to hear. After all, you’re doing well! Why would you mess with it? Very simply, because perhaps you could improve upon what you’re doing to see even better results.
That’s where A/B testing comes in.
With A/B testing, you are able to determine which variation of a specific variable performs best. The “A” is your control, the “champion,” while B is the treatment, the “challenger.” You create two almost identical versions of a tactic, strategy, text, or anything really, with only one minor difference. Then, you see which one performs better.
Experiments like these help you decide what works best for your dealership without making any guesses. Every decision is based on real data, and you are able to know what is most effective, instead of assuming so.
You can A/B test anything, really. You utilize this type of experimentation in your everyday life, often without thinking about it. When you make the same recipe twice with a slight tweak to see what tastes better, you’re A/B testing. When you take a different route home to see which is faster, you’re A/B testing.
In terms of business, you can use A/B tests to optimize almost every area of your dealership: your website, advertising, marketing tactics, follow-up processes, and more. Not only can you make use of this type of testing on the macro-level, but each of your staff members can as well. A salesperson, for example, can use two different approaches when facilitating a test drive, to identify which performs better and closes more sales.
This may all sound great, but now to tackle the next question: how exactly do you A/B test? Let’s take it step by step.
Step 1: Set up goals
The most important thing to do before running an A/B test is to determine what exactly you are trying to optimize. The answer to this question needs to be a very specific area, and one that comes with a measurable metric. For example, you can’t utilize A/B testing to perfect your emails– there are too many variables that go into an email to effectively run an experiment. You can, however, attempt to improve your open-rates on your marketing emails, by testing different subject lines or sending times. This is the type of element that is able to be tested and measured.
Step 2: Design your test
Once you determined what you are trying to optimize, in our case the open-rates of emails, you have to consider how you will actually run your test. Think about what element you’d like to test first– everything else about the two versions needs to be exactly the same, so that you can be sure that the variable that you are testing is responsible for different results. Let’s say you decide to test your subject lines– every other factor that goes into the two emails needs to be exactly the same. This means that they should be sent at the same time, with the same content and design, to two totally random audiences. This way, if one does get opened by more people, you’ll know that your subject line is responsible, and not the time of day you sent the message or some other factor.
Step 3: Wait
That’s all. You need to be patient while you gather data with which you will make decisions. The longer you allow the test to run and the bigger the audience, the more accurate your results will be. Ideally, you should try to have at least 100 engagements per variation, but really over 300 is best. The reason is that there are obviously outliers in each group, and you want to be sure that your results are typical and duplicatable.
Step 4: Analyze your results
This is where you become a data scientist. Don’t be intimidated by this new task– there are plenty of tools that you can use to help. First, look at your results with a skeptical eye. Try to see if there is any explanation for your results besides for the variable you were testing. Think about any errors that might have occurred over the course of the test. If you see shocking results, take them with a grain of salt. Be sure to abandon any preconceived notions you may have, so that you can see the results for what they truly are, and not what you think or want them to be.
Don’t be surprised if your tests fail. In A/B testing, failure means that there is no, or very little, measurable difference between the two tests– they seem to have performed the same. This happens, a lot. In fact, Google and Bing, two huge companies that A/B test thousands of variables and employ data scientists just to design and analyze their tests, only see positive results from 10-20% of their experiments.
Another way your test might not be successful is if you cannot determine which variation performed better with confidence. Confidence is the degree to which you can statistically assume that your test is accurate. Websites like A/B Testguide and Kissmetrics both offer free calculators to determine the confidence level. All you have to do is input your audience size and results, and it will return a percentage of confidence. If you have below 95% confidence, you cannot trust your results. Best if you can achieve over 98%.
With A/B testing, the opportunities really do become endless. To achieve your largest goals, you need to optimize every specific tactic. Think about it as if you are zooming in: in order to make your entire dealership as successful as possible, you need to accomplish certain overarching goals, like reaching more customers, improving the buying experience, and streamlining the process for your staff and systems. In order to accomplish those things, you need to optimize your strategies, like email marketing campaigns and lead generation. These tactics can only be improved upon if you know that each individual aspect is running as well as it possibly can. Those are what you are A/B testing. While it may seem minute and detail-oriented at the time, these tests are going to help you achieve your long-term goals, and become the best dealer you can be.