Ecommerce A/B Testing
Get The Print Version
Tired of scrolling? Download a PDF version for easier offline reading and sharing with coworkers.
A link to download the PDF will arrive in your inbox shortly.
Key highlights:
Ecommerce A/B testing is a method of comparing one element change to another to determine how it impacts user behaviour.
A/B testing in ecommerce is important as it allows a business to figure out which efforts are most impactful, which elements are holding them back, and how they can improve overall.
A/B testing works by analysing metrics and data to find weak points, forming a hypothesis, changing a single element, and then running the test until completion.
It’s possible to A/B test almost every facet of an ecommerce business, from navigation to CTAs to product pages to email marketing efforts.
There are a number of tools that can help with A/B testing, as well as agencies who specialise in A/B testing and data analysis.
As an ecommerce business, you have access to more data than a traditional brick-and-mortar could dream of. What are you to do with this veritable treasure trove of data?
Hint: It’s ecommerce A/B testing.
With ecommerce A/B testing, or split testing, you can leverage data and tools to test everything from tweaks to your product pages to button placement to approaches with marketing strategies.
Best of all, ecommerce A/B testing utilises information you likely already have.
Why is A/B testing important?
Every effort in ecommerce consumes resources, whether that’s time or money.
Ecommerce A/B testing is important as it allows you to determine which efforts are working better than others, so you can utilise your resources most effectively.
For example, you only have so much set aside for your marketing budget. You’re hopefully collecting various ecommerce metrics and site stats, which let you know whether sales and traffic are going up or down.
But, how do you know if your marketing campaign is delivering all it can?
With the right A/B testing strategies, you can try different marketing approaches using a small sample size. This lets you know if approach A or B works better, and which you should implement for the larger campaign.
A/B testing provides valuable insights beyond campaigns, too. Testing can tell you:
If ecommerce site changes are working
Which messaging performs best across various segments
If SEO tactics are having an impact
Which tweaks to your checkout process are impacting cart abandonment
The above are only a handful of the use cases for A/B testing, too. Testing can help you across nearly every facet of your business, with the only limiting factor being your creativity, and the confines of time. (It comes for us all.)
How does A/B testing work?
Simply put: A/B testing works by presenting two different groups with two different versions of the same thing, with only a single change, to see which performs better.
This is different from multivariate testing, which involves testing numerous changes at once, rather than one element.
What’s A/B testing look like in action?
Hypothesis formed on how a change to X might impact user behaviour
Change implement to X
Revised version of X shown to half of audience group
Original version of X shown to other half of audience group
Results analysed for key takeaways
The testing process will look different on a granular level depending on what you’re testing, but the overall approach is the same — you’re always testing one change, showing it to a split audience, and drawing what insights you can.
What can you A/B test?
A/B testing is all about removing consumer pain points, reducing friction in the shopping process. By doing so, you can significantly increase the chances of conversions and improve sales.
But, what exactly can you test?
In the context of your own ecommerce website, here are some common elements to A/B test:
Homepage and landing pages
Product pages, category pages, and product descriptions, and product image placement
Social media post language and layouts
Social proof (reviews and testimonials) placement
CTA buttons and copy
SEO tactics such as meta descriptions and keyword volume
The beauty of A/B testing is that you can systematically work through each of these areas, incrementally and consistently improving each one to increase conversions.
Examples of ecommerce A/B testing
The number of things you can A/B test within ecommerce are virtually endless. The following are a few of the more common ecommerce testing ideas, but keep in mind there are far, far more.
Navigation A/B test.
Navigation A/B testing can help you determine if your site is currently set up in the most user-friendly, optimal way. This is possible by testing navigation elements like:
Dropdown menu labels
Sticky vs. static navigation bars
Categories nested inside navigation labels
Design elements within the navigation menu (Colour, font, etc)
Great site navigation can guide a shopper to a product and checkout with ease, while poor navigation can put potential customers off entirely.
No matter how beautiful your webpage is, don’t neglect the navigational experience.
Product page A/B test.
Product pages do a lot of heavy lifting. Even a great product can fall flat if the page is poorly thought out, or if the description itself is sloppy.
In fact, 38% of shoppers say poor grammar and bad product copy is a “dealbreaker.”
With a product page A/B test, you can determine which product page layouts, copy, product images, and so on have the most positive effect on the add-to-cart rate.
Form A/B test.
When you’re in ecommerce, there’s no getting around forms. At the very least, you need to collect someone’s information when they place an order. But, there are email lists, pop-ups with offers, and other elements that can entail a form.
A form A/B test can help you determine whether you’re asking for too much, if fields are laid out in the optimal manner, if text or drop downs work better, and beyond.
Checkout process test.
If you want your customer to complete a purchase, your checkout process needs to flow as smoothly as possible — which is why A/B testing is so crucial.
A checkout process test will help determine:
Whether a simple multipage checkout or a single-page checkout converts better
The effect of merging several steps within the checkout process
If checkout forms are laid out effectively
How express checkout options impact conversions
Where people bounce in the checkout process
Cart abandonment sits at just above 70%.
Abandonment happens for a number of reasons, which can include a poor checkout process. This makes checkout A/B testing all the more important.
P.S. Don’t forget to send a nice discount email to those who added something to their cart and left it. This is a great way to potentially recapture lost revenue and win someone back over.
Call-to-action (CTA) testing.
Your ecommerce journey is peppered with CTAs, from those on your homepage to your product pages to social media ads. Call-to-action testing is how you make sure those little buttons are doing all they can.
With CTA testing, you can assess:
CTA copy
Button placement
CTA design
Pop-up vs on-page
When running a CTA test, you’ll want to look for click-through rates (CTR) to determine if people are actually clicking. If so, this tells you the CTA is doing its first job: getting people to click.
If conversions are falling off on the destination page, you’ve got a whole new can of worms to unpack. A can that’s likely labelled, “Something is wrong with your landing page.”
What do you need to run an A/B test?
Before you begin testing, it’s a good idea to conduct thorough research on your target audience and create a roadmap that will guide you specifically through what to test and in what order.
Some research functionality you should have before testing includes:
Analytics: Identify your most popular pages, CTR, traffic sources, pages with a high bounce rate, etc.
Heatmaps: Discover exactly where and how people navigate your site. See where they click, where they pause, where they leave, and whether a page is used as intended.
User Tests: Personally watch users navigate your site to discover where points of friction occur.
Surveys: Directly ask your customers about their shopping experiences. Did they encounter any problems? What would they like to see improved?
After doing a thorough analysis of your customer’s behaviour, you can begin to develop hypotheses about your current online store and how you might improve the overall customer experience. These will form the basis of your A/B testing roadmap.
Note: While you can A/B test at any point in your ecommerce journey, the larger your audience, the better. Having an email list in the thousands, weekly site traffic in the thousands, and so on, will allow you to draw more conclusive insights, versus a few hundred.
What are the benefits of ecommerce A/B testing?
As mentioned, A/B testing is a powerful tool for crafting the ideal customer experience, but that’s not the only perk you’ll come across. Let’s take a look at some other great reasons for implementing A/B testing:
It’s entirely based on data: There’s no relying on gut feelings or instinct — changes are only implemented if they’re validated by data.
It reduces risks: Instead of taking a big gamble on an entire site redesign, you’re making small, incremental improvements, one element at a time. And, further implementation only happens when validated by data.
It improves resource deployment: You only have so much time and money in a year. Every effort needs to count, and A/B testing helps you determine where those efforts will count the most.
It improves the customer experience: When you’re A/B testing an element, you’re not only making it more profitable for you, but also ensuring it helps the customer have a better experience.
Best of all, A/B testing is something you can do again and again, allowing you to benefit all over again.
Common ecommerce A/B testing challenges
While A/B testing delivers a number of benefits, it isn't without its challenges. This is especially true for those new to A/B testing, as mastering it requires some first-hand experience (and the mistakes that come with it).
The following are a handful of the elements that may cause challenges with A/B testing:
It requires experience: Some of the specific tools used in A/B testing have a steep learning curve. There also needs to be a fundamental understanding of web design, statistical analysis, and domain-specific knowledge.
It’s a lengthy process: A/B testing isn’t a quick fix or magic cure all. A single test is unlikely to do much for your revenue. Instead, it’s about developing a long-term plan and targeting specific goals and improvements over time.
Traffic is required: While the amount of site visitors will vary depending on the test and objectives, you need a fair amount of traffic to ensure that your tests are statistically valid, ideally in the thousands of visitors per-week.
Don’t let the above put you off of running A/B tests. Temper your expectations, especially if you haven’t run any A/B tests before, and know that things will only get better with each test.
How A/B testing impacts KPIs
Sure, A/B testing for ecommerce can provide grand benefits to your company as a whole. But, what about on a granular level?
There are several key performance indicators (KPIs) that A/B testing can have a direct influence on.
Improved return on ad spend (ROAS).
The more friction you remove from your website, the more paid traffic turns into sales, effectively increasing your return on ad spend.
Think of it this way: If you spend money on ads to send people to your site, and your navigation or checkout process is poorly optimised, people won’t convert. The opposite is true for an A/B tested, optimised experience.
Reduced customer acquisition cost.
Similar to how testing helps improve ROAS, the money you spend on other methods of customer acquisition will benefit from the reduced friction and shrink your customer acquisition cost (CAC).
Not convinced?
After implementing extensive testing, one case study noted a client reduced their CAC by 25%.
Increased lifetime customer value (LTV).
Testing makes your website more engaging, which helps drive return business.
In other words, the easier you make the shopping process, the more customers will want to come back again and again, thus increasing your customer lifetime value.
Increased email signup conversion rate.
Your email list can be a huge revenue driver. But, only if people sign up.
A/B testing which signup form layouts, copy, or CTAs work the best can increase the chances people subscribe. This can result in a larger, healthier email list, which can act as a great outlet for your business in the long run.
Once you’ve grown your email list, make sure you’re utilising ecommerce email marketing to the fullest.
Increased average order value (AOV).
When you A/B test which product copy converts, which checkout process is smoothest, price points that are most palatable, and so on, you ultimately increase the chances your AOV increases.
You can even A/B test popups, add-ons, and bundle pricing, furthering your opportunities to increase AOV.
Reduced cart abandonment rate.
Every time a person abandons their shopping cart, that’s money walking out your digital door. Using A/B testing to optimise your checkout process can remove friction, confusion and mistrust and reduce your cart abandonment rate.
Psst. Did you know a simple image on your product page can mean the difference between a sale and a pass? Learn more in our piece on reducing cart abandonment in 2026.
How to approach A/B testing
A/B testing may only involve comparing one change at a time, but it still requires the right strategy, skills, tools — and if all else fails — an outside agency.
It might sound like a lot, but with a little strategy you’ll be testing whether an animated CTA that dodges a mouse cursor performs better than static in no time. (Please don’t do this.)
Find a problem to address.
Most A/B tests set out to fix or improve something. Find your starting point by examining data, looking for problems like high bounce rate, low conversions, cart abandonment, and so on.
If you don’t have any glaring problems to fix, you can also focus on strengthening your efforts.
Do you think you could get more value out of your email marketing?
Are competitors still pulling ahead and you think an improved site might help?
Do you have a theory around why a certain product page is doing better than the rest?
A/B testing, while useful for fixing problems, can be a great way to better understand your audience, and can result in surprising revelations that lead to businesswide improvements.
Note: The entire A/B testing process hinges on historical data, and the ability to collect data. If you’re not already, make sure you’re utilising common ecommerce tools to track core metrics.
Create a hypothesis.
With your problem or curiosity identified, it’s time to form a hypothesis.
A hypothesis should be something you can prove with data and link to customer behaviour.
Some example hypothesis include:
By breaking up product page copy with bulleted text, bounce rate should decline.
Email marketing open rates will increase if subject lines include personalisation.
Cart abandonment will decrease if the checkout process is reduced to one page versus two.
Depending on the area you’re testing, you may need to bring in additional stakeholders. For instance, if you’re looking for an email marketing test, make sure your email marketing specialist is heavily involved.
Identify a test element.
It’s important you only have a single element change, otherwise it will be difficult to pinpoint which change resulted in the test outcomes.
You can test just about anything with an A/B test, including:
CTA placement, size, colour, style
Subject line length, personalisation, emoji content, etc.
Product page image placement
Navigation style and copy
Pop-up copy, style, placement, frequency, etc.
Remember: The test element is directly tied to your hypothesis, so if you can’t determine which element to test, return to your hypothesis and determine if it’s the right fit.
Run the test.
Once you’ve settled on a hypothesis and element to test, it’s time to run tests.
Typically, you want to set a data threshold. The higher, the better, as it ensures you have enough data to draw conclusions. For example, you can run a product page test until you hit 1,000 views on the test, at minimum.
When running the test, make sure you remove as much bias as possible. If you’re testing site changes, have the test version of the page display to every other visitor. If it’s an email test, make your test group as varied as possible, and so on.
Once you’ve hit your desired data threshold, stop the test and compare equal datasets. If you ran the test page until 1,000 visitors, compare performance to the first 1,000 visitors on the original version of the site.
Note: Within testing, the confidence interval measures how sure you are of the outcome. Many researchers aim for the highest level possible, from 95-99%, which translates to the hypothesis being true 95 or 99 times out of 100. The larger the sample group, the more confident you can be. For further information, read this in-depth breakdown.
Draw conclusions from the test.
It’s time to put your data cap on and analyse the results. If you have any in-house data experts, it’s their time to shine.
Compare metrics from the test versus the original version of the element. You’re wanting to confirm or deny your hypothesis with data, while also drawing what conclusions you can around user behaviour. This means asking:
Was your hypothesis accurate?
If your hypothesis was true, how significant was the difference?
If the hypothesis was wrong, why? What follow-up test could you run to explore this further?
How difficult would these changes be to roll out to the rest of your business? Is it financially feasible? Were the benefits great enough to outweigh any resource costs?
Again, if you don’t have the in-house expertise to properly analyse the results, don’t hesitate to hire a conversion rate optimisation (CRO) agency to assist.
Ecommerce A/B testing support
In the best case scenario, an A/B test goes according to plan, your hypothesis is true, and you’re able to grow as a result.
In reality?
A/B testing can be unpredictable, whether your hypothesis is wrong or the data shocks and confounds you. Hey, it happens.
Fortunately, CRO agencies exist to help with this kind of thing, as do a number of great A/B testing tools.
Conversion optimisation agencies.
Specialised CRO agencies excel at A/B testing, helping you identify problem areas, run efficient tests, and draw powerful insights.
But, not all CRO agencies are the same. Some are going to be better than others. On top of this, some marketing agencies can offer A/B testing or CRO services, but provide little more than a surface-level analysis.
To ensure you find the best CRO services possible, look out for the following when vetting a vendor:
Whether they offer CRO as an add-on to other marketing services or focus exclusively on CRO.
Case studies that demonstrate previous CRO success. These case studies should make clear the specific processes they used to achieve their results.
Whether they’re data-driven or use flashy language like “insights.” (You can’t run a successful A/B test on vibes.)
If they have a defined CRO process they use as a template.
Do they focus on the short term or long term? A proper CRO agency should have a long-term plan in place, rather than focusing on short term wins.
Bottom line: Do your research and ask the right questions before committing to a CRO agency — your research will pay off.
Find your favourite features.
Explore all of the capabilities of the BigCommerce platform.
Examples of A/B testing tools for ecommerce.
There are various A/B testing tools available that can help with streamlining the testing process, analysing results, and beyond.
To help you make the most informed decision possible, we’ve put together a table of the most popular testing tools below.
Tool | Description | Starting Price |
Optimizely | Enterprise-grade experimentation platform for A/B testing, personalisation, and feature experimentation. | Custom pricing, estimated at ~$36K+/year to start |
VWO | All-in-one CRO platform combining A/B testing, heatmaps, and user behaviour analytics. | ~$199/month (Free plan available) |
AB Tasty | AI-powered experimentation and personalisation platform for optimising customer experiences. | Custom (~$1K–$5K/month typical) |
Adobe Target | Enterprise testing and personalisation tool within Adobe Experience Cloud. | Custom (Enterprise pricing) |
Convert | Privacy-focused A/B testing platform with strong compliance and server-side capabilities. | ~$99/month |
HotJar | Part of Contentsquare, Hotjar is a powerful tool for analysing user behaviour, known for in-depth heatmaps. | ~$39/month (Free plan available) |
Unbounce | Landing page builder with built-in A/B testing and AI traffic optimisation. | ~$74/month |
Crazy Egg | Affordable A/B testing and heatmap tool for small businesses optimising UX. | ~$29/month |
The final word
With A/B testing, you can get to the root cause of problems, uncover powerful insights, and power continual refinement of the ecommerce experience you provide customers.
Best of all, you can do it all with data and receipts to back any theories.
This isn’t to say you should discount your intuition. If something feels “off” and you think you might know why, start drawing up a hypothesis, formulate a test, and prove whether it’s a gut feeling or a data-backed issue.
Speaking of data, did you know BigCommerce can make it easier to utilise the powerful data that comes with running an ecommerce business?
BigCommerce offers big data solutions, allowing you to utilise the data tools you’re already comfortable with, easily keep tabs on your business with comprehensive dashboards, and quickly pull the information you need to fuel your next A/B test.
Transform your business with data solutions tailored to your needs.
Gain a holistic view with business intelligence tools that help you unify, analyse, and transform data.
FAQs for ecommerce A/B testing
A/B testing in ecommerce is a way to test and compare one element change to another, making it possible to determine how customer behaviour is influenced by various components of your business.
A/B testing in ecommerce can take place within the ecommerce site, within email marketing, and more.
An A/B test should run for two full business cycles, which is typically 2-4 weeks, as it ensures there aren’t any irregularities that skew the data. If an A/B test is run during a busier period, like a major holiday, it’s important to compare the data to similar holiday data, not that of a “normal” week or month.
Within A/B testing, statistical significance is a measurement of whether the data confidently supports a hypothesis. Many researchers cite 95% as an ideal confidence level, as that essentially means the hypothesis holds true 95 times out of 100.
Yes, small ecommerce sites can run A/B tests. The biggest challenge is ensuring a small business gathers enough test data to make the experiment worthwhile.
For example, if a small ecommerce site is testing changes to a product page, they want to aim for at least 1,000 site visits, as this amount of data is less likely to be drastically influenced by any irregularities.
Find your favourite features.
Explore all of the capabilities of the BigCommerce platform.