article thumbnail

What is A/B Testing? The Complete Guide: From Beginner to Pro

CXL

A/B testing splits traffic 50/50 between a control and a variation. A/B split testing is a new term for an old technique — controlled experimentation. Table of contents What is A/B testing? What is A/B testing?

article thumbnail

Is A/B testing dead?

Martech

“Today, A/B testing is thriving — it’s been a huge improvement from non-A/B testing,” said George Khachatryan, CEO of AI company OfferFit, in a recent webinar. “At Fortunately, A/B testing is evolving.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

A/B testing is dead, long live A/B testing!

Martech

Odds are, you’re A/B testing. But this method is now a thing of the past … and why is that? What’s making A/B testing obsolete? Register today for “A/B Testing is Dead, Long Live A/B Testing!”

article thumbnail

Multi-Armed Bandit: When to Do Bandit Tests Instead of A/B Tests

CXL

When should you use bandit tests, and when is A/B/n testing best? Though there are some strong proponents (and opponents) of bandit testing, there are certain use cases where bandit testing may be optimal. What is bandit testing? Short tests 2.

article thumbnail

Master the art of A/B testing

Search Engine Land

Learn how you can test before launching, improve your return and understand why your consumers selected the asset they did. The post Master the art of A/B testing appeared first on Search Engine Land.

article thumbnail

Bayesian A/B Testing: A More Calculated Approach to an A/B Test

Hubspot

What are some of the reasons you run an A/B test? When I think of the benefits of A/B testing, I think of one of the most popular and concrete ways to experiment with ad designs that are effective for target audiences. Bayesian A/B Testing.

article thumbnail

How to A/B Test CTAs Like HubSpot Experts

Hubspot

The call to action is a put up or shut up moment that can turn curiosity into a legitimate business opportunity. It's the bridge between visitors and leads — a point of entry that allows you to capitalize on the web traffic you generate. A lot goes into creating an effective CTA.

article thumbnail

Master the art of A/B testing

Martech

Traditional A/B testing only lets you test messaging, creative and other marketing assets after they have gone live in the market – with no warning of how they will perform. The post Master the art of A/B testing appeared first on MarTech.

article thumbnail

How to Use A/B Testing and Personalization Best Together

ConversionXL

If you ask most marketers, they will tell you that A/B testing and personalization are two completely different things. Every marketer wants higher conversion rates, and A/B testing and personalization are simply two ways to drive more conversions. Still, most marketers treat testing and personalization as independent efforts when, in fact, they are best applied together. Why and when you should use A/B testing and personalization.

article thumbnail

How to A/B Test Your Pricing (And Why It Might Be a Bad Idea)

Hubspot

Choosing the right pricing for your product is a little bit like Goldilocks. Too high, and you risk alienating a large majority of your potential customers. Too low, and you likely won't have enough revenue to run a sustainable business. This is where A/B testing comes into play.

article thumbnail

Predicting Winning A/B Tests Using Repeatable Patterns

ConversionXL

If you ever ran a highly trustworthy and positive a/b test, chances are that you’ll remember it with an inclination to try it again in the future – rightfully so. Testing is hard work with many experiments failing or ending up insignificant. In 2017 we systematically started to categorise similar test results as patterns to help us better predict more winning tests. Sometimes we simply take a piece of paper and sketch our ideas.

article thumbnail

How To Visualize A/B Test Results

ConversionXL

You may be wondering, “why should I make my own visualization of my A/B test results?” ” Because the A/B testing tools in the market already provide you all the necessary tables and graphs, right? They tell you when an A/B test is significant and what the expected uplift is. The thing is, these tables and graphs are comprehensible when you – a data driven analyst – take a look at them.

article thumbnail

PXL: A Better Way to Prioritize Your A/B Tests

ConversionXL

If you’re doing it right, you probably have a large list of A/B testing ideas in your pipeline. Some good ones (data-backed or result of a careful analysis), some mediocre ideas, some that you don’t know how to evaluate. We can’t test everything at once, and we all have a limited amount of traffic. You should have a way to prioritize all these ideas in a way that gets you to test the highest potential ideas first.

article thumbnail

How to A/B Test Your Pinterest Ads: A Step-by-Step Guide

Hubspot

Personally, I’m a huge fan of the platform and use it as a source of inspiration — I make Pinterest boards related to my interests (travel, style, etc.). Pinterest’s 335 million active users are always searching for new ideas and your ads can be a great way to pique their interest.

article thumbnail

How to Segment A/B Test Results to Find Gold

ConversionXL

You run an A/B test , and it’s a winner. Does it mean that the treatments that you tested didn’t resonate with anyone? If you target all visitors with the A/B test, it merely reports overall results – and ignores what happens in a portion of your traffic, in segments. Why is conducting post-test segmentation important? Analyzing data in your post-test analysis. Segmenting your data: before or after your test?

article thumbnail

What Does A/B Testing Have to do with Machine Learning?

Hubspot

To some marketers, A/B tests are a no-brainer. Even though the test is a timeless marketing technique, it can be a little frustrating because of the uncertainty of it all. What is an A/B Test and Machine Learning? A/B testing.

article thumbnail

How to Minimize A/B Test Validity Threats

ConversionXL

You have an A/B testing tool, a well-researched hypothesis and a winning test with 95% confidence. There are factors threatening the validity of your test, without you even realizing it. Quite simply, validity threats are the factors that threaten the validity of your A/B test results. Type I errors occur when you find a difference or correlation when one doesn’t exist. After a week, the tool declared a 23.8%

article thumbnail

Ignorant No More: Crash Course on A/B Testing Statistics

ConversionXL

A major portion of your test results are probably invalid. While testing tools are getting more sophisticated, blogs are brimming with ‘inspiring’ case studies, and experimentation is becoming more and more common for marketers – statistics know-how is still severely lacking. If you don’t know basic statistics, you won’t be able to tell whether your tests suck. Why Do I Need to Know A/B Testing Statistics? During the test?

article thumbnail

Why Your A/B Tests Are Failing

ConversionXL

In a 2013 study by eConsultancy & RedEye , surveying almost 1,000 client-side & agency marketers, it was found that 60% found A/B testing to be “quite valuable” to their business. Yet, only just over a quarter (28%) report being satisfied with their conversion rates. What’s interesting is that another study by VWO found that only 1 in every 7 A/B tests is a statistically significant winning test. A/B testing.

article thumbnail

7 of the Best A/B Testing Tools for 2019

Hubspot Marketing

This naturally begs the questions -- what should you do when you can’t find a solution to one of your problems on the internet? And, as a marketer, one of the best investigative tools at your disposal is A/B testing. Best A/B Testing Tools.

article thumbnail

A/B testing alternatives for low-traffic websites

ConversionXL

In a world where A/B tests are done by over 70% of online businesses , choosing not to follow a data-driven methodology to make informed decisions on website changes might seem unreasonable. Or if your management refuses to justify the costs of A/B testing?

article thumbnail

How to Analyze Your A/B Test Results with Google Analytics (Updated)

ConversionXL

A/B testing tools like Optimizely or VWO make testing easy, and that’s about it. They’re tools to run tests, and not exactly designed for post-test analysis. Most testing tools have gotten better at it over the years, but still lack what you can do with Google Analytics – which is like everything. Looking at a summary screen like this is not enough: Use these at-a-glace views for a quick check to see what the overall status is.

article thumbnail

Maximize the Results of your Nurture Campaigns Through A/B Testing

Heinz Marketing

For many years, I worked in B2C marketing for large technology companies where A/B testing was an ongoing process with teams dedicated to testing and refining in all areas of marketing. Testing is not always a part of the equation. Testing is key to making any marketing program a success. That’s why testing is necessary. That’s why companies should always test. Start by testing your subject lines.

article thumbnail

A/B Testing on Facebook: How to Do It Right

Hubspot

To illustrate, do you always have a tab devoted to Facebook Advertising efforts? Further, are you testing those ads before they get published? Facebook offers so many ways to test the performance of your ads before they go live. A/B Testing Facebook Ads.

article thumbnail

A Crash Course on A/B Testing Facebook Ad Campaigns

ConversionXL

Having a well-thought-out plan for A/B testing Facebook ad campaigns is essential if you want to improve your performance reliably and consistently. And the more you test, the better. A study of 37,259 Facebook ads found that “most companies only have 1 ad, but the best had 100’s” A/B testing Facebook ad campaigns can get complicated quickly (and easily produce invalid results). The Structure of a Facebook Ad Campaign.

article thumbnail

Confidence Intervals: A Guide for A/B Testing

ConversionXL

Confidence intervals are a standard output of many free and paid A/B testing tools. Most A/B test reports contain one or more interval estimates. Even if you’re simply a consumer of such reports, understanding confidence intervals is helpful. Note: If you want a deeper dive, you’re in luck: I just released a book, Statistical Methods in Online A/B Testing , and I teach CXL’s course on A/B testing statistics.).

article thumbnail

The Key Difference Between Multivariate Testing & A/B Testing

Hubspot

There's seemingly no end to the things you can test in your marketing, and if you've laid a solid framework for your inbound marketing programs, now's a great time to start optimizing and making what works pretty well work amazingly well. And the best way to get started is to conduct some A/B tests ! Or.multivariate tests? What's the difference between A/B tests and multivariate tests? What Is an A/B Test?

article thumbnail

11 Things That Work More Often Than Not in A/B Tests

ConversionXL

This article will give you some of these tactics to test for yourself. Let’s be realistic: websites are highly contextual, and what works for website A, doesn’t work for website B. In addition, the concept of a tactic is one thing, its impact is all about the specific implementation. That being said, there are a few tactics that we’ve seen win time and time again. Sure enough, a lot of the answers were similar. What is a value proposition?

article thumbnail

11 Things That Work More Often Than Not in A/B Tests

ConversionXL

This article will give you some of these tactics to test for yourself. Let’s be realistic: websites are highly contextual, and what works for website A, doesn’t work for website B. In addition, the concept of a tactic is one thing, its impact is all about the specific implementation. That being said, there are a few tactics that we’ve seen win time and time again. Sure enough, a lot of the answers were similar. What is a value proposition?

article thumbnail

Don’t Do A/B Testing [Rant]

ConversionXL

A/B testing is highly useful, no question here. But a lot of businesses should not be doing it. A lot of microbusinesses, startups and small businesses just don’t have that transaction volume (yet). You *might be* able to run A/B tests with just 500 transactions per month too (read: how many conversions do I need? ), but you need bigger impacts per successful experiment to improve the validity of those tests.

article thumbnail

18 Top A/B Testing Tools Reviewed by CRO Experts

ConversionXL

Finding a proper A/B testing tool isn’t the problem anymore. If you work in conversion optimization – whether at an agency, in-house, or as a consultant – you almost certainly run A/B tests. Though you could hire someone full time to sift through and analyze the pros and cons of each tool, it’s easier to learn from the experience of others and make a decision based on that. Some cons: WYSIWYG Editor a bit slow to load.

article thumbnail

How to Do A/B Testing: A Checklist You'll Want to Bookmark

Hubspot

But basing marketing decisions off of a "feeling" can be pretty detrimental to results. Rather than relying on guesses or assumptions to make these decisions, you're much better off running conversion rate optimization (CRO) tests -- sometimes called A/B tests. CRO testing can be valuable because different audiences behave, well, differently. But these tests can also be complex. What Is A/B Testing? How to Conduct A/B Testing.

article thumbnail

Sample Pollution: The A/B Testing Problem You Don’t Know You Have

ConversionXL

Here’s an uncomfortable truth about conversion rate optimization: lots of people are running bad tests without even knowing it. In Bart’s words, “If you don’t know about sample pollution, stop testing.”. The first step to running better tests is to really understand sampling. It’s how many visitors or conversions you need in your test. If you don’t calculate your sample size before you run your test, you’ll run into bad data, likely without even realizing it.

article thumbnail

Beyond “One Size Fits All” A/B Tests

ConversionXL

If you’re invested in improving your A/B testing game, you’ve probably read dozens of articles and discussions on how to plan and run A/B tests. You might also have heard it’s best to come up with many variants to test against the control to improve your chance of finding the best option. No matter what rule is offered, such advice seems to rest on the assumption that there is a one-size-fits-all solution that works in most situations.

article thumbnail

5 Uncomfortable A/B Testing Questions

ConversionXL

AB testing is supposed to be straightforward and extremely transparent. You ran a test, and had a solid winner – the sample size was adequate, test ran for 2 business cycles, confidence level was above 95% – and the lift was like 10%. Yes, re-testing it seems to be the easy way out, but in reality it might not be so easy. Testing something is an opportunity cost – means you can’t test something else.

article thumbnail

How to Analyze Your A/B Test Results with Google Analytics

ConversionXL

A/B testing tools like Optimizely or VWO make testing easy, and that’s about it. They’re tools to run tests, and not exactly designed for post-test analysis. Most testing tools have gotten better at it over the years, but still lack what you can do with Google Analytics – which is like everything. Looking at a summary screen like this is not enough: Use these at-a-glace views for a quick check to see what the overall status is.

article thumbnail

What Do You Do With Inconclusive A/B Test Results?

ConversionXL

So you ran a test – and you ran it correctly, following A/B testing best practices – and you’ve reached inconclusive results. A surprising amount of tests end up inconclusive. According to Experiment Engine’s data, anywhere from 50% to 80% of test results are inconclusive, depending on the vertical and stage of the testing program. In Massey’s case, tests of video footage on an apparel site came up inconclusive.

article thumbnail

How 8 Different A/B Testing Tools Affect Site Speed (Original Study)

ConversionXL

But A/B testing tools actually may slow down your site. We researched 8 different testing tools to show how your site performance is affected by each one. Visitors have a patience threshold. Just like in a any shop. For you, that means a slower site is less revenue. How a Testing Tool Cost Us 12 Points On Google Page Speed. Simyo (telco company) had a 99/100 Google Page Speed score for their homepage.

article thumbnail

B2B Reads: Metrics, A/B Testing, and Content Experiences

Heinz Marketing

We’ll miss a ton of great stuff, so if you found something you think is worth sharing please add it to the comments below. A look at some of the specific elements that make for the best conversion rates. Earning your audience’s trust isn’t an easy task, but here are some strategies that might help build a relationship with your clients. Having a large audience is nice, but it shouldn’t be your main goal when it comes to social platforms like LinkedIn.

article thumbnail

How to Estimate a “Net Value” for Your A/B Testing Program

ConversionXL

In my experience, I find that teams and organizations report many winning A/B tests with high uplifts, but somehow they don’t seem to bring those uplifts in reality. Five types of A/B test “wins” can exaggerate discovered uplifts. I use the acronym “de FACTO”: F alse winners; A nti-winners; C hanging winners; T ricked winners; O verestimated winners. So keep increasing the quality, (statistical) trustworthiness, and velocity of your A/B tests.