If you are generating demand for a software as a service (SaaS) business, you know that A/B testing is the expected experimental baseline nowadays. But how do you go beyond running split tests to simply stay afloat and instead use split testing to gain a real edge on your competitors?
The answer: Leveraging the understanding that you have for your own demand generation strategy and your target personas, focus A/B testing on your signup and pricing pages methodically and experimentally fashion in order to compound the growth of your lead generation in the long run.
In today’s demand generation landscape, A/B testing is not a silver bullet. Most a/b tests won’t produce large gains in the beginning but that’s okay. The challenge is getting started today so that you can jumpstart the process of continuous improvement. Understand and learn user behavior relative to your customer journey and with continuous split testing in place, you will begin to yield a lift in conversions.
We will introduce you to what a good A/B testing framework looks like, why you should only be testing one variable at a time and share with you best practices to run inside your organization to field and execute only the best experiments.
Don’t expect overnight successes with A/B testing. Expect a lot of hard work, a lot of work digging in deep, and adjusting to an experimental mindset that will yield real results in the medium and long run as long as you are putting in the effort day in and day out. We will help you get started with conversion rate optimization for your SAAS landing and pricing pages today.
“Take a risk and keep testing, because what works today won’t work tomorrow, but what worked yesterday may work again.”
Amrita Sahasrabudhe
Chapter #1
Quick Introduction to A/B Testing For SaaS
Let us get you on the path of leveraging A/B testing to drive repeatable results on your landing pages and pricing pages.
Every edge that you can get counts significantly in the competitive SAAS space, whether you’re a venture-funded multi-million dollar startup unicorn or a bootstrapped small business.
It’s tough enough to drive a great Quick Ratio when dealing with customer retention and churn for existing customers. Get started with A/B testing today, especially for your signup pages, to keep that lead generation flowing to increase your odds of converting new, fresh customers.
A/B testing, by definition, is a method of comparing two versions of a web page or app against each other to determine which performs or converts the best.
This graphic from SplitMetrics shows how AB Testing works.
The concept of A/B testing is simple and the competitive advantage lies in the prioritization of experiments and in the execution of all the fine details.
Check out the number of tests these companies run:
What do you get from executing all the fine details properly? According to Tomaz Tunguz, the qualitative data that you get to mine from substantial A/B testing on your landing and pricing pages is the prize. This applies to businesses of all sizes.
Whereas big companies like Google and Salesforce can A/B test and seek a one to five percent gain, we are confident that if you A/B test properly, you will see percentage gains that are much larger than that over time.
“Every page on your website has one unique goal in delivering the value you offer to your customers.”
Chapter #2
Use Cases: How Has A/B Testing Helped key SAAS Businesses?
There are SAAS businesses that have clearly taken it to the next level with A/B testing in a very intentional manner. We will start with an insider story from Intercom, a B2B startup unicorn, and how they improved signup conversion rates by aggressively testing their signup pages. And we will end with a pricing page use case.
Intercom
With a mature account structure already in place and messaging that was clearly aligned with their products, Intercom needed a new way to juice performance of their Google AdWords search campaigns. What did they turn to for their SAAS business? A/B testing on their signups pages.
Michele Pendergast, now head of demand generation at Envoy, piloted and launched a strategy that would get Intercom’s AdWords campaigns to drive a higher volume of highly qualified sales leads and to do more more efficiently.
How she executed the Direct Response Landing Page strategy: Segmenting non-branded campaign traffic into two cohorts for the same keywords, two groups were tested into 50/50 traffic:
- A set of generic pages non-customized to ad copy
- A set of landing pages with headlines and body copy dynamically optimized to align with what was written in the ad copy
Michele successfully theorized that by having certain instances of ad copy and keyword copy be the same in that second group of campaigns, Google would respond well by assigning a higher Quality Score to those keywords, which resulted in a lower Cost Per Click and a higher Clickthrough Rate.
Sample of Direct Response Landing Pages from Intercom
The net result, after all, was said and done: A 20% increase in Click Through Rate, while decreasing Cost Per Signup by about 15%. The key performance metrics for this test were focused around increasing lead and sales volume, and it was a cherry on top to reduce the cost per lead as well.
All gains seen from incrementally testing signup pages for a SAAS business.
RJ Metrics
As told on the Crazy Egg blog, by building a cost calculator onto a key pricing page, seeing early successes, then making user experience related tweaks until they arrived at an acceptable conclusion, RJ Metrics saw a large lift in the key performance metric they were optimizing against.
They wanted to get their existing customers to adopt more and more features on their platform to increase retention and stickiness. A few key lessons here:
- Know what key performance metric you want to measure against upfront and stay laser-focused on that metric.
- Think critically. Understand that the first phase of the experiment will most likely lead to directional results and most definitely useful learnings.
- From there, iterate and compound learnings until you arrive at the outcome that you are aiming for.
From this work, they were able to increase conversions on their pricing page by about 310%.
The winning page design from RJ Metrics.
So, there you have it: Two SAAS businesses that iterated their way into large successes with A/B testing. Let us dive into how you can get started with A/B testing in a way that increases the odds of producing an edge for your business, helping you grow revenue and profit.
“Creating a thoughtful and targeted landing page experience as a continuation of your personalized ad campaigns will help reduce friction for your audience, connect the dots for them, and hopefully help them convert more easily too.”
Chapter #3
Have a Process to Drive Repeatable Experimental Results
You do not have to have three million visitors a month or a stable of developers to be able to A/B test properly.
As long as you have a process similar to the one that we will describe and have a laser focus on your sign-up and pricing pages, you will be setting yourself up for success in the long run.
A/B testing success will not happen for you overnight, but you cannot have A/B testing success without laying the groundwork as soon as you can and dedicating time to test frequently and aggressively. Often, especially for smaller businesses, the qualitative results and learnings that you drive with A/B testing will be worth more than statistical significance.
A guide to AB Testing From Optimizely
These are a few of the key foundational muscles that you will want to exercise in order to be able to drive a successful process.
#1: Always be writing copy and refining messaging. Keep it in your back pocket. It also helps to have extra voices either writing copy alongside you or giving you feedback on copy. Iron sharpens iron.
#2: It’s also useful to have a reliable hybrid design and engineering resource in-house who can help you make rapid and accurate changes on your landing pages.
#3: If you don’t have a design and engineering unicorn in-house and can spare some of your own bandwidth, have access to an easy-to-use landing page creator and modifier like Unbounce or Instapage. Once your A/B testing is mature enough, it will also be useful to have a tool such as Crazy Egg help you interpret insights over time and scale and evolve the learnings from your experimentation.
Once you have these foundational elements in place, you can start implementing and shifting the mindset of your team toward having an experimental process, or what can be defined as a growth marketing mentality.
The growth mindset involves:
- Knowing what key performance indicator you want to optimize toward and understanding that all secondary metrics must eventually route back to this path.
- Being patient. Understanding that it is more than okay to change one variable at a time.
- Being able to iterate and adjust on the fly.
- Knowing how to stage your experimentation process.
- Having discipline and strong project management skills. Knowing when to start a growth experiment and when to end one, and using that end period to compound and set the stage for the next experiment
If you are testing copy, start with copy, iterate, and get the learnings. Then iterate on copy further. Finally, layer on additional learnings and upgrade to testing new designs and user experience.
When you are able to inspire your team to drive forward systematically, you can compound growth for your business over time.
“The more targeted and strategic an A/B test is, the more likely it’ll be to have a positive impact on conversions.”
MICHAEL AAGAARD, SENIOR CONVERSION RATE OPTIMIZATION CONSULTANT
Chapter #4
Source Your A/B Test Ideas
Before you source your A/B test ideas, you need to have a solid understanding of your key business objectives and the bottlenecks to achieving them.
If your key business objective for the quarter is to drive more qualified leads but you have noticed that the Click to Sign-Up conversion rate is lower than expected, then you can begin creating ideas on how to overcome those bottlenecks and barriers. In this case, you could decide that you want to test different language for your call-to-action button.
If you run a business-to-business SAAS business and have noticed that the bottleneck is that fewer than expected Signups are converting into Marketing Qualified Leads, you may want to consider qualifying visitors more granularly at Signup, such as by making it mandatory to enter a work email instead of a personal email address.
Start documenting your ideas at scale. One way to do this would be to create a Google Sheet and just like with the exercises above, name off experiment ideas associated with the top successes and opportunities that you believe may exist for your signup and pricing pages.
Your list could include also thoughts such as, “I like the landing page copy on my signup pages but I believe there is opportunity to increase the sense of urgency.” This idea could lead to spinning off 3-4 ideas to test more urgent copy on your call-to-action buttons.
- Get started
- Try for free
- Subscribe today
- Learn more
And just like that, from one thought, you have four potential A/B tests that you could experiment with. It is also helpful to source A/B test ideas from across your company, whether it is a Customer Support Manager, Marketing Manager, or even a Head of Engineering.
If you find yourself stuck, it is often helpful to go on Google and search for growth experiments that other SAAS businesses have run in the past. What has worked for other businesses may not work directly for yours, however, so please take this as a word of caution.
Additional inspiration may include:
Sign-Up Pages
- Testing headline copy
- Testing body copy to make the sign-up more appealing
- Creating urgency in the messaging for your call-to-action buttons
Pricing Pages
- Testing $1.99 vs. $2
- Moving your call-to-action above the fold
- Creating a cost-calculator, though this is higher effort with more upside
And to further pique your interest, here are additional ab testing ideas to explore.
“Finding out who your ideal visitors are and where they’re coming from is one of the shortest paths to a higher conversion rate.”
Chapter #5
Prioritize Your SaaS A/B Testing Ideas
The next step – prioritization – is equally important. Because the first A/B test experiments you run are the foundation for the rest, it is really important that you start with your best.
Focus only on the most impactful concepts. Avoid the temptation of snacking on good-but-not-best ideas. Your bandwidth is precious and it will crucially be needed down the line with A/B testing.
Methods to cull your best tests may include:
- Ranking experiments by effort vs. impact. For example, if you want to overhaul the design of a page completely, do you have access to a landing page tool such as Unbounce?
- Let’s say that you want to test different messaging on your call-to-action buttons in order to measure a lift in conversions. Do you already have conversion tracking set up in Google Analytics, and if not, how much effort would it take to get it installed properly?
You are on the right course if you have whittled your 20-30 ideas into your top-5.
“There are no sacred cows. Just because an expert somewhere said something is true, you don’t know if that will be true for your company until you test it.”
Chapter #6
Measure Results To Scale SaaS Website Conversions
Before you run your test, set realistic expectations with yourself. If you run an A/B test and you do not get the results that you want at first pass, it is not a failure as long as you monitor results and clearly document what you are learning.
Once you have all the boxes checked off, and when you have your first idea picked out assuming that you have conversion tracking set up, run the test and monitor your results. Give time for the results to pour in more conclusively.
As you are getting early insights on your experiments, begin thinking through and mapping out how you would want your A/B testing to evolve. Open up a note-taking app and jot down your thoughts in the early days of a test, sleep on it, revisit, and put it all through the lens of common sense.
For example, if you are seeing that changing a call-to-action from “Buy” to “Add to Cart” on a signup page and you are beginning to see a small lift in conversion rate, what experiment would you want to perform next in order to build upon learning that specificity and urgency is working for your SAAS business?
Build on your experiments and create an ongoing and virtuous cycle for A/B testing. Keep culling, keep tracking, keep running, and you will be there in no time. Success or fail, keep iterating, keep documenting based on your results, and keep the experimentation cycle going.
Summarizing a few of these findings:
- Scope of the test
- Check on test results
- Keep brainstorming
- Call successes and failures. Learn
- Compound
Given that you have both landing pages and pricing pages, you can run experiments on both sets at the same time since those page sets are fairly exclusive from each other in the customer journey as well. And for most businesses, it is fine to call an experiment based on qualitative results. You do not always have to wait for statistical significance.
“Marketers recognize the value of A/B testing. We are firmly in the era of the data-driven CMO: Marketing ideas must be proven, and backed by sound data.”
Conclusion
Take Your Next Steps
Start A/B testing on your signup and pricing pages today and get on the path toward compound returns. One way to increase your odds that you can get from A/B testing is by working with Insivia to foundationally understand your most important key performance metrics and target audiences and personas.
By doing a deep dive and working sessions via Insivia’s Breakthrough Program, you can set up your SAAS business effective A/B testing in the long run.
A/B Testing Frequently Asked Questions
How long should I run my test?
Time isn’t as much a factor as how many people are tested. The more you test, the greater confidence you can have of accurate data and assumptions. Focus on testing 250 – 1000 visitors however long it takes, not 2 or 3 months.
What is the biggest A/B test mistake?
A/B Testing that has too many factors is not valuable. If you are testing two pages that are structured and look completely different, then there is no real way to tell what really makes the difference. Limit to minor changes to know specifically what works better.
Does A/B Testing impact SEO?
This is a simple no, Google’s Matt Cutt advises running A/B tests to improve the functionality of your site. Website Optimizer has a good breakdown of the myth too, and why it doesn’t hold up.
Can you run AB Tests for things other than landing pages?
Yes, you can run tests on a wide range of marketing tactics. Some of the most popular are emails and paid advertising.
This article has 28 different tests you can try.
What are other things to look out for?
There are several other factors that can throw off your results:
Time of year. Certain seasons or holidays may significantly impact the volume, type, and intent of visitors.
History Effect. If major political or other factors are happening in the world, it can actually be in impact.
Instrument Issues. If your software is not set up correctly or is not working, then that can cause a big problem.