Many small and medium ecommerce storeowners have heard of A/B testing and decided that it is not for them. Perhaps they thought A/B testing is complicated, the tools are expensive, or that they lack the capability to conduct successful tests.
But Google Optimize, which is free, is a testing tool that is accessible to everyone. In this post, I’ll explain how to get started with A/B testing using Google Optimize.
What Is Google Optimize?
Google has transformed the way we construct websites and write content for search engines, and with Optimize, currently in beta, it’s making inroads on how we optimize for humans. Optimize has the early indicators of becoming a game changer.
Google Optimize is an evolution of, and replacement for, a seldom-used feature in Google Analytics: Content Experiments. Experiments was a good idea. It was a free tool anyone could use from within his Google Analytics account to conduct A/B tests — i.e., content experiments.
Its downfall was that it required extensive changes to the website code to set up every experiment. It was a clumsy solution, especially when most other services offer graphical interfaces that allow users to create experiments more efficiently.
Realizing this limitation, Google introduced Optimize, in beta, in late 2016. Rather than simply creating a graphical interface, Google did what Google does: re-thought the entire process of optimizing a website through testing. The result is a tool — Optimize — that is much easier to use than its previous incarnation, with capabilities similar to other premium A/B testing tools.
Features of Google Optimize
A primary benefit of Google Optimize is that it’s integrated with a much larger package: Analytics, Tag Manager, Data Studio, and Surveys. These tools complement each other to make a one-stop solution for most conversion-rate-optimization activities. Their shared interface improves efficiency and allows CRO practitioners to spend more time on actual research.
Optimize is a standalone web-based application, but it relies on and integrates with Analytics data. This means you can:
- Use Tag Manager to create various measuring tags, implement customizations, and implement Optimize itself.
- Use Google Analytics and Google Surveys to gather data to postulate hypotheses to test and to track their impact in tests.
- Use Google Data Studio to display the results in informative reports.
The free version, Google Optimize, offers enough capability to conduct a limited, yet viable, program of A/B tests, multivariate tests (limited to 16 combinations), and redirect tests where the testing variation is represented by separate web pages with different URLs. The free version allows only three concurrent experiments. Also, you can’t use Google Analytics’ Audience data for experiment targeting, making the free version much less versatile — though you can still use Google demographics data and custom user attributes.
The paid version, Google Optimize 360, offers Google Analytics audience targeting, full factorial multivariate testing, ad-hoc (or dynamic) experiment objectives, advanced simultaneous experiment capabilities, implementation services, and customer support. Its price depends on your requirements. And to obtain the price, Google requires you to contact its sales team. However, the entire Analytics 360 package costs $150,000.
As a part of the entire Google tool kit, Optimize makes it possible to have most conversion optimization activities in one place. Optimizers will find it easy to implement using Tag Manager, since it integrates with Google Analytics by adding just a single line of code to your tracking snippet. Once integrated, Optimize can readily access every custom segment, event, or goal you set up in Google Analytics to use for targeting and objectives, as well as report the results directly to Google Analytics.
But what really sets Optimize apart? It uses Bayesian statistics, which differentiates it from almost all other major testing platforms, which use frequentist methods.
Here’s an overview of the features in Google Optimize (free) and Optimize 360 (paid).
The Bayesian Method
Optimization testing platforms tend to use the frequentist method to calculate statistical significance.
The frequentist method is what is taught in most college intro courses. It makes predictions based only on the data from the current experiment, and determines the probability of an outcome based on significance — i.e., how likely it is that data can be explained by something other than pure chance. Frequentist statistics are limited in that they don’t tell us whether our variation is better or worse, only that the observed result wasn’t an accident.
The Bayesian method, however, uses prior knowledge to establish the likelihood of each variation actually winning, updating the prediction as the experiments run. This type of informed testing allows the experimenter to reach a conclusion sooner and with a smaller sample size.
Unlike the frequentist method, which only tells you the likelihood that the result wasn’t accidental, Bayesian testing tells whether the variation is better or worse than the original.
Doing all of this requires more computational power, which is why the Bayesian method hasn’t been widely used. New technology and vast amounts of data have made it a realistic option.
Using the Bayesian method is an interesting differentiator. But there’s no proof that Bayesian is more or less accurate than the frequentist method. There are devotees of both.
It’s important, however, to understand the difference between what a frequentist result means, versus a Bayesian one.
A frequentist result, for example, might tell you that your hypothesis has a 95 percent chance of being significant. The Bayesian result might read like this: “The variation X has an 83.2 percent chance of being better than the baseline.” It can take some getting used to.
For more, read “Bayesian vs Frequentist A/B Testing – What’s the Difference?,” an excellent article on the ConversionXL blog.
Getting Started with Google Optimize
The link to the Optimize site is readily available, in the upper right corner of your Google Analytics interface. Clicking on the Optimize icon takes you to the site, to set up an account.
Setting up Optimize requires no more effort than setting up Google Analytics or Google Tag Manager — and even less if you already use Tag Manager. As with all Google tools, you only need a functional email linked to a Google Account.
Registering Optimize and setting it up is easy. You’ll see a dialog box, similar to the one in Google Analytics, where you fill in your basic data. The rest of the process is as simple as following the checklist.
After you set up your account and add more users, if necessary, you will have to link Optimize to the Google Analytics property (website) on which you want to run experiments. This step is required since Optimize uses Analytics for reporting and analyzing test results.
To make Google Optimize work for your website, you will need to install the Optimize snippet, which can be done in one of two ways.
First, if your Google Analytics code has already been installed independently, you will need to update it with one line of code:
ga('require', 'GTM-XXXXXXX');
This line should be placed within the Google Analytics snippet:
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-64683703-2', 'auto');
ga('require', 'GTM-P3MLH2P');
ga('send', 'pageview');
</script>
Second, you’ll need to implement a hiding script on your website. It’s not strictly necessary, but is recommended. Hiding scripts serve to minimize flicker, an effect that shows the original web page to a user before the variation is loaded. Google provides the hiding script as well; place it as high in the header of your web page code as possible.
Google Tag Manager
Optimize can also be installed using Google Tag Manager. Although Google recommends implementation outside of Tag Manager framework, it is simply not an option for many sites. Besides, implementing and deploying Optimize using Tag Manager is simpler, faster, and does not depend on cooperation of front end developers.
To implement with Tag Manager, create a new tag (in Tag Manager) using the predefined tag configuration for Optimize.
The hiding snippet should also be implemented using Tag Manager. However, to ensure that the snippet is as high in the header code as possible, place the hiding snippet in a new custom HTML tag.
<style>.async-hide { opacity: 0 !important} </style>
<script>(function(a,s,y,n,c,h,i,d,e){s.className+=' '+y;h.start=1*new Date;
h.end=i=function(){s.className=s.className.replace(RegExp(' ?'+y),'')};
(a[n]=a[n]||[]).hide=h;setTimeout(function(){i();h.end=null},c);h.timeout=c;
})(window,document.documentElement,'async-hide','dataLayer',4000,
{'GTM-XXXXXX':true});</script>
Be sure to change the GTM-XXXXXXX line that contains your Optimize account ID number with your Google Tag Manager ID. This ensures the snippet will be placed correctly, to avoid flicker.
When this is over, you are ready to start the experimentation process.