Using Google Analytics with Google Website Optimizer
Google Website Optimizer is great. It's a powerful tool for determining what kinds of changes to make to your site. Statistics-based recommendations are an analyst's dream.
The only drawback with GWO is that it's a binary measurement: did the visitor convert or not? Granted, you have some flexibility in defining what that "conversion" is, but it's still not a holistic view of what's really happening. You also have to predefine any measurements that an experiment may impact, and that's simply not possible in every scenario.
The solution to this is to use Google Analytics in conjunction with GWO.
How to Put Website Optimizer Data into GA
Integrating Website Optimizer with Google Analytics means that you will need to be able to see within GA which combination of an experiment a visitor saw. Put this into a custom variable by using a function from the Website Optimizer library,
utmx('combination'). This will return the combination number the visitor is viewing, where zero is the original. (For a description of each combination, see the Website Optimizer reports.)
On the page you are performing the experiment, place code like this before
_trackPageview() and after the first block of Website Optimizer code.
pageTracker._setCustomVar(1, "GWO", utmx('combination').toString(), 1);
This code checks first whether the utmx function has been declared yet. Then it creates a visitor-level variable (that gets stored in the cookie) in slot 1. The name is GWO. The value is the combination number. We have to either convert it to a string (so that GA doesn't dismiss zeroes), or we have to concatenate it to a string, like
"Combination: " + utmx('combination').
Choosing the right variable scope
A word should be said about choosing the scope of the custom variable that stores this combination. The example above uses a visitor-level variable, but that's not the ideal solution for every case. The ideal solution will depend on what your goals are, but here are some guidelines.
- If you are running multiple experiments on the same site, you will either want to use page-level custom variables, or assign each experiment to a different variable slot.
- If an experiment affects multiple pages on the site (eg. a navigation menu, call to action buttons, etc.), a visitor- or visit-level variable is probably best.
- If an experiment only affects one or two pages on the site, a page-level variable will be most accurate, because it's not likely that one page will impact much else about the visit.
Although it's beyond the scope of this article, it's worth mentioning that because the utmx function is not object-oriented, if you are doing multiple experiments on a single page, you will need to set the custom variables immediately after each block of Website Optimizer code that you want to track in GA.
Analyzing Website Optimizer Experiments in Google Analytics
With this integration in place, you will now be able to do more thorough analysis of each variation of an experiment.
Considering the site's goals, create a custom report where the value of each custom variable is the dimension and key KPIs are the metrics. Here is a sample report. (Click on it for a higher resolution image.)
Since these variations are delivered randomly, if they had no impact on anything else, one would expect all of these numbers to be pretty similar. However, some of these metrics seem to be influenced by which variation is shown.
Create advanced segments for each of the variations. This will allow you to see in greater detail exactly what different visits look like when exposed to a different variation of the experiment. (For example, if your experiment is testing different variations of the navigation menu, you can see clearly how each variation affects visitor click-paths.)
By digging deeper into each combination and doing further segmentation, we can begin to decide why different variations perform differently and whether Website Optimizer's suggestion is, in fact, the best option.