Archive for the ‘Analytics’ Category

Saved from Information Overload

January 23rd, 2009 by Cedric Savarese

Inspired by a recent post by 37signals, I set out once more to try to optimize our sign-up funnel on

We track a lot of information with Google Analytics. Goal conversion rates, e-commerce revenues, adword campaigns, search keywords, referrers and so on. Collecting information is easy, but trying to make sense of it can be frustrating.

Google wants me to drown in a sea of data

I often find myself wandering through the Google Analytics reports, marveling at the shiny graphs or noting with curiosity that the middle of the month is always slower. While my intellectual curiosity might be satisfied, I’m still left confused, unsure about which variable to tweak and nothing to act on.

I eventually learned that, like Ulysses who tied himself to the mast to resist the alluring chants of Sirens, diving into the reports requires self-discipline and focus. First identify a specific page, then a metric, write it down and do not look at anything else.  Then go and try to improve it.

Another thing I learned is that you can’t just compare the performance of a page before and after a change and decide whether the change was a good idea or not.

Traffic is not a constant. It varies over time, quite significantly. One week the conversion rate is great, and the next one it sucks even though nothing changed on the site. Perhaps it’s a surge of unqualified visitors, or people who get antsy over the latest economic news, or just a random fluke.

I’m not willing to wait days or weeks for a clear pattern to emerge. If my new page is not performing as well, I’d like to  undo my mistake as quickly as possible.

That’s where A/B testing is really useful. It works by randomly splitting your incoming traffic between 2 different versions of a same page. With the appropriate tracking tool, it’s then very easy to see which page performs better.

Google the merciful rewards the faithful

So I finally decided to take a second look at Google’s Website Optimizer, a free A/B testing service.  I wasn’t convinced the first time around. It was rather limited, and I thought that our esteemed visitors would not be swayed one way or another just because we make our marketing copy a bit more clever. After all, we firmly believe in the greatness of the product we’re building,  shouldn’t it just sell by itself?

Well, it turns out that Website Optimizer got a lot better. It’s very easy to set up, you can test pretty much anything and it provides unambiguous results quickly.

No more trying to decipher the reports and understand what’s going on. No more agonizing over a design decision. Should it be a link or a button? Should it say ‘Sign up now’ or ‘Try it for Free’? Just try both and let Google tell you which one works best.

And when the result  is a 300% improvement in conversion rate, I sure feel stupid for not doing it sooner.