always be testing

I recently came across the book Always Be Testing: The Complete Guide to Google Website Optimizer written by Bryan Eisenberg, John Quarto-Vontivadar and Lisa T. Davis 1.

One of the key takeaways from the book for me was understanding the difference between testing, measuring and optimizing:

testing

When you test, the purpose is to compare elements on your website, or elements of your campaigns, to see which variation is best in persuading your visitors to take the action you want them to take. That action could be anything including clicking a link, making a purchase, signing up for a newsletter, or even going from one page to another in your selling process. Testing involves configuring your site in either an A/B split testing format, or having different pages and making use of multivariate testing methods.

measuring

When you measure, you monitor what visitors do as they navigate your site. By using a web analytics program you can determine the key performance indicators such as site traffic, overall conversion rate, page rejections, length of time spent on a page, and so forth. These analytics (or measurements) help you evaluate where you have problems, which in turn helps you prioritize your testing.

optimizing

When you optimize, you put your test results in place to improve your conversion rate. The output of an A/B split test or multivariate test is a determination of what works best (a so-called “winner”), but optimizing is more than just implementing the “winners” on your site. As you test and measure, you reinforce and refine the plans you developed to create your site or campaign. You grow more familiar with the nature and needs of your audience, which makes it possible for you to create more meaningful options to test so you can continue improving. Optimization requires a human element to access and interpret the “facts” from testing and measuring in order to lead it in more efficient directions.

Although the book was published in 2008 (which seems like decades in internet time) the concepts still survive and I’ll be sharing more insights in the future posts. I’ll also be sharing how these insights can be implemented in WordPress to help you test, measure and optimize your WordPress site.


  1. Eisenberg, B., Quarto-Vontivadar, J., & Davis, L. T. (2008). Always Be Testing (First Edition.). Wiley Publishing, Inc.