Customer Conversion: Testing Brings Science To Marketing

There are two things every ecommerce business should strive to achieve. The first is increasing conversion, turning a higher percentage of browsers into buyers. The second is optimizing marketing spend, driving higher rates of return from existing advertising dollars being spent.

The most consistent way to increase these two key metrics and grow an ecommerce business is through continuous testing. By testing variations of web pages and by achieving slight increases in efficiency, an ecommerce business is able to generate substantial, incremental profits.

Testing is fundamental to selling

Success in ecommerce is achieved when retailers “beat their control.” The term “beat your control” has been used in the direct marketing world for decades and is a tactic that describes the continuous testing and optimization of content, copy and page layout to profitably grow sales.

What a direct marketer strives for with testing is to improve upon the previous month’s or week’s metrics. By doing so, the business gains efficiencies as it relates to customer acquisition and retention.

Typically, online retailers have utilized standard A/B tests, otherwise known as “split tests,” to try and improve marketing and merchandising approaches. An A/B test allows a retailer to split traffic across two different versions of a web page in an effort to identify topperforming marketing variables.

During these testing scenarios marketers try to identify promotions that significantly outperform others, while testing titles and headers for the highest level of customer engagement.

For example, let’s say a business was trying to run a split test for two different landing pages within a paid-search campaign. On each page, a different headline was shown above the primary image. Because no other variables were changed within test, a business can attribute the performance increase (typically sales/conversion) to the unique variable on the page. This allows the business to maintain test integrity and to pinpoint the exact factors that caused the improvement.

The problem is that an A/B test can only track one variable at a time. Multiple tests must be done to construct ideal and optimized pages. Findings from an A/B test can only be statistical-ly valid for conversion improvement when testing one distinct variable at a time. It’s a painfully long process that can eat up your valuable time.

Multivariate testing – The future is now

As online retail has progressed during the last few years, so has the sophistication of ecommerce testing. No longer are marketers handcuffed with the limitations of A/B testing, since multivariate tests (multiple variables being tested at once) have become less costly and easier to implement within an ecommerce business operation.

The core difference between multivariate tests and standard split tests is that multivariate tests allow an ecommerce business to concurrently test numerous variables of a web page to identify what variables perform best and in which combinations. The goal is to use the test data to create the most engaging and compelling page possible so visitors shop at an improved conversion rate.

For each page variable (e.g., landing-page headline), two or more versions are tested. In some instances a business may have up to seven unique versions of each variable. When the test is deployed to visitors, the variables are configured into “recipes” and have a unique mix of the different testing variables. The page versions are randomly split among traffic so that highest performing variables/recipes can be identified.

Here’s an example to simplify the concept. Let’s say a home-products retailer wants to utilize a multivariate test for the sofa-category page. The marketer who is running the test wants to see which combinations perform best for the variables of headlines, promotions and imagery, as described below.


A headline should be the core value proposition that should be communicated. The business we are using within this example could test the following:

  • Comfort, Elegance and Value.
  • 100 percent satisfaction on all sofas … Guaranteed.
  • Simply the best in sofa design.
  • Wide varieties of brands available.
  • All three of the headlines have a unique message and will likely resonate differently with shopping segments. Typically the corevalue proposition will be located with or integrated into the primary creative within a page.


Delivering timely promotions that meet customer desires are a fundamental aspect of ecommerce success. Varied promotions perform at different levels, and a marketer must test varied types to identify which ones drive the most incremental business. Examples could be the following:

  • Free shipping over $200.
  • 20 percent of all leather sofas.
  • Free coffee table with love seat and sofa purchase.


Ecommerce businesses owners often underestimate the importance of images within the consumer’s purchase decision. Images create emotion within a web shopper and should portray the value of a product. Obviously all images should be of high quality and resolution, but in our example situation above, a furniture business may want to test three different images that reflect the quality of the products and the feeling of the brand.

With nine different variables within our example, running A/B tests would take an extremely long time to gain insights into the best performing creative. A multivariate test, however, instantaneously swaps out all of the variables mentioned and identifies the best performing “recipe” for ecommerce success.

Multivariate testing will allow your business to use science when making marketing and site-merchandising decisions. Don’t guess during your next big marketing push — test the variables that are critical to your messages so your business can achieve improvements to key performance metrics.

PEC Staff
PEC Staff
Bio   •   RSS Feed