Use Hypotheses to Optimize Conversions

In “Conversion Optimization: Research First, Then Test,” I addressed the four types of conversion research to conduct before commencing A/B or other tests. When that research process is complete, the first order of business is to identify the issues that hinder your visitors converting into customers. Once you know what those issues are, it is time to think what the solutions could be.

Say, for example, that your research identifies these conversion problems:

• Google Analytics uncovers that mobile visitors have lower conversions than desktop ones;

• There are leaks in your conversion funnel in the payment step;

• Google Analytics shows the proportion of female and male visitors is roughly the same, but male visitors convert better.

Note that I’ve identified, hypothetically, these conversion problems using the research process that I explained in the “Research First, Then Test” article.

Using heuristic research — i.e., the website conforms to best practices and principles of user interface design — you may uncover that mobile visitors cannot find how to add products to the cart and that they have a hard time navigating through product categories. This is the type of issue that heuristic research can uncover. It allows you to improve user experience in order to make it easier for visitors to fulfill their goal.

Using quantitative research — i.e., research that relies on metrics — you find that leaks in your conversion funnel mostly occur in the payment step. It may indicate several problems, such as lack of trust, lack of payment options, or expensive shipping.

To find out which one of these is the main issue, you can conduct qualitative research — i.e., determining why visitors take a certain action. This could be, for example, asking visitors in a survey what may have stopped them from completing transactions and similar questions.

Technical research — i.e., ensuring that the website is functional and enables all the visitors to convert — I’ve left out. I’m assuming that the site is functional.

Hypothesizing

Having an idea for solving a problem is different from creating a hypothesis.

An idea is a vague notion of how to fix a problem, such as “we noticed that dropout rate in the funnel at the payment step is 80 percent. We presume this is due to credibility issues, so trust factors should be improved.”

This does not give us much to work with. It may identify the problem and give some indication in which direction we should seek for solution, but nothing specific.

Hypothesis, on the other hand, should tell us more information, including specific ways that the issue can be resolved, with the issue’s priority. The idea is to create an explanation of why the observed issue happens, whom it affects, and what areas of the website are affected. The hypothesis should also contain information on how thoroughly it would improve or solve the issue.

Hypothesis, on the other hand, should tell us more information, including specific ways that the issue can be resolved, with the issue’s priority.

The most important part of any hypothesis is the proposed solution that will remove the observed issues. Without proposed solutions, a hypothesis is worthless.

An example of successful hypothesis would resemble the following.

• Observed issue. Visitors drop out at the payment step of the conversion funnel.

• Number of visitors affected. Dropout rate is 80 percent, affecting approximately 3,000 visitors.

• Scope of the issue (what part of the site it affects). Checkout, the process of paying for the purchase.

• Proposed solution. Adding additional payment options, trust indicators, and a discount for new customers.

• Effort to implement solution. Moderate.

• Confidence this solves the problem. High.

This list allows you to estimate the importance of hypothesis. If you had but one hypothesis for the entire website, it would be a non-issue and you would not be concerned with importance of hypothesis.

But, websites are complex. Solving issues typically requires multiple hypotheses. You will likely need to order them according to a criteria. The primary objective of an ecommerce website is to create revenue; an optimizer’s task is to increase it. It makes sense, therefore, to start optimizing by solving the issues that have the greatest potential to increase revenue with least amount of effort.

Prioritizing Hypotheses

To prioritize the hypotheses — focusing on solutions that improve conversions in the shortest time — create a list of criteria that allows you to rank the hypotheses.

Conversion-rate-optimization practitioners have come up with multiple frameworks to do this. What follows are some of the most popular.

PIE. This framework was developed by Chris Goward, a pioneer of conversion optimization. According to this framework, you rank hypothesis by Potential (for improvement), Importance, and Ease. Each of these criteria is assigned a numerical score from 1 to 10, with 10 being the best. The hypothesis that scores largest aggregate score, will be tackled first.

Potential refers to how much improvement implementing the hypothesis will create. Importance designates the overall value of the pages we test according to number of visitors and revenue they bring in. Ease tells us how easy it is to make actual changes to the page we want to experiment on.

The advantage of this method is that it is relatively simple and quick to establish priorities and, consequently, a testing program to conform to it. To execute the PIE method, simply create a table with list of hypothesis and enter their score.

The main disadvantage of PIE is that it includes Ease, an arbitrary element. Ease includes not only the effort and resources necessary to effect changes to the page, but also the “political” element of how acceptable the change will be. This arbitrariness may interfere with objective valuing of the hypothesis and result in skewing the priority list to conform more to the influence of opinions than data.

TIR. This framework for prioritization ranks the hypothesis according to Time needed to implement it, the Impact it may have on conversions, and the Resources necessary to implement in terms of skills and tools.

Each of the criteria is ranked numerically from 1 to 5, with 5 being the highest. The final ranking is the sum of the three scores. The main advantage of this framework is that it has less subjective elements, although “Impact” remains a guess.

This framework is also tied into a larger framework of Plan, Measure, and Improve, which represents the entire method of research used to identify issues.

ICE. Impact, Confidence, and Ease refers to the model from GrowthHackers, a consulting firm, that ranks a hypothesis according to impact the solution will have on conversions, confidence that the solution will actually work, and ease of implementation.

Criteria are ranked from 1 to 10, with 10 being the best score. There is another version of this framework that evaluates hypothesis according to Impact, Cost, and Effort. This latter variation tends to work better, because it avoids the subjective Confidence criteria.

PXL. PXL is a framework developed by ConversionXL, a conversion-rate-optimization agency. The idea is to expand the criteria to cover every conceivable angle, starting with the solution on the page. It also looks into whether the solution is supported by data, and what the goal is.

PXL is customizable and additional elements to grade can be added as required. The main advantage of this framework is that it tends to be objective and quantifies every criteria. Its drawback is that it is a bit complicated and requires some work to actually create.

To prioritize the hypotheses — focusing on solutions that improve conversions in the shortest time — create a list of criteria that allows you to rank the hypotheses.

Why Prioritize?

When you complete the research and create a list of hypotheses, prioritization allows you to take the shortest and most efficient route to the solution, which will increase revenue.

Creating a list of hypotheses in this way can easily be translated into a testing program. If prioritization is done correctly, it can bring fast results.

While research is an important step and a necessary precondition to optimizing conversions, unless it is followed by hypotheses that propose solutions, it is worthless. And while testing is possible without the proper hypothesis, the results will be haphazard at best and useless at worst.

The most important thing is to use data, not opinions, to create solutions. That is, after all, the purpose of a hypothesis: eliminating the guesswork from testing.