Using Real People to Test a Website

Knowing how consumers interact with an ecommerce site can be greatly beneficial. There’s a company that provides real-time user tests, where humans navigate through a site and record their experiences. The results can identify potential problems, which merchants can then correct to improve their conversion rates.

The company is UserTesting.com. We spoke with its vice president of marketing, Chris Hicken, about the company and the user-testing process.

Practical eCommerce: What is UserTesting.com?

Chris Hicken: “It’s easier to understand what we do in the context of the problem that we are trying to solve. Most websites have a 3 percent conversion rate. That means that for every 100 visitors that come to your site, 3 of them are going to go on to place an order, or sign up for a newsletter, or whatever you determine to be a conversion.

“The question for website operators to answer is, ‘Why are the other 97 percent of the people leaving the site without buying?’ That’s where our service comes in. We make it easy for website owners to find people that are a target audience and watch them perform common tasks on their websites. Some tasks might include searching for a product or placing an order or signing up for a newsletter. Through that process, you are going to discover how and why people get stuck on your website and why they end up leaving your site without buying. That’s what we do.”

Chris Hicken

PEC: Is it mainly conversion testing that you are looking at, or is it other types of testing?

Hicken: “I would say it is heavily focused on usability. But in the context of your audience, most people that are online retailers will use us to figure out why people are leaving the website without buying. So it’s really conversion testing service.”

PEC: What’s that testing process going to cost our readers, ecommerce merchants?

Hicken: “It’s $39 per person that tests your site. Typically a normal usability test will be about 15 minutes long. All that you need to get started is a URL that you want to test and a list of tasks that you want someone to perform. For example, lets say a company like Amazon.com wants to watch five people place an order for a TV on its website. They’ll instruct our users to start on Amazon.com and they’ll probably say, ‘Use whatever method you’d normally use to search for a TV that you would buy for yourself. Then go through the process of buying it.’ That’s really it.

“Then, we’ve got a panel of people located all across the U.S., Canada, and the U.K., and on a first-come first-served basis people will accept that work from Amazon.com and they will record their computer screens as they perform those common tasks at Amazon.com.”

PEC: You’ve just mentioned Amazon. Can you tell us other ecommerce clients that use your service?

Hicken: “A lot of the big online retailers use us, including Walmart, Amazon, Staples, Dell, HP, Groupon. But because our service is relatively inexpensive, we also have start-up companies using us just to get some early feedback about how good or bad their website is. And also medium-sized retailers often use us to improve their website usability and the conversion rates.”

PEC: There are other testing methods, such as split testing, multivariate testing, that sort of thing. Google offers these types of tests for free on its Website Optimizer. Why should merchants pay for your testing when they can do those other forms of testing for free?

Hicken: “I would highly recommend all merchants use Google Analytics and Google Website Optimizer — all those tools that are free. They work very well with our service. You can think about user testing as a problem discovery tool. So as you are watching people go through your site, you are going to discover where people are getting stuck and why. That will give you an idea of the things on your site that need to change first. So you are going to figure out what the low hanging fruit is first.

“You can discover, for example, that the navigation on your website is really confusing for users by using our service. Then you’ll go on to build a split test, an A version and a B version of your site — maybe multiple versions. You’ll use Google’s Website Optimizer to test the performance of the new design that you come up with.”

PEC: Can you give us some examples of ecommerce companies that have used UserTesting.com and made changes to their sites based on what they learned?

Hicken: “Yes. A major online retailer used our service to do some testing of a new shopping cart that it had released and it discovered two major problems with the cart. The first one was that all Internet Explorer users received a security warning in the shopping cart. The error said something like, ‘This page is not secure. Would you like to continue?’ And that’s a major no-no in shopping cart development. So that was the first problem they discovered.

“The second one was that as users went through the shopping cart process, they could either check out as a guest or they could check out using their existing accounts. Our testers found when they clicked the ‘check out as a guest’ button, they were taken to a page where they were asked to fill in information as if they were registering to be a regular user. That was really frustrating people as they went through the checkout process, because when they clicked ‘check out as a guest’ they expected not to have to register for an account. So the retailer went on to fix both of those problems. It experienced major improvements in consumers getting through their shopping cart process.”

PEC: In that example, how many testers did the retailer use to ascertain those findings?

Hicken: “It took five users to discover those problems. The retailer did run another set of ten users to validate the findings. But the initial findings were done with just five users.”

PEC: Walk us through the process of doing all this with your company. How does a merchant sign up? How long does it take? How are the results delivered? That sort of thing.

Hicken: “You start by going to UserTesting.com. There will be a button that says, ‘Try It.’ The four pieces of information that you need to have are: (a) the URL you want users to start, (b) the scenario you want users to assume, such as ‘Imagine you want to buy a TV,’ (c) a list of tasks you want the users to perform, and (d) the number of testers that you’d like to use.

“For an early test, if it’s your first time doing this. I’d recommend just starting with three testers to make sure you get some interesting results. Of course if you like it, you can keep doing more tests.

“In terms of what you can expect to get back, you’ll get a video recording of each tester’s computer screen and audio. You will be able to hear them speaking their thoughts out loud as they use your site. You get about a 15-minute video recording and you also get written responses — up to four questions that are answered after the test is over. So if you want to ask some questions about how hard or easy it was to use the site, you can do that.”

PEC: How long does that take?

Hicken: “Generally you will get results back within 1 hour of placing the order on our site. You can place an order just about any time in the day and get results back in an hour.”

PEC: How are those results delivered to the merchant?

Hicken: “You will get an email notification that the results are ready for you to review. To watch them, you will go to UserTesting.com and log in to your account.”

PEC: Speaking of UserTesting.com, tell us about the company. Where is it located? When was it founded?

Hicken: “The company was founded in 2007 by Dave Garr and Darrell Benatar. We are located in Silicon Valley — Mountain View, California.”

PEC: Anything else?

Hicken: “Our tool is used in three common ways. First, people test their competitors’ websites to see what their competitors are doing better. You can watch someone do three tests on your site and then another three tests on your competitors’ and see what their websites do better than yours. That’s the first thing.

“The second thing is website operators will watch testers begin their searches on a search engine — such as Google or Bing. For a lot of online retailers, they know when they get consumers coming to their sites, but they don’t understand why someone has clicked on their pay-per-click ad or why someone has clicked on their link in the search engine results. So you can have users begin their testing process on Google. You can watch which link catches their attention and why and which ones they clicked on.

“The third thing is we also have a mobile device testing service. If you’d like to test your mobile website and see how easy or difficult it is to use on an iPhone or Android device, you can use our service to do that.”