Practical Ecommerce

Does Visual Search Improve Ecommerce Conversions?

Last month I addressed the rise of visual search for ecommerce. Many larger merchants now offer the feature as part of their mobile app. This holiday season gave me an opportunity to test the capability, for insights into the state of visual search and whether it appears to increase conversions. I encountered quite a few problems, however. What follows are my observations.

10 Problems with Visual Search, for Ecommerce

1. Incorrect product type. It was very common for a visual search function to assume a product type, without confirming or asking me for input. It was frequently wrong. For example, at the Neiman Marcus mobile app I uploaded the photo below of a blanket.  The search results displayed shirts with a similar pattern. The visual search function did not ask for the product type. It assumed, incorrectly, that I was interested in shirts.

When the author uploaded a photo of a blanket on the Neiman Marcus mobile app, the visual search results incorrectly displayed shirts (below) with a similar pattern.

When the author uploaded a photo of a blanket on the Neiman Marcus mobile app, the visual search results incorrectly displayed shirts (below) with a similar pattern.

Visual search results at NeimanMarcus.com incorrectly displayed shirts when the author upload a photo of a blanket with a similar pattern.

Visual search results at Neiman Marcus’s mobile app incorrectly displayed shirts when the author upload a photo of a blanket with a similar pattern.

2. Frequent “no results found” message. Text-based search has evolved to where search results will produce recommendations or similar products when there is not an exact match. This was not my experience with visual search. Several retailers apparently had a limited set of images available as part of search.  Hence, I encountered the “no results found” message frequently.

The "no results found" message appeared frequently, as several retailers — including NeimanMarcus.com, shown here — apparently had a limited set of images available for visual search. This could have been avoided by showing recommendations or a list of best-selling products instead of a blank page with an error message.

The “no results found” message appeared frequently, as several retailers — including Neiman Marcus’s app, shown here — apparently had a limited set of images available for visual search. This could have been avoided by showing recommendations or a list of best-selling products instead of a blank page with an error message.

3. Search algorithm wrong. For some retailers, the visual search algorithm was trying to be too intelligent. It tried to produce accurate results without human input or confirmation. In my tests, however, the algorithm was frequently wrong. For example, in the photo below the algorithm for Nordstrom is demonstrating its intelligence by showing it identified the color, the pattern, and the product type. In fact, the color and product type were wrong. I was offered no option to correct them.

This visual-search algorithm for Nordstrom attempted to identify the color, the pattern, and the product type. In fact, the color and product type were wrong, with no option to correct them.

This visual-search algorithm for Nordstrom attempted to identify the color, the pattern, and the product type. In fact, the color and product type were wrong, with no option to correct them.

4. Incorrect image quality error.  In a few instances, the visual search blamed the image quality for not being able to show any results. But the image was actually of high quality and very clear. It produced results on other retailer sites.

It's frustrating for users of visual search to upload a high-quality image using their smartphone and receive a "image quality" message error, such as this example from J.C. Penney.

It’s frustrating for users of visual search to upload a high-quality image using their smartphone and receive a “image quality” message error, such as this example from J.C. Penney.

5. Search did not work for home goods. Visual search mostly worked well for clothing items. But when I searched for home goods, such as table lamps or flower vases, the search failed. It could not consistently identify the object and it often failed in matching the color of the object in the photo.

6. Phone froze while using app. More than once, for multiple retailers, my iPhone 6S Plus smartphone froze while it was trying to open the camera from the retailer’s app. The app had access to the camera but for some reason it kept freezing, repeatedly. The only way out was to restart the phone.

7. Search was slow. The visual search was slow for several of the retailers. After I uploaded an image the search did not show any results, even after waiting several minutes. I had to  cancel the search.

8. Product recommendations did not match the searched item. This was a frequent occurrence when I searched for a clothing item or a lamp. The recommendations displayed beside the results were unrelated. In the example below, from Home Depot, Nashua Tape is being recommended with comforters and quilts.

When searching for a clothing item or a lamp at Home Depot, the visual search recommendations were unrelated. In this example, Nashua Tape is being recommended with comforters and quilts.

When searching for a clothing item or a lamp at Home Depot, the visual search recommendations were unrelated. In this example, Nashua Tape is being recommended with comforters and quilts.

9. Color match did not work. The visual search for several retailers ignored the color. The search produced results that matched the product type but not the color. For example, while searching for a white towel on the Neiman Marcus mobile app I receive these results.

When searching for a clothing item or a lamp at Home Depot, the visual search recommendations were unrelated. In this example, Nashua Tape is being recommended with comforters and quilts.

A visual search for a white towel on the Neiman Marcus mobile app produced towels, the correct product type. But the colors were mostly brown or blue, not white.

10. Lack of omnichannel integration. Visual search is perfect for omnichannel uses. Shoppers can take a photo of a product, find matching products from a brick-and-mortar retailer’s mobile app, and then pick up the item in the physical store. Unfortunately, none of the retailers I tested that have a physical presence tried to send me to a physical location to pick up the products in the search results. This is a lost opportunity to generate revenue as integrating the physical location based on product availability and proximity to the customer can increase the chances of closing the sale.

If you have opinions or experiences on visual search, please share.

Gagan Mehra

Gagan Mehra

Bio   •   RSS Feed


email-news-env

Sign up for our email newsletter

Comments ( 3 )

  1. Martin Peniak December 9, 2016 Reply

    Next time try cortexica vision systems, the true leader of visual search. You’ll not find any of the issues you listed here

  2. Babu December 14, 2016 Reply

    You can check out VizSeek at https://www.vizseek.com/ to see how we overcome lot of these issues.

  3. Caitlin Crawford January 16, 2017 Reply

    Hi there, I do think there is some truth to your comments. Although I must say that a lot of the big brands are not working with the right companies. There are some smaller players that have excellent visual recognition tech and they aren’t the ones powering Nordstrom, Macys, at least not yet ;) Take Fashwell, a Swiss based company. They only focus on Fashion, and specialize in user generated images so they are really good at finding fashion products in Instagram images. This is a space we need to watch, since I think there will be some companies shaking things up here quick.