Google’s mobile crawler Googlebot-Mobile has traditionally focused on content designed for feature phones and their extremely limited browsers. Google recently announced, however, a new version of Googlebot-Mobile that crawls using an iPhone user agent, enabling it to discover content designed for smartphones, which are more robust than feature phones.
As an avid searcher and a search engine optimization professional, I’ve been disappointed in Google’s treatment of smartphone search results — I addressed here a few months ago, in “Google Says Smartphone Sites Aren’t Mobile.” When a searcher takes the time to tap in a query on their smartphone’s keyboard, the search engine should reward that searcher’s effort by favoring smartphone content over desktop content. Instead, Google has left the task to each individual site to detect smartphones and serve the appropriately formatted content — with mixed results.
Walmart’s Smartphone Woes
For example, let’s say I want to check Walmart’s prices on electronics. If I search on my HTC Thunderbolt smartphone for Walmart and click on the first search result to go to www.walmart.com, Walmart very kindly detects my smartphone browser’s user agent and redirects me to mobile.walmart.com — see the image below on the left. From there I can search walmart.com for electronics, and find a smartphone-formatted page for electronics — see the image below in the middle. But if I instead click on Google’s result for Walmart’s electronics page, Walmart serves my smartphone a desktop version of the electronics page — see the image below on the right. Walmart only detects smartphone browsers on its home page, leaving the rest of its millions of pages to present a negative mobile search experience.
Google’s new Googlebot-Mobile for smartphones will theoretically take care of this poor smartphone search experience. Assuming Googlebot-Mobile can crawl Walmart’s smartphone site, it will discover the electronics page formatted for smartphones and be able to serve that to smartphone searchers instead of the desktop version. However, given the relative scarcity of text on Walmart’s smartphone site and its heavy reliance on internal search versus a category browsing structure, it’s unlikely that Google will find Walmart’s smartphone site either relevant or crawlable.
If Walmart gets smart and begins redirecting all of its desktop pages to the appropriate smartphone pages for smartphone browsers, Google’s job is much easier. Clearly Google already understands which page on the Walmart site is most relevant for desktop searches for electronics. If Walmart redirects smartphone users to the mobile version of that page, Googlebot-Mobile will also be redirected and will be able to associate the less SEO-friendly smartphone page with the stronger desktop page. Google will then skip the desktop page and send smartphone searchers straight to the smartphone version, saving searchers about a second of load time by bypassing that redirect.
Making Sites Smartphone Search Friendly
Technically, Googlebot-Mobile for smartphones crawls using an iPhone user agent. If you already have a smartphone site, the simplest way to take advantage of Google’s new smartphone crawler is to detect smartphone browsers on every page and serve them a smartphone version of that page. It’s good for search, but it’s also best practice for smartphone users coming to your site from direct links, email campaigns, banners, and any other marketing channel.
The key is to treat Googlebot-Mobile exactly the same way you’d treat any smartphone user. For example, serving Googlebot-Mobile a heavily optimized page and other smartphone users a different page would be a violation of webmaster guidelines and would likely get your site penalized for cloaking, the practice of serving different web pages for search engines versus humans.
If your site already detects Googlebot-Mobile for the purposes of serving feature phone content and you want to alert Googlebot-Mobile to both your feature phone and smartphone sites, then you’ll want to update the user agents for the different versions of Googlebot-Mobile.
Feature phone Googlebot-Mobile:
SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/220.127.116.11.c.1.101 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
- Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)
Note that both versions are called Googlebot-Mobile, but the device it’s associated with determines which type of mobile content to serve.