Technical SEO

Ecommerce Know-How: Bot Herding, A SEO Internal Linking Tactic

A relatively simple SEO linking strategy can improve how well and how often your store’s product pages are indexed and displayed in organic search engine results.

This linking technique goes by several names. Bruce Clay Inc., an Internet business consultancy, refers to it as “siloing.” SEOmoz uses the term “PageRank sculpting.” And the Search Engine Roundtable called it “bot herding” in a recent discussion. While these names and practitioners claim subtle differences in meaning and application, the general technique seeks to concentrate PageRank on a site’s most important landing and product pages, improving how those pages perform in search engine results.

Ecommerce Know-How: Bot Herding

Bot herding or siloing can be divided into two categories. First, you have structural siloing.

“Siloing is creating solid themes on your site based on linking together all the relevant pages,” said Virginia Nussey, Associate Media Writer and Blogger for Bruce Clay Inc. “A site that has a number of pages on a subject is viewed by the search engine as a relevant result to serve to a user looking for information on that subject. Creating well-supported themes helps increase your site’s subject relevance in the eyes of the search engines.”

Second, there is a sort of virtual bot herding that simply seeks to encourage or discourage search spiders and bots to follow one link or another. In this “eCommerce Know-How,” I will focus on this second—virtual—aspect of bot herding.

The nofollow Attribute

In its simplest form, bot herding, uses the rel=” nofollow” attribute to prevent search spiders and bots from following some links, effectively herding those bots to other links.

<code>&lt;a href="" rel="nofollow"&gt;World's Best Source for Ecommerce Information&lt;/a&gt;

When Googlebot, MSNbot, Yahoo! Slurp (yes that is the bot’s name), and other search spiders encounter this link they will effectively not follow it, instead focusing on other available links. More importantly, these bots won’t transfer any of a page’s ranking power to nofollow links.

Figure showing Ranking Power Distribution to three pages

Don’t Dilute Ranking Power

Exactly how Google, Yahoo!, or MSN rank a page is a trade secret, but it is assumed that each search engine looks at least two aspects of every web page: (1) relevancy and (2) importance.

Relevancy describes whether or not a particular page’s content matches a given search query. If I search for “ecommerce tips,” returning pages from the Practical eCommerce site would be relevant. But if I searched for “Russian restaurant in San Francisco,” those same Practical eCommerce pages would likely be irrelevant.

Importance seeks to measure the value of a page’s content. If, for example, there were two web pages with a similar word count and both included the keyword phrase “ecommerce tips,” how would a search engine know which page was a better result for the search query? The answer: links. Search engines count incoming links to determine how important a web page is. And it is for this reason that bot herding matters.

Imagine that there is a page that has ten points of ranking power and ten links. Search engines will divide those ten points of ranking power evenly across all ten links, distributing one vote of importance to each destination page. If we added ten more links, the total ranking power (number of votes) would stay the same, but each destination page would only get a half (0.5) vote. If we added 20 more links, each vote would be worth a quarter (0.25) vote of importance. And remember importance may determine how high a page shows up in search engine results.

Conversely, if we had a page with ten links, and we “fenced off” five of those links using the nofollow attribute, we would pass two votes to each of the open links and no votes to the nofollow/fenced off links. We would be focusing a page’s ranking power on those links we deemed important, herding the spiders and bots in the direction we want them to go.

“Yahoo! Slurp obeys the rel=’nofollow’ attribute for links,” Yahoo! said in its webcrawler FAQs. “Web publishers and blog owners can apply a rel=’nofollow’ attribute to any hyperlink on their page to indicate that the link may not be an approved or trusted link. Note: Slurp may use a ‘nofollow’ link for discovering content, but the link will not be considered an ‘approved’ link for consideration for ranking of the target page.”

Notice what Yahoo! says above, essentially nofollow links won’t boost a target page’s rank.

Putting Up Fences

By focusing ranking power on some links, we hope to improve a target page’s importance and thereby improve how well it performs in search engine results.

So now it is time to put up fences on your site, herding search engine spiders and bots—or more importantly ranking power—to links that matter. For example, if you have a product category page with several links to product detail pages, you want push as much ranking power to those product detail pages as you possibly can. But I will bet that those same product category pages all have links to your store’s privacy policy, terms and conditions, contact us page, about us page, shopping cart, etc. None of these links need to perform well in search engines and, therefore, not an ounce of ranking power should be headed their way.

Put a nofollow fence on all of these sorts of links:

  • Privacy policies
  • Terms and conditions
  • Customer service
  • About us
  • Contact us
  • Newsletter registration
  • Shopping cart
  • Contests
  • Links in product reviews

Focus Ranking Power

Use the rel=”nofollow” attribute on links to any pages that don’t need to perform well in search engines and focus ranking power on your product pages. The technique is simple and the results can be huge.

Bot Herding Video


Armando Roggio
Armando Roggio
Bio   •   RSS Feed