Knowing how search engine crawlers interact with your site is crucial for organic rankings. The activity of crawlers reveals discovered web pages, followable links, accessible code, and more.
Many commercial tools emulate the behavior of search crawlers. But none beats data straight from the search engines themselves: Google Search Console and Bing Webmaster Tools.
Both tools are essential for search engine optimization. They are free and connect to your site easily.
Google Search Console
Search Console has many useful reports for both beginners and experienced SEOs. The tool augments basic search info with specifics about how Google’s bot has interacted with the site.
To get started, verify your site with Search Console by connecting it to the Google Analytics account or using a plugin.
Search Console discloses the pages Google has indexed and provides helpful tips on improving other metrics, such as click-through rates, page speed, and more. The platform also shows keywords that users search to find your site.
I listed last month six critical Search Console reports. But there are other valuable sections.
- The “Enhancements” tab addresses your rich snippets and whether Google has used them. The section shows only the snippets you quality for. For example, you won’t see “Product” in this section unless you’ve applied “product” markup on your site (using Schema.org or similar).
- “Removals” allow site owners to request the removal of any URL from Google’s index.
- “International targeting” (Legacy tools and reports > International targeting) allows site owners to specify a country to rank in.
- “Change of address” (Settings > Change of address) alerts Google of upcoming domain name changes to mitigate the impact.
Bing Webmaster Tools
Bing Webmaster Tools provides helpful (and free) SEO tools. Bing provides an easy way to import your data from Google Search Console into a Webmaster Tools account. Verification is near-instant. Populating all reports can take up to 48 hours, in my experience.
The appearance of Webmaster Tools is similar to Search Console. Site performance reports include search queries and clicks. Other reports show crawl errors and inspection results of any URL to ensure search engines can access and index it.
For indexed URLs, Bing offers an enhancement report with optimization tips and missing elements.
“Site Explorer,” another helpful feature, provides an overview of your site’s structure and a detailed review of each section or URL. It’s essentially a free crawler to see how a search bot might interpret your site and its hierarchy. Site Explorer shows crawl errors, a page’s size and header response, and mistakes on Robots.txt that prevent crawls.
Webmaster Tools’ “Keyword Research” report can uncover new keyword opportunities and competitors. Click any word or phrase in the list to see:
- Related keywords,
- “Question keywords” those in a question format to include in your copy,
- Newly discovered and trending keywords.
Unlike Search Console, Webmaster Tools does not assess your site’s security. But it does disclose the status of a site’s copyright infringement claims, wherein a verified owner has filed with Bing a copyright infringement notice against another site. It’s an example of the two tools complementing each other.
Google Search Console and Bing Webmaster Tools can help site owners improve organic traffic. Both provide info on their own search engine. Yet each offers a different angle, making it a good idea to use them concurrently.