Web Search - Googlebot

What is Googlebot ?

googlebot is the crawler bot of Google that search and feed the index of the Google search engine


When Googlebot renders a page, it flattens:

Googlebot see only content that's visible in the rendered HTML.

Check it by looking at the rendered HTML with:



Crawl Rate

By default, Googlebot will crawl every 90 days (Ref) but you can notify it of a change at the Google Search Console (manually or via the API)

Crawl URL parameters

How to know if the bot is Google Bot

To verify Googlebot as the caller:

Server:  amplifi.lan

Name:    crawl-66-249-66-1.googlebot.com
  • Verify that the domain name is in either googlebot.com or google.com
    • The domain name is crawl-66-249-66-1.googlebot.com : check
  • Run a forward DNS lookup on the retrieved domain name using:
    • the host command on linux
    • the nslookup command on windows
nslookup crawl-66-249-66-1.googlebot.com
Server:  amplifi.lan

Non-authoritative answer:
Name:    crawl-66-249-66-1.googlebot.com
  • Verify that it is the same as the original accessing IP address from your logs.
    • check !

Documentation / Reference

Powered by ComboStrap