Googlebot
Google's web crawler, which fetches pages for Google Search, mostly as a smartphone browser.
Googlebot discovers URLs from links and sitemaps, fetches them and passes them on to be rendered and indexed. It crawls predominantly with a smartphone user agent, in line with mobile-first indexing.
Anything claiming to be Googlebot can be verified with a reverse DNS lookup; scrapers often fake the name. Google publishes separate user agents for other products, and Google-Extended controls use of content for its AI models.
Crawling & Indexing timeline · Latest crawling & indexing news · What Google said