The stories about Crawling & Indexing from this period, ranked by likely impact on a single site.
Ranked by estimated impact on a single site, not by publication time.
Google Search seems to be having indexing or serving issues in the past hour or so. New content from the past hour doesn't seem to be showing up in Google Search results. Content from the Wall Street…
Crawling & Indexing
A page that isn't linked from anywhere, sits behind a login, or requires a form submission to reach essentially doesn't exist to a search engine, no matter how well-written its content is.…
Crawling & Indexing
You’ve probably noindex-ed pages you don’t want showing up in search. It’s a solid practice for product variants or paid destination pages, and it works for Google rankings.
Crawling & Indexing E-commerce
Not every page on a site needs to be — or should be — indexed and competing for search visibility. Understanding when strategic noindexing genuinely helps versus when it's unnecessary or…
Ranking & Algorithms Crawling & Indexing
Crawl budget refers to the finite amount of attention and resources a search engine allocates to crawling any given site within a given period. For small sites this is rarely a limiting factor, but…
Crawling & Indexing
I have had a number of site owners reach out to ask for help with indexing issues lately. In most cases, I am finding that Google has classified their pages as “crawled-not currently indexed” in the…
Crawling & Indexing
Your traffic dropped after a migration and nobody can explain why. Important pages sit in “discovered – currently not indexed” […]
Crawling & Indexing Site Architecture
When a technical SEO audit involves tens of thousands of pages, the constraint is…
Crawling & Indexing
Search News You Can Use May 12, 2026 Here are the topics I have been discussing with my clients recently. 1. A rise in crawled - currently not indexed pages There has been significant chatter on…
Crawling & Indexing
Search indexing was built to help humans decide what to read. Grounding is being built to help AI systems decide what to say and simultaneously helping people to verify, refine, or challenge what the…
Ranking & Algorithms Crawling & Indexing AI Search & Visibility
We’ve packed our bags, triple-checked the weather forecast (don’t want to jinx it, but it’s looking 👌), and we’re getting ready to crawl our way down to the coast. The Screaming Frog team is…
Crawling & Indexing
As SEOs, we obsess over performance. Speed. Crawl efficiency. Core Web Vitals. Conversions. But we rarely talk about the environmental cost of what we ship. The internet now accounts for roughly 4%…
Crawling & Indexing Speed & Core Web Vitals
Assembled from our own archive of 551 stories, which currently runs from 4 October 2023. Ranking is by the impact score described on the front page — an estimate of how much a story is likely to affect one website, not a measure of how much attention it got.
The archive only holds what it has seen. Periods before it started are empty rather than quiet, and it gets more complete every week it runs.
