Indexing is the process where a search engine analyzes a crawled page, evaluates its quality and uniqueness, and decides whether to store it in its searchable database — the index. Being crawled is not the same as being indexed, and a real share of crawled pages never clear that second bar. This article is part of the What Is SEO? guide and goes deeper into indexing specifically than the overview of how search engines work.
The Basics, Quickly
Once a crawler reads a page, Google evaluates it before deciding whether to keep it — analyzing the text, images, and other content, checking whether it duplicates something already indexed, and applying a basic quality filter often referred to as index selection. Only pages that clear that bar get stored in the index, which is what actually makes them eligible to appear in search results at all. For the full three-stage picture, see the dedicated guide; this article covers what happens specifically at this stage, and why it doesn’t always go the way a site owner expects.
Why “Crawled” Doesn’t Mean “Indexed”
A crawled page can still get excluded from the index for reasons that have nothing to do with being blocked. Common ones: the content is thin or largely duplicates another page, the page was crawled but Google decided it wasn’t worth indexing yet, or the page explicitly told search engines not to index it. This gap between crawling and indexing is one of the more common sources of confusion for site owners who assume a page showing up in server logs as “crawled” means it’s also searchable — it isn’t, until it clears indexing too.
→ For how crawling itself works, including crawl budget and robots.txt: What Is Crawling?
Reading Your Site’s Index Coverage
Google Search Console’s Page Indexing report shows exactly which of a site’s pages are indexed and, for the ones that aren’t, why. Common statuses worth recognizing:
| Status | What It Means |
|---|---|
| Indexed | The page is in Google’s index and eligible to appear in search results |
| Crawled – currently not indexed | Google read the page but chose not to index it, usually due to thin or duplicate content |
| Discovered – currently not indexed | Google knows the URL exists but hasn’t crawled it yet, often due to low crawl priority |
| Excluded by ‘noindex’ tag | The page explicitly told Google not to index it, and Google complied |
| Duplicate without user-selected canonical | Google found duplicate content and picked a different URL as the canonical version |
| Alternate page with proper canonical tag | Working as intended — this page correctly points to another URL as the canonical version |
| Blocked by robots.txt | A crawling issue, not an indexing one — Google can’t read the page, so it can’t be properly indexed even if it appears in results without a snippet |
Not every status here is a problem. A page correctly excluded because it’s a duplicate or intentionally set to noindex is working as intended — the statuses worth investigating are ones where a page that *should* be indexed isn’t.
noindex and Canonical Tags: Two Different Tools
These two get confused constantly, and they solve different problems. The noindex tag tells search engines a page should not appear in search results at all — it’s a direct instruction, and Google complies with it. The canonical tag doesn’t remove a page from consideration — it tells search engines which version of near-duplicate content should be treated as the primary one, consolidating ranking signals onto that version rather than splitting them across duplicates.
Use noindex when a page genuinely shouldn’t be searchable — internal search results pages, thin filtered views, staging content. Use canonical when multiple URLs legitimately show similar or identical content and you want one specific version to be the one that’s indexed and ranked. Google’s own guidance on combining the two has been described as situational: for near-identical pages where you don’t strongly care which version is chosen, canonical alone is usually enough; for cases where a page must not be indexed at all, noindex is the more reliable signal.
Duplicate Content and Index Bloat
Search engines are built to avoid indexing near-identical content redundantly — when multiple URLs on a site show essentially the same thing, Google typically picks one to index and treats the rest as duplicates, similar to how a directory wouldn’t list the same business under five slightly different addresses. This is a normal, expected behavior, not a penalty.
Index bloat is the related but distinct problem of too many low-value, thin, or near-duplicate pages accumulating in a site’s index over time — filtered product views, tag archive pages, or auto-generated variants that technically differ but add little unique value. A bloated index doesn’t just waste crawl and index resources; it can dilute a search engine’s overall impression of a site’s quality, since so much of what’s indexed offers little on its own.
Rendering and JavaScript: A Modern Wrinkle
Content that requires JavaScript to display isn’t always indexed the same way as plain HTML content. Google generally has to render a page — actually execute its JavaScript, similar to how a browser would — before it can read content that JavaScript generates. This adds an extra processing step, and content that depends heavily on client-side JavaScript can sometimes be indexed later, or incompletely, compared to content that’s present in the raw HTML from the start.
Common Indexing Mistakes
- Accidentally applying noindex site-wide through a plugin or template setting, silently dropping previously indexed pages
- Treating every “Excluded” status as a problem, when many represent correct, intentional exclusions
- Combining noindex and a canonical tag pointing elsewhere carelessly, sending mixed signals about which version should be indexed
- Letting thin, auto-generated pages accumulate (filtered views, tag pages, near-empty archives) into index bloat
- Assuming heavily JavaScript-dependent content is indexed immediately, rather than accounting for the extra rendering step
Frequently Asked Questions
If a page isn’t indexed, can it still show up in search results?
Generally no — indexing is the prerequisite for appearing in regular search results. The one exception is a page blocked by robots.txt that’s linked to from elsewhere, which can occasionally appear without a snippet, since Google knows the URL exists without having read its content.
How long does indexing take after a page is crawled?
It varies significantly — sometimes minutes, sometimes much longer, depending on the page’s evaluated quality, the site’s overall crawl activity, and how much competing content Google is processing.
Does having more indexed pages always help SEO?
No. A large index full of thin or duplicate pages can hurt more than help — index bloat is a real, documented failure pattern, not just a theoretical concern.
The Bottom Line
Indexing is the quality gate between being found and being searchable — a page can be perfectly crawlable and still never make it into the index if it’s thin, duplicated, or explicitly excluded. Understanding the difference between crawling and indexing, and reading the Page Indexing report correctly, is what separates a genuine indexing problem from a status that’s working exactly as intended.
