Why Your Site Isn’t Getting Indexed: A 30-Minute Diagnosis

If a site isn’t getting indexed, the cause is almost always technical: the page is blocked by robots.txt or a noindex meta tag, it’s swallowed by a wrong canonical, it’s been flagged as a duplicate, or Google simply hasn’t reached it yet due to a lack of crawl budget. Below is a 30-minute express diagnosis: go through the checklist of 8 causes and verify each one in Google Search Console.

“Not indexed” or “not ranking”: tell them apart first

Before fixing anything, pin down the symptom. If the page isn’t in the index, Google hasn’t stored it and it won’t show up for any query. If the page is indexed but not in the top results, that’s a matter of relevance, content and links, not indexing. Quick check: search for site:yourdomain.com/page. If nothing comes up, the page isn’t indexed, and you work through the checklist below.

30-minute express diagnosis: 8 causes

1. The site is blocked in robots.txt

This is the first thing to check. A Disallow directive can block crawling of specific sections or the whole site, a common problem after moving from a staging server that had Disallow: /. Open yourdomain.com/robots.txt and make sure the sections you need aren’t disallowed. Note: robots.txt blocks crawling but doesn’t always remove a page from the index; sometimes it’s shown without a description. A real indexing block comes from noindex.

2. A noindex meta tag or X-Robots-Tag header

The tag <meta name=”robots” content=”noindex”> in the page code or the X-Robots-Tag HTTP header tells Google directly not to add the page to the index. On CMS platforms this is often left on after development (in WordPress, it’s the “Discourage search engines from indexing this site” checkbox). Check the page source and the server response.

3. A wrong canonical

A rel=”canonical” tag pointing to a different URL tells Google: “the canonical version is over there, don’t index this one”. If canonicals are set incorrectly (for example, every page points to the homepage), Google ignores their content. Make sure every important page has a self-referencing canonical.

4. Duplicate pages

If content is duplicated (www and non-www, http and https, pages with and without parameters, product cards with identical descriptions), Google picks one version and marks the rest as “Duplicate without user-selected canonical”, and those don’t get indexed. Set up a single primary host with 301 redirects and make the content unique.

5. Thin or low-quality content

Google increasingly delays or declines indexing for pages it considers low-value: empty categories, auto-generated pages, texts a few lines long. In the report this shows up as “Crawled, currently not indexed” or “Discovered, currently not indexed”. The fix: strengthen the content, and merge or noindex weak pages.

6. An orphan page: nothing links to it

Google finds pages through links. If no internal link points to a page and it isn’t in the sitemap, the bot may simply never discover it. Add the page to the sitemap and to your internal linking. External links also speed up discovery and indexing; we explain how in our article on speeding up link indexing.

7. Technical barriers: speed, server errors, JS

A slow server response, 5xx errors, timeouts and content that only loads through heavy JavaScript prevent the bot from fetching and processing the page. Check the response code (it should be 200), the load speed, and whether the main content is visible without running scripts.

8. Not enough crawl budget

Crawl budget is the number of pages Google is willing to crawl on your site over a given period. On large sites with thousands of URLs, lots of duplicates and parameters, the bot spends the budget on junk and never reaches the important pages. Remove duplicates, block useless URLs, speed up the site, and the budget gets redistributed to what matters. Separately, check for manual penalties in the Manual actions section.

Table: cause → how to check → how to fix

Checking in Google Search Console: step by step

Search Console is the free, primary diagnostic tool. If you haven’t connected it yet, start with our article on setting up Google Search Console.

• Step 1. URL Inspection. Paste the page URL into the top bar of GSC. The response “URL is not on Google” plus the reason shows exactly where the blockage is (robots, noindex, canonical, duplicate).
• Step 2. The Page indexing report. Look at the list of reasons pages aren’t indexed and how many URLs fall into each group; that’s how you spot a systemic problem.
• Step 3. Sitemap. Make sure the sitemap has been submitted and processed without errors, and that the URLs you need are in it.
• Step 4. Request indexing. Once you’ve fixed the cause, click “Request indexing”. This speeds up the recrawl. At scale, the issue is solved by fixing the root cause, not by manual requests.

Links dropping out of the index: what it means

A separate scenario: pages or backlinks keep getting indexed and then disappearing. This is usually a symptom of a weak crawl budget, low-quality donors or unstable technical SEO. With external links, placing them isn’t enough; you also need stable indexing, otherwise they don’t pass link equity. You have to work on both the donor sites and on getting them crawled faster.

When to order an audit

If pages still don’t get indexed after the checklist, the problem usually runs deeper: site architecture, duplicates or crawl budget allocation. In those cases it’s faster and cheaper to run a technical SEO audit: it shows all indexing barriers at once and gives you a ready, prioritized fix plan.

Don’t want to dig into it yourself? Send us a request and we’ll diagnose your indexing, find the cause and draw up a plan: order an SEO audit.

FAQ