Part of troubleshooting your site is determining whether a page or URL is available to Google. Available means that Google's crawler (Googlebot) can reach your page. Various reasons why a page might not be available is that it might require a password, it might be blocked to Google via a robots.txt file on your site (which prevents crawling), or it might have a noindex tag or directive, which doesn't prevent access, but tells Google not to index the page.
You can use the following techniques to test the availability of a web page or an image to Google.
If the URL is within a Search Console property that you own
- Open the URL Inspection tool.
- Enter the URL of the page or image to test.
- To see whether Google could access the page the last time it was crawled, expand the "Coverage" section and examine the results as described below.
- To see whether Google can access the resource now, click Test live URL, expand the "Coverage" section, and examine the results as described below.
- Examine these values in the test results:
- Crawl allowed? - Should be "Yes". If not, the page is not available to Google (probably because it is blocked by a robots.txt rule).
- Page fetch: Should be "Successful". If not, the page is not available to Google, and the result will explain why.
- Indexing allowed? - Should be "Yes". If not, you have a noindex tag or directive that will prevent that resource from appearing in Google results.
If the URL is not within a Search Console property that you own
For example, it is an image on your page that is hosted on another domain
- Open the Rich Results test
- Enter the URL of the page or image to test and click Test URL.
- In the results, expand the "Crawl" section.
- You should see the following results:
- Crawl allowed? - Should be "Yes". If not, the page is not available to Google (probably because it is blocked by a robots.txt rule).
- Page fetch: Should be "Successful". If not, the page is not available to Google, and the result will explain why.
- Indexing allowed? - Should be "Yes". If not, you have a noindex tag or directive that will prevent that resource from appearing in Google results.