Skip to main content

Non-Indexable URLs in Sitemaps

Non-indexable pages which were found in the sitemaps included in the crawl.

Priority: Low

Impact: Negative

What issues it may cause​

Linking to non-indexable pages in sitemaps encourages search engines to crawl and process these URLs which is an unnecessary usage of crawl budget.

How do you fix it​

Review all of the non-indexable pages that are included in Sitemaps and either remove them from Sitemaps or make them indexable if appropriate.

What is the positive impact​

Having clean Sitemaps with all valid, indexable and unique pages help search engines to crawl, index and update all of the important pages of your website more efficiently.

Crawl budget spent crawling the non-indexable pages may be reduced, allowing crawl budget to be used on more important pages, or save on server costs.

How to fetch the data for this report template​

You will need to run a crawl for report template to generate report. When report has been generated and you have crawl id you can fetch data for the report using the following query:

query GetReportStatForCrawl(
    $crawlId: ObjectID!
    $reportTemplateCode: String!
    $after: String
   ) {
      getReportStat(
        input: {crawlId: $crawlId, reportTemplateCode: $reportTemplateCode}
      ) {
        crawlUrls(after: $after, reportType: Basic) {
          nodes {
            pageTitle
            url
            foundAtUrl
            foundAtSitemap
            deeprank
            level
            sitemapsInCount
            indexable
            httpStatusCode
            noindex
            canonicalizedPage
            nofollowedPage
            disallowedPage
            unavailableAfter
            foundInGoogleAnalytics
            foundInGoogleSearchConsole
            foundInBacklinks
            foundInList
            foundInLogSummary
            foundInWebCrawl
            foundInSitemap
          }
          totalCount
          pageInfo {
            endCursor
            hasNextPage
          }
        }
     }
   }

Try in explorer