Skip to main content

AI Bot Blocked

Priority: Critical

Impact: Negative

What issues it may cause​

When your content is blocked, you miss the opportunity to have your brand and information featured directly in AI-generated responses, which are a growing source of user information.

How do you fix it​

Review the list of blocked pages. If the block is unintentional, update your robots.txt file or remove the relevant meta tags to allow access for specific or all AI user agents.

What is the positive impact​

Blocking AI bots can prevent your content from being included in generative AI answers and summaries, potentially reducing visibility in new search experiences.

How to fetch the data for this report template​

You will need to run a crawl for report template to generate report. When report has been generated and you have crawl id you can fetch data for the report using the following query:

query GetReportStatForCrawl(
    $crawlId: ObjectID!
    $reportTemplateCode: String!
    $after: String
   ) {
      getReportStat(
        input: {crawlId: $crawlId, reportTemplateCode: $reportTemplateCode}
      ) {
        crawlUrls(after: $after, reportType: Basic) {
          nodes {
            pageTitle
            url
            description
            foundAtUrl
            deeprank
            level
            aiBotBlocked
            httpStatusCode
            indexable
            foundInGoogleAnalytics
            foundInGoogleSearchConsole
            foundInBacklinks
            foundInList
            foundInLogSummary
            foundInWebCrawl
            foundInSitemap
          }
          totalCount
          pageInfo {
            endCursor
            hasNextPage
          }
        }
     }
   }

Try in explorer