Disallowed JS (Uncrawled)
JavaScript files which are disallowed in robots.txt
Priority: Critical
Impact: Negative
What issues it may causeโ
If the resources are disallowed in the robots.txt, the search engines may be unable to render the pages correctly.
How do you fix itโ
The resources should be reviewed using tools such as the Search Console Live URL Test or Mobile-Friendly Test to determine if the resources are required. If so, the robots.txt should be updated to allow the URLs to be fetched.
What is the positive impactโ
Search engines will be able to render the pages correctly and process all the content and meta data so the pages can be indexed as expected.
How to fetch the data for this report templateโ
You will need to run a crawl for report template to generate report. When report has been generated and you have crawl id you can fetch data for the report using the following query:
- Query
- Variables
- cURL
query GetReportForCrawl($crawlId: ObjectID!, $reportTemplateCode: String!) {
getCrawl(id: $crawlId) {
reportsByCode(
input: {
reportTypeCodes: Basic
reportTemplateCodes: [$reportTemplateCode]
}
) {
rows {
nodes {
... on CrawlUncrawledUrls {
url
foundAtUrl
foundAtSitemap
level
restrictedReason
robotsTxtRuleMatch
foundInWebCrawl
foundInGoogleAnalytics
foundInGoogleSearchConsole
foundInBacklinks
foundInList
foundInLogSummary
foundInSitemap
}
}
}
}
}
}
{"crawlId":"TjAwNUNyYXdsNDAwMA","reportTemplateCode":"disallowed_js_uncrawled"}
curl -X POST -H "Content-Type: application/json" -H "apollographql-client-name: docs-example-client" -H "apollographql-client-version: 1.0.0" -H "x-auth-token: YOUR_API_SESSION_TOKEN" --data '{"query":"query GetReportForCrawl($crawlId: ObjectID!, $reportTemplateCode: String!) { getCrawl(id: $crawlId) { reportsByCode( input: { reportTypeCodes: Basic reportTemplateCodes: [$reportTemplateCode] } ) { rows { nodes { ... on CrawlUncrawledUrls { url foundAtUrl foundAtSitemap level restrictedReason robotsTxtRuleMatch foundInWebCrawl foundInGoogleAnalytics foundInGoogleSearchConsole foundInBacklinks foundInList foundInLogSummary foundInSitemap } } } } } }","variables":{"crawlId":"TjAwNUNyYXdsNDAwMA","reportTemplateCode":"disallowed_js_uncrawled"}}' https://api.lumar.io/graphql