.Google's John Mueller answered an inquiry about why Google indexes pages that are actually forbidden from crawling through robots.txt and why the it's safe to dismiss the similar Look Console reports concerning those creeps.Crawler Website Traffic To Inquiry Parameter URLs.The individual asking the question documented that bots were creating hyperlinks to non-existent concern parameter URLs (? q= xyz) to pages with noindex meta tags that are additionally blocked in robots.txt. What motivated the question is that Google.com is actually crawling the hyperlinks to those pages, receiving blocked out through robots.txt (without seeing a noindex robots meta tag) after that obtaining turned up in Google Look Console as "Indexed, though blocked through robots.txt.".The individual inquired the complying with concern:." But listed below is actually the major inquiry: why would certainly Google mark pages when they can't even see the content? What's the conveniences because?".Google's John Mueller validated that if they can't crawl the page they can not view the noindex meta tag. He also produces an intriguing acknowledgment of the website: hunt driver, advising to overlook the outcomes since the "ordinary" consumers will not observe those end results.He wrote:." Yes, you're appropriate: if our company can not crawl the webpage, our company can not view the noindex. That claimed, if our team can not crawl the pages, then there's not a great deal for our company to index. Therefore while you may find some of those webpages with a targeted website:- inquiry, the ordinary consumer won't observe all of them, so I would not fuss over it. Noindex is likewise great (without robots.txt disallow), it only indicates the URLs will definitely find yourself being crawled (and also wind up in the Search Console document for crawled/not listed-- neither of these statuses cause problems to the remainder of the website). The fundamental part is that you don't create all of them crawlable + indexable.".Takeaways:.1. Mueller's answer confirms the limitations in using the Web site: hunt advanced search operator for diagnostic explanations. Among those factors is due to the fact that it's not linked to the frequent hunt index, it is actually a separate thing entirely.Google.com's John Mueller talked about the web site hunt driver in 2021:." The quick solution is actually that a site: query is actually not implied to be total, nor used for diagnostics functions.A site query is a certain sort of hunt that confines the end results to a particular site. It's primarily only words internet site, a digestive tract, and after that the site's domain.This concern restricts the outcomes to a particular site. It's certainly not implied to become a complete selection of all the pages from that website.".2. Noindex tag without using a robots.txt is actually alright for these sort of situations where a robot is actually connecting to non-existent web pages that are actually getting found through Googlebot.3. Links along with the noindex tag will definitely produce a "crawled/not indexed" item in Search Console and also those will not possess an adverse result on the rest of the website.Go through the inquiry as well as respond to on LinkedIn:.Why will Google.com index web pages when they can not also observe the content?Featured Graphic through Shutterstock/Krakenimages. com.