Google: Disallowed URLs Through Robots.txt Does Not Affect Crawl Budget

  •   June 21, 2019
Want create site? Find Free WordPress Themes and plugins.

Gary Illyes from Google added to the big crawl budget article he published on the webmaster blog in January 2017 a new question and answer. It basically says that if you disallow a URL in your robots.txt file, then those URLs do not affect your crawl budget.

He added this question and answer:

Q: Do URLs I disallowed through robots.txt affect my crawl budget in any way?
A: No, disallowed URLs do not affect the crawl budget.

He posted last night that he added the Q&A for a former Googler, Pierre Far:

Forum discussion at Twitter.

This marketing news is not the copyright of Scott.Services – please click here to see the original source of this article. Author: (Barry Schwartz)

For more SEO, PPC, internet marketing news please check out

Why not check out our SEO, PPC marketing services at

We’re also on:

Did you find apk for android? You can find new Free Android Games and apps.