Robots.txt is actually a file that exists on the foundation directory of each Web page and can be used to instruct engines like google on which directories/documents of the web site they are able to crawl and incorporate in their index. Amazon BSR scores go up and down according to https://technik.forum