One of the takeaways from the Google Webmaster Conference was that if Google tries to access your robots.txt file is unreachable but it does exist then Google won't crawl your site. Google said about ...
Some results have been hidden because they may be inaccessible to you
Show inaccessible results
Feedback