Alejandro Posted October 28, 2014 at 07:21 PM Share Posted October 28, 2014 at 07:21 PM We wanted to share some news with you that we think is important. Yesterday the following article was published in Google Webmaster http://googlewebmastercentral.blogspot.com/2014/10/updating-our-technical-webmaster.html. In this article it says: For optimal rendering and indexing, our new guideline specifies that you should allow Googlebot access to the JavaScript, CSS, and image files that your pages use. This provides you optimal rendering and indexing for your site. Disallowing crawling of Javascript or CSS files in your site’s robots.txt directly harms how well our algorithms render and index your content and can result in suboptimal rankings. Nobody wants suboptimal rankings! So what to do? we've tested the following method for allowing js, css and images in folders without letting anything else pass. You can use Webmaster Tools to test it using both the "Fetch as Google" feature as well as the robots test tool. The key here is to make sure you allow the file extensions in specific folders before you disallow the whole folder. You can adjust and fine-tune to suite your specific requirements. Make sure you don't have duplicates for the Disallow lines if you are copying this into your existing robots.txt file. Allow: /components/*.js Allow: /components/*.css Allow: /components/*.png Allow: /components/*.jpg Allow: /components/*.gif Allow: /components/*.woff Allow: /components/*.svg Allow: /components/*.eot Allow: /components/*.ttf Disallow: /components/ Allow: /templates/*.js Allow: /templates/*.png Allow: /templates/*.jpg Allow: /templates/*.gif Allow: /templates/*.css Allow: /templates/*.woff Allow: /templates/*.svg Allow: /templates/*.eot Allow: /templates/*.ttf Disallow: /templates/ Allow: /media/*.js Allow: /media/*.css Allow: /media/*.png Allow: /media/*.jpg Allow: /media/*.gif Allow: /media/*.woff Allow: /media/*.svg Allow: /media/*.eot Allow: /media/*.ttf Disallow: /media/ Allow: /images/*.png Allow: /images/*.jpg Allow: /images/*.gif Disallow: /images/ For a more complete robots.txt file with additional suggestions read this post http://www.reviewsforjoomla.com/forum/index.php?topic=27331.0 Link to comment
Recommended Posts