Jump to content

Google says disallowing js and css in robots can harm your rankings


Alejandro

Recommended Posts

We wanted to share some news with you that we think is important. Yesterday the following article was published in Google Webmaster http://googlewebmastercentral.blogspot.com/2014/10/updating-our-technical-webmaster.html. In this article it says:

 

For optimal rendering and indexing, our new guideline specifies that you should allow Googlebot access to the JavaScript, CSS, and image files that your pages use. This provides you optimal rendering and indexing for your site. Disallowing crawling of Javascript or CSS files in your site’s robots.txt directly harms how well our algorithms render and index your content and can result in suboptimal rankings.

 

Nobody wants suboptimal rankings! So what to do? we've tested the following method for allowing js, css and images in folders without letting anything else pass. You can use Webmaster Tools to test it using both the "Fetch as Google" feature as well as the robots test tool.

 

The key here is to make sure you allow the file extensions in specific folders before you disallow the whole folder. You can adjust and fine-tune to suite your specific requirements. Make sure you don't have duplicates for the Disallow lines if you are copying this into your existing robots.txt file.

 

Allow: /components/*.js
Allow: /components/*.css
Allow: /components/*.png
Allow: /components/*.jpg
Allow: /components/*.gif
Allow: /components/*.woff
Allow: /components/*.svg
Allow: /components/*.eot
Allow: /components/*.ttf
Disallow: /components/
Allow: /templates/*.js
Allow: /templates/*.png
Allow: /templates/*.jpg
Allow: /templates/*.gif
Allow: /templates/*.css
Allow: /templates/*.woff
Allow: /templates/*.svg
Allow: /templates/*.eot
Allow: /templates/*.ttf
Disallow: /templates/
Allow: /media/*.js
Allow: /media/*.css
Allow: /media/*.png
Allow: /media/*.jpg
Allow: /media/*.gif
Allow: /media/*.woff
Allow: /media/*.svg
Allow: /media/*.eot
Allow: /media/*.ttf
Disallow: /media/
Allow: /images/*.png
Allow: /images/*.jpg
Allow: /images/*.gif
Disallow: /images/

 

For a more complete robots.txt file with additional suggestions read this post http://www.reviewsforjoomla.com/forum/index.php?topic=27331.0

Link to comment
×
×
  • Create New...