Hi All,
I want to add robots.txt for opencart installation. Below is the content .
User-Agent: *
Disallow: /admin/
Disallow: /cache/
Disallow: /library/
Disallow: /system/
Disallow: /catalog/
Disallow: /download/
Disallow: /image/
Disallow: /cgi-bin/
Disallow: /vqmod
Allow: /
Is this fine ? do i need to following paths ?
Disallow: /*&limit
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Disallow: /*?route=affiliate/
Please help me on this.
It's important to remember that robots.txt doesn't provide security for those directories (the exact opposite in fact). It just suggests to search engines areas that are inefficient to spend time indexing. I would recommend using:
Code: Select all
User-agent: *
Disallow: /*&filter
Disallow: /*&limit
Disallow: /*&sort
Disallow: /*?route=account/
Disallow: /*?route=affiliate/
Disallow: /*?route=checkout/
Disallow: /*?route=product/search-Ryan
Other than what RPH suggested, you also need to know that you should never use robots.txt to totally steer robots away from your website. You should always use a <meta tag="robots" content="noindex,nofollow"> in the header of your site. Otherwise you can run the risk in Google not re-indexing you for months.satish11nov wrote:Hi All,
I want to add robots.txt for opencart installation. Below is the content .
User-Agent: *
Disallow: /admin/
Disallow: /cache/
Disallow: /library/
Disallow: /system/
Disallow: /catalog/
Disallow: /download/
Disallow: /image/
Disallow: /cgi-bin/
Disallow: /vqmod
Allow: /
Is this fine ? do i need to following paths ?
Disallow: /*&limit
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Disallow: /*?route=affiliate/
Please help me on this.
[/quote]
Other than what RPH suggested, you also need to know that you should never use robots.txt to totally steer robots away from your website. You should always use a <meta tag="robots" content="noindex,nofollow"> in the header of your site. Otherwise you can run the risk in Google not re-indexing you for months.[/quote]
i was under the impression that it should be index, follow as noindex,nofollow would tell google etc to stay away
Other than what RPH suggested, you also need to know that you should never use robots.txt to totally steer robots away from your website. You should always use a <meta tag="robots" content="noindex,nofollow"> in the header of your site. Otherwise you can run the risk in Google not re-indexing you for months.[/quote]
i was under the impression that it should be index, follow as noindex,nofollow would tell google etc to stay away
Koeltechnische deurrubbers eenvoudig online op maat bestellen.
Alle niet stekplichtige onderdelen zoals scharnieren, sloten, randverwarming en verlichting voor alle typen koelingen en vriezers.
https://koelcel-onderdelen.com
Please re-read what I wrote. And no, sometimes you need to tell robots to stay away if you are not ready to launch on the WWW. When developing websites sometimes it's necessary to launch the domain before it's actually time to want search engines to crawl. This is what the meta tag is used for.
Who is online
Users browsing this forum: No registered users and 93 guests
