Hello!
I know it is good to have Robots.txt file.
So what should be the default?
This is what i done:
User-agent: *
Disallow: /adminFolder/
Disallow: /catalog/
Disallow: /download/
Disallow: /image/
Disallow: /system/
Is this right?
To be honest, I see no real advantage to using a robots.txt file other than to stop google indexing pages you don't want it to. If you don't have a link to your admin pages anywhere on your site (which you shouldn't) then google will never spider there as it doesn't guess what links you have, it reads your site, collates links and goes to where they are, recursively doing the same on those pages
The same is true for the image folders and the likes
The same is true for the image folders and the likes
Yes, it is for All search engines...and not just the Google ones... Also, some of these bots are not so "clever" and they browse something, which is not need to..JAY6390 wrote:To be honest, I see no real advantage to using a robots.txt file other than to stop google indexing pages you don't want it to. If you don't have a link to your admin pages anywhere on your site (which you shouldn't) then google will never spider there as it doesn't guess what links you have, it reads your site, collates links and goes to where they are, recursively doing the same on those pages
The same is true for the image folders and the likes
That's why the smart people discovered the robots.txt file, there is it in the web masters tools into Google. And google will bot up all which it can, it is natural. So, you are not right at your note. There is tons of information about that...
So, do you know the proper "default" robots.txt for opencart?
My robots file is:
not a bad idea on the other folders also, i think i'll add them to mine
Code: Select all
User-agent: *
Disallow: /admin/
Disallow: /index.php?route=common/home
Disallow: /index.php?route=product/search
Disallow: /index.php?route=product/category
Disallow: /index.php?route=information/informationwhy would you not want google to spider your information pages and category pages?
OpenCart commercial mods and development http://spotonsolutions.net
Layered Navigation
Shipment Tracking
Vehicle Year/Make/Model Filter
I've got this set up for the new products that I haven't seo'd yet.
I sometimes set up a product in a rush, and don't always give it the full care and attention I want, so in order to not have the rushed or temporary product indexed by google I block the category page.
I sometimes set up a product in a rush, and don't always give it the full care and attention I want, so in order to not have the rushed or temporary product indexed by google I block the category page.
The thing is, unless you link to yoursite.com/admin/ then google or any other bot won't try and spider it, so I just find it pretty pointless. If you've got pages that are visible to the public and linked then that's fair enough, but otherwise it's pretty redundant. ALSO, giving a robots.txt file your admin if anything is worse, since anyone with half a brain can read your robots.txt file, find your admin folder name and start hammering away at trying to break into your admin area
There does seem to be one help in having a robots.txt file, which is to direct bots to your xml sitemap. I'm wondering what effect it has to not have a robots.txt file, since they all do seem to look for it.
Using OC version 1.4.8b
http://catandcaboodle.com/
Hi Guys, I get the idea of what the txt should say but how do I find the this in the root file. For example break it down like cataloge-model-ect ect/ I am a complete newbie and need exact instructions. Also, once I have changed the files to disallow how do I upload that. Again please remember i am new. Thank you guys in advance.
trankhanh89 wrote:in SEO's perspective, robots.txt is really good choice to improve your site as it helps SE to have a direction when crawling your site.
By defalt, Opencart is SEO friendly already. However, adding robots.txt will make your site better than your competitor's.
This one (below) in my SEO's experience is quite a good one to consider (to enable sitemap to google, log in to your admin and go to extension-->productfeed, then enable Google sitemap) :
user-agent: *
Disallow: /*&limit
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Disallow: /*?route=affiliate/
Sitemap: http://yoursite.com/index.php?route=feed/google_sitemap
Do you mean when i go to extension-->productfeed and enable google sitemap i shall write (in the text area field) all of the below:
user-agent: *
Disallow: /*&limit
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Disallow: /*?route=affiliate/
Sitemap: http://mysite.com/index.php?route=feed/google_sitemap
Is this correct?
create a file in the folder where you installed opencart
call the file robots.txt
open the this robot.txt file in edit
past your code, click save and exit out of it.
done.
call the file robots.txt
open the this robot.txt file in edit
past your code, click save and exit out of it.
done.
Who is online
Users browsing this forum: No registered users and 316 guests
