Post by rph » Wed Aug 31, 2011 2:57 pm

robots.txt

Code: Select all

User-agent: *
Disallow: /*&limit
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Disallow: /*?route=affiliate/
Allow: /
Regular sitemap is automatically generated. Look at the footer of the page. Google Sitemap is enabled in Admin at Extensions->Product Feeds.

-Ryan


rph
Expert Member

Posts

Joined
Fri Jan 08, 2010 5:05 am
Location - Lincoln, Nebraska

Post by uksitebuilder » Wed Aug 31, 2011 3:09 pm

You can add a sitemap link to the bottom of the file like so:

Code: Select all

Sitemap: http://www.yourdomain.com/index.php?route=feed/google_sitemap

ImageImageImageImageImage

For Friendly Professional Support - Click Here


User avatar
Guru Member

Posts

Joined
Thu Jun 09, 2011 11:37 pm
Location - United Kindgom

Post by uksitebuilder » Thu Sep 01, 2011 2:32 am

The following should work.

There is no need to put in an 'Allow' statement

Code: Select all

user-agent: *
Disallow: /*&limit
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Disallow: /*?route=affiliate/

Sitemap: http://sentraherbal.net/index.php?route=feed/google_sitemap

ImageImageImageImageImage

For Friendly Professional Support - Click Here


User avatar
Guru Member

Posts

Joined
Thu Jun 09, 2011 11:37 pm
Location - United Kindgom

Post by rph » Thu Sep 01, 2011 4:01 am

uksitebuilder wrote:There is no need to put in an 'Allow' statement
For what it's worth that's how Google auto-generates them in webmaster tools.

-Ryan


rph
Expert Member

Posts

Joined
Fri Jan 08, 2010 5:05 am
Location - Lincoln, Nebraska

Post by uksitebuilder » Thu Sep 01, 2011 2:23 pm

All files by default are allowed access by robots, so there is no reason to to give them permission per se

ImageImageImageImageImage

For Friendly Professional Support - Click Here


User avatar
Guru Member

Posts

Joined
Thu Jun 09, 2011 11:37 pm
Location - United Kindgom

Post by uksitebuilder » Sat Sep 03, 2011 4:34 pm

Could you share your working version for all who come across this

ImageImageImageImageImage

For Friendly Professional Support - Click Here


User avatar
Guru Member

Posts

Joined
Thu Jun 09, 2011 11:37 pm
Location - United Kindgom

Post by silver0 » Sun Sep 11, 2011 6:02 pm

newbie007 wrote:Absolutely solved!!!

Analysis of ROBOTS.TXT file
No errors have been found.
1 Sitemap(s) found
Value: 100 %

Analysis Sitemap Files
Analysis Result: 1 Sitemap(s) found
Value: 100 %

Analysis Sitemap Validation
Analysis Result: No errors in sitemap(s) found.
Value: 100 %

Analysis Meta tag "robots"
Value: 100 %

Thank you rph
Thank you uksitebuilder
Thank you steve

You all GREAT!!
Hi,

How do you check this?
Which tool do you use?

Newbie

Posts

Joined
Wed Aug 17, 2011 2:31 am

Post by nyomanmusdina » Thu Sep 29, 2011 9:53 am

Please advise how to use that robot.txt file?

Thanks

Nyoman BGC
Your Balinese Fellow
http://www.baligoodjewelry.com


User avatar
New member

Posts

Joined
Fri Sep 02, 2011 9:06 am
Location - Bali, Indonesia

Post by labeshops » Wed Nov 09, 2011 10:47 pm

Is there a way to use sitemaps for multistores? Should I put each domain in as a sitemap link at the bottom?

Running Opencart v2.2 with multi-stores from https://www.labeshops.com which has links to all my stores.

Image


User avatar
Expert Member

Posts

Joined
Thu Aug 04, 2011 4:41 am
Location - Florida, USA

Post by uksitebuilder » Wed Nov 09, 2011 11:02 pm

edit .htaccess file

at the top put

AddType x-mapp-php .txt
AddHandler x-mapp-php .txt

Now you can put php in a text file and will be parsed by the PHP parser

edit your sitemap line in your robots.txt

something like

Sitemap: http://<?php echo $_SERVER['HTTP_HOST']; ?>/index.php?route=feed/google_sitemap

ImageImageImageImageImage

For Friendly Professional Support - Click Here


User avatar
Guru Member

Posts

Joined
Thu Jun 09, 2011 11:37 pm
Location - United Kindgom

Post by deano65 » Wed Nov 09, 2011 11:06 pm

Guys, will Google Webmaster automatically find my sitemap once its enabled??

I have added my website to my webmaster tools and enabled my sitemap but when I paste in the sitemap url its invalid.
Last edited by deano65 on Wed Nov 09, 2011 11:07 pm, edited 1 time in total.

New member

Posts

Joined
Thu Oct 20, 2011 4:18 pm

Post by uksitebuilder » Wed Nov 09, 2011 11:07 pm

To be safe, you should submit it to your webmaster tools account

ImageImageImageImageImage

For Friendly Professional Support - Click Here


User avatar
Guru Member

Posts

Joined
Thu Jun 09, 2011 11:37 pm
Location - United Kindgom

Post by deano65 » Wed Nov 09, 2011 11:57 pm

uksitebuilder wrote:To be safe, you should submit it to your webmaster tools account
I tried this, but Google said that its not a valid feed.

http://www.my-website.co.uk/index.php?r ... le_sitemap

New member

Posts

Joined
Thu Oct 20, 2011 4:18 pm

Post by uksitebuilder » Thu Nov 10, 2011 12:01 am

what is the full link to your sitemap with correct domain ?

ImageImageImageImageImage

For Friendly Professional Support - Click Here


User avatar
Guru Member

Posts

Joined
Thu Jun 09, 2011 11:37 pm
Location - United Kindgom

Post by deano65 » Thu Nov 10, 2011 12:11 am

uksitebuilder wrote:what is the full link to your sitemap with correct domain ?
yes that's the full link the the sitemap. I slightly changed my url as to not show everyone my website ;)

New member

Posts

Joined
Thu Oct 20, 2011 4:18 pm

Post by uksitebuilder » Thu Nov 10, 2011 12:16 am

then i have no way of knowing what errors you are getting in your sitemap ;)

ImageImageImageImageImage

For Friendly Professional Support - Click Here


User avatar
Guru Member

Posts

Joined
Thu Jun 09, 2011 11:37 pm
Location - United Kindgom

Post by deano65 » Thu Nov 10, 2011 12:21 am

im unable to send you a PM

New member

Posts

Joined
Thu Oct 20, 2011 4:18 pm

Post by deano65 » Mon Nov 14, 2011 8:10 pm

im still not having any luck with this using v1.5.1.3

New member

Posts

Joined
Thu Oct 20, 2011 4:18 pm

Post by labeshops » Mon Nov 14, 2011 8:51 pm

What about sitemaps for multistores? Could you put in multiple sitemap statements in robots.txt and name them storesitemap.xml and have it work?

I have uksb's mod for multistore sitemaps and have set the paths to them in google webmaster tools but when I scan my site with seo tools, they do not find the sitemaps. I know I can save them with the tool linked above, just not sure how to tell seo tools they are there for each site.

Running Opencart v2.2 with multi-stores from https://www.labeshops.com which has links to all my stores.

Image


User avatar
Expert Member

Posts

Joined
Thu Aug 04, 2011 4:41 am
Location - Florida, USA

Post by jimaras » Sun Jan 15, 2012 8:12 pm

uksitebuilder wrote:The following should work.

There is no need to put in an 'Allow' statement

Code: Select all

user-agent: *
Disallow: /*&limit
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Disallow: /*?route=affiliate/

Sitemap: http://sentraherbal.net/index.php?route=feed/google_sitemap
Is any option to disallow other languages except my default?

http://e-kreopoleio.com/


Active Member

Posts

Joined
Thu Sep 15, 2011 1:45 am
Who is online

Users browsing this forum: No registered users and 47 guests