Page 1 of 1

Recommended and best robots.txt file for opencart 3.x

Posted: Tue Sep 25, 2018 1:09 pm
by kg0925
please anyone help me with robots.txt file. i don't know much about it. before i was not using robot file in my website root. but after adding it my website traffic rank dropped. may be some useful links blocked by robot file.
so please suggest well tested and seo friendly best robot file for my store. thanks.

Re: Recommended and best robots.txt file for opencart 3.x

Posted: Tue Sep 25, 2018 1:25 pm
by khnaz35
kg0925 wrote:
Tue Sep 25, 2018 1:09 pm
please anyone help me with robots.txt file. i don't know much about it. before i was not using robot file in my website root. but after adding it my website traffic rank dropped. may be some useful links blocked by robot file.
so please suggest well tested and seo friendly best robot file for my store. thanks.
Share your robots.txt here

Re: Recommended and best robots.txt file for opencart 3.x

Posted: Tue Sep 25, 2018 2:35 pm
by kg0925
khnaz35 wrote:
Tue Sep 25, 2018 1:25 pm
kg0925 wrote:
Tue Sep 25, 2018 1:09 pm
please anyone help me with robots.txt file. i don't know much about it. before i was not using robot file in my website root. but after adding it my website traffic rank dropped. may be some useful links blocked by robot file.
so please suggest well tested and seo friendly best robot file for my store. thanks.
Share your robots.txt here
User-agent: *
Disallow: /*&limit
Disallow: /*?limit
Disallow: /*?sort
Disallow: /*&sort
Disallow: /*?order
Disallow: /*&order
Disallow: /*?price
Disallow: /*&price
Disallow: /*?brand_tabletpc
Disallow: /*&brand_tabletpc
Disallow: /*?color_default
Disallow: /*&color_default
Disallow: /*?filter_tag
Disallow: /*&filter_tag
Disallow: /*?mode
Disallow: /*&mode
Disallow: /*?cat
Disallow: /*&cat
Disallow: /*?dir
Disallow: /*&dir
Disallow: /*?color
Disallow: /*&color
Disallow: /*?product_id
Disallow: /*&product_id
Disallow: /*?minprice
Disallow: /*&minprice
Disallow: /*?maxprice
Disallow: /*&maxprice
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Disallow: /*?page=1
Disallow: /*&create=1
Disallow: /?route=information/contact
Disallow: /*?route=affiliate/
Disallow: /*?keyword
Disallow: /*?av
Disallow: /admin/
Disallow: /system/
Disallow: /catalog/

Re: Recommended and best robots.txt file for opencart 3.x

Posted: Tue Sep 25, 2018 10:18 pm
by paulfeakins
Why are you disallowing indexing of almost all your site lol?

Re: Recommended and best robots.txt file for opencart 3.x

Posted: Wed Sep 26, 2018 6:05 am
by storiqax
Interest for this one too. What would you recommend regarding robots.txt for better SEO?

Re: Recommended and best robots.txt file for opencart 3.x

Posted: Wed Sep 26, 2018 12:53 pm
by kg0925
paulfeakins wrote:
Tue Sep 25, 2018 10:18 pm
Why are you disallowing indexing of almost all your site lol?
to avoid duplication. can you suggest me better robots file?? as i said i don't know much about it i found this robots file to many blogs as well it suggested by https://marketplace.webkul.com/optimiza ... rketplace/

please help me with it if you know better robots file.

Re: Recommended and best robots.txt file for opencart 3.x

Posted: Wed Sep 26, 2018 2:38 pm
by uksitebuilder
Here's a basic robots.txt file for OpenCart

Code: Select all

User-agent: *
Disallow: /*&limit
Disallow: /*?sort
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search

Sitemap: https://www.yoursitemapurlhere

Re: Recommended and best robots.txt file for opencart 3.x

Posted: Wed Sep 26, 2018 6:38 pm
by kg0925
uksitebuilder wrote:
Wed Sep 26, 2018 2:38 pm
Here's a basic robots.txt file for OpenCart

Code: Select all

User-agent: *
Disallow: /*&limit
Disallow: /*?sort
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search

Sitemap: https://www.yoursitemapurlhere
is it good for seo??

Re: Recommended and best robots.txt file for opencart 3.x

Posted: Wed Sep 26, 2018 8:08 pm
by khnaz35
kg0925 wrote:
Wed Sep 26, 2018 6:38 pm
uksitebuilder wrote:
Wed Sep 26, 2018 2:38 pm
Here's a basic robots.txt file for OpenCart

Code: Select all

User-agent: *
Disallow: /*&limit
Disallow: /*?sort
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search

Sitemap: https://www.yoursitemapurlhere
is it good for seo??
Yes it is , just dont forget to mention your site map

Re: Recommended and best robots.txt file for opencart 3.x

Posted: Sat Sep 07, 2019 4:53 pm
by newbie87
Regarding the robots/txt file,
/Disallow: /admin
Disallow: /catalog
Disallow: /download
Disallow: /mods
Disallow: /system
Disallow: /system_
Disallow: /vendor
Disallow: /vqmod
why does any search engine need to access these folders in the backend?

Re: Recommended and best robots.txt file for opencart 3.x

Posted: Sun Sep 08, 2019 3:46 pm
by uksitebuilder
robots.txt should list only links that are accessible via a robot that follows links in your site.

Therefore no admin folder or other OC folders should be listed really. Plus listing them gives script kiddies something to attempt to play with.

robots.txt is not always adhered to (depending on the robot that is crawling) Also, you should be careful not to accidentally block robots from valid pages that you do want indexed.

Re: Recommended and best robots.txt file for opencart 3.x

Posted: Sun Sep 08, 2019 4:08 pm
by letxobnav
why does any search engine need to access these folders in the backend?
robots.txt is about public url's not your folder structure.

Instead of, or in addition to, robots.txt entries, you can add robots meta tags and X-Robots-Tag headers which are more flexible and dynamic.

Re: Recommended and best robots.txt file for opencart 3.x

Posted: Mon Jun 22, 2020 4:27 pm
by stavrosa
This is my setup

Code: Select all

User-agent: *
Disallow: /*&limit
Disallow: /*?sort
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search

User-agent: SemrushBot
Disallow: /

User-agent: DotBot
Disallow: /

User-agent: MauiBot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: MJ12Bot
Disallow: /

Sitemap: https://YOUR-DOMAIN/index.php?route=information/sitemap