please anyone help me with robots.txt file. i don't know much about it. before i was not using robot file in my website root. but after adding it my website traffic rank dropped. may be some useful links blocked by robot file.
so please suggest well tested and seo friendly best robot file for my store. thanks.
Share your robots.txt herekg0925 wrote: ↑Tue Sep 25, 2018 1:09 pmplease anyone help me with robots.txt file. i don't know much about it. before i was not using robot file in my website root. but after adding it my website traffic rank dropped. may be some useful links blocked by robot file.
so please suggest well tested and seo friendly best robot file for my store. thanks.
Urgent Questions shoot here: khnaz35@gmail.com
Enjoy nature 
User-agent: *khnaz35 wrote: ↑Tue Sep 25, 2018 1:25 pmShare your robots.txt herekg0925 wrote: ↑Tue Sep 25, 2018 1:09 pmplease anyone help me with robots.txt file. i don't know much about it. before i was not using robot file in my website root. but after adding it my website traffic rank dropped. may be some useful links blocked by robot file.
so please suggest well tested and seo friendly best robot file for my store. thanks.
Disallow: /*&limit
Disallow: /*?limit
Disallow: /*?sort
Disallow: /*&sort
Disallow: /*?order
Disallow: /*&order
Disallow: /*?price
Disallow: /*&price
Disallow: /*?brand_tabletpc
Disallow: /*&brand_tabletpc
Disallow: /*?color_default
Disallow: /*&color_default
Disallow: /*?filter_tag
Disallow: /*&filter_tag
Disallow: /*?mode
Disallow: /*&mode
Disallow: /*?cat
Disallow: /*&cat
Disallow: /*?dir
Disallow: /*&dir
Disallow: /*?color
Disallow: /*&color
Disallow: /*?product_id
Disallow: /*&product_id
Disallow: /*?minprice
Disallow: /*&minprice
Disallow: /*?maxprice
Disallow: /*&maxprice
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Disallow: /*?page=1
Disallow: /*&create=1
Disallow: /?route=information/contact
Disallow: /*?route=affiliate/
Disallow: /*?keyword
Disallow: /*?av
Disallow: /admin/
Disallow: /system/
Disallow: /catalog/
Why are you disallowing indexing of almost all your site lol?
UK OpenCart Hosting | OpenCart Audits | OpenCart Support - please email info@antropy.co.uk
to avoid duplication. can you suggest me better robots file?? as i said i don't know much about it i found this robots file to many blogs as well it suggested by https://marketplace.webkul.com/optimiza ... rketplace/paulfeakins wrote: ↑Tue Sep 25, 2018 10:18 pmWhy are you disallowing indexing of almost all your site lol?
please help me with it if you know better robots file.
Here's a basic robots.txt file for OpenCart
Code: Select all
User-agent: *
Disallow: /*&limit
Disallow: /*?sort
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Sitemap: https://www.yoursitemapurlhereis it good for seo??uksitebuilder wrote: ↑Wed Sep 26, 2018 2:38 pmHere's a basic robots.txt file for OpenCart
Code: Select all
User-agent: * Disallow: /*&limit Disallow: /*?sort Disallow: /*&sort Disallow: /*?route=checkout/ Disallow: /*?route=account/ Disallow: /*?route=product/search Sitemap: https://www.yoursitemapurlhere
Yes it is , just dont forget to mention your site mapkg0925 wrote: ↑Wed Sep 26, 2018 6:38 pmis it good for seo??uksitebuilder wrote: ↑Wed Sep 26, 2018 2:38 pmHere's a basic robots.txt file for OpenCart
Code: Select all
User-agent: * Disallow: /*&limit Disallow: /*?sort Disallow: /*&sort Disallow: /*?route=checkout/ Disallow: /*?route=account/ Disallow: /*?route=product/search Sitemap: https://www.yoursitemapurlhere
Urgent Questions shoot here: khnaz35@gmail.com
Enjoy nature
Regarding the robots/txt file,
/Disallow: /admin
Disallow: /catalog
Disallow: /download
Disallow: /mods
Disallow: /system
Disallow: /system_
Disallow: /vendor
Disallow: /vqmod
why does any search engine need to access these folders in the backend?
/Disallow: /admin
Disallow: /catalog
Disallow: /download
Disallow: /mods
Disallow: /system
Disallow: /system_
Disallow: /vendor
Disallow: /vqmod
why does any search engine need to access these folders in the backend?
robots.txt should list only links that are accessible via a robot that follows links in your site.
Therefore no admin folder or other OC folders should be listed really. Plus listing them gives script kiddies something to attempt to play with.
robots.txt is not always adhered to (depending on the robot that is crawling) Also, you should be careful not to accidentally block robots from valid pages that you do want indexed.
Therefore no admin folder or other OC folders should be listed really. Plus listing them gives script kiddies something to attempt to play with.
robots.txt is not always adhered to (depending on the robot that is crawling) Also, you should be careful not to accidentally block robots from valid pages that you do want indexed.
robots.txt is about public url's not your folder structure.why does any search engine need to access these folders in the backend?
Instead of, or in addition to, robots.txt entries, you can add robots meta tags and X-Robots-Tag headers which are more flexible and dynamic.
Crystal Light Centrum Taiwan
Extensions: MailQueue | SUKHR | VBoces
“Data security is paramount at [...], and we are committed to protecting the privacy of anyone who is associated with our [...]. We’ve made a lot of improvements and will continue to make them.”
When you know your life savings are gone.
This is my setup
Code: Select all
User-agent: *
Disallow: /*&limit
Disallow: /*?sort
Disallow: /*&sort
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
User-agent: SemrushBot
Disallow: /
User-agent: DotBot
Disallow: /
User-agent: MauiBot
Disallow: /
User-agent: AhrefsBot
Disallow: /
User-agent: MJ12Bot
Disallow: /
Sitemap: https://YOUR-DOMAIN/index.php?route=information/sitemap
Who is online
Users browsing this forum: dparakhiya, Semrush [Bot] and 412 guests
