How do I set up a robots.txt for a multistore?
My main domain had a robots.txt. e.g. www.mysite1.com/robots.txt
I have added a multistore. It is causing problems with server overload as the yahoo slurp robot is trying to crawl too much too quickly. According to Yahoo the crawl rate of Yahoo Slurp bot can be controlled by robots.txt.
How do I set up different robots.txt for my mulltistore e.g. www.mysite2.com/robots.txt
SEO Pack Pro
All Products From Subcategory in Parent Category
All Products From Subcategories in Parent Category Filters
Remove Return Links
Bug fixes applied:
Fix for Tag Filters not working (see OpenCart 1.5.5 Bug Thread)
http://www.opencart60s.com/opencart/rob ... 02314.html
http://www.magikcommerce.com/blog/how-t ... -opencart/
https://weblenti.com/how-to-create-mult ... r-opencart
Please don't send me OC Forum Personal Messages, just contact: firstname.lastname@example.org
OC 126.96.36.199 LIGHT Test Site: http://www.bigmax.ch/shop/
OC 188.8.131.52 V-PRO Test Site: http://www.jacob.ch/shop/
My Github OC Site: https://github.com/IP-CAM
2'600+ FREE OC Extensions on the World's largest Github OC Repository Archive Site.
Users browsing this forum: No registered users and 43 guests