[SOLVED] robots.txt (after upgrade from old system)
Posted: Sun Feb 05, 2017 7:42 pm
Hi guys just upgraded my site from 1.5.6.4 to 2.3.02 and just copied my old robots.txt file anyone any comments, think it's a bit long winded
User-agent: *
# Directories
Disallow: /admin/
Disallow: /download/
Disallow: /image/
Disallow: /security/
Disallow: /system/
# Files
Disallow: /php.ini
Disallow: /config.php
Disallow: /user.php
Disallow: /user_permission.php
Disallow: /address.php
Disallow: /account.php
Disallow: /cart.php
Disallow: /checkout.php
Disallow: /confirm.php
Disallow: /history.php
Disallow: /manual.php
Disallow: /payment_address.php
Disallow: /shipping_address.php
Disallow: /order.php
Disallow: /transaction.php
Disallow: /wishlist.php
Disallow: /reward.php
Disallow: /voucher.php
Disallow: /success.php
Disallow: /pagination.php
Disallow: /password.php
Disallow: /search.php
Disallow: /upload.php
Disallow: /error_log.php
Disallow: /mail_log.php
Disallow: /quote_log.php
Disallow: /edit.php
Disallow: /not_found.php
Disallow: /permission.php
Disallow: /maintenance.php
# Sitemap
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/robotstxt.html
User-agent: *
# Directories
Disallow: /admin/
Disallow: /download/
Disallow: /image/
Disallow: /security/
Disallow: /system/
# Files
Disallow: /php.ini
Disallow: /config.php
Disallow: /user.php
Disallow: /user_permission.php
Disallow: /address.php
Disallow: /account.php
Disallow: /cart.php
Disallow: /checkout.php
Disallow: /confirm.php
Disallow: /history.php
Disallow: /manual.php
Disallow: /payment_address.php
Disallow: /shipping_address.php
Disallow: /order.php
Disallow: /transaction.php
Disallow: /wishlist.php
Disallow: /reward.php
Disallow: /voucher.php
Disallow: /success.php
Disallow: /pagination.php
Disallow: /password.php
Disallow: /search.php
Disallow: /upload.php
Disallow: /error_log.php
Disallow: /mail_log.php
Disallow: /quote_log.php
Disallow: /edit.php
Disallow: /not_found.php
Disallow: /permission.php
Disallow: /maintenance.php
# Sitemap
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/robotstxt.html