[SOLVED] robots.txt (after upgrade from old system)

Hi guys just upgraded my site from 1.5.6.4 to 2.3.02 and just copied my old robots.txt file anyone any comments, think it's a bit long winded
User-agent: *

# Directories
Disallow: /admin/
Disallow: /download/
Disallow: /image/
Disallow: /security/
Disallow: /system/

# Files
Disallow: /php.ini
Disallow: /config.php
Disallow: /user.php
Disallow: /user_permission.php
Disallow: /address.php
Disallow: /account.php
Disallow: /cart.php
Disallow: /checkout.php
Disallow: /confirm.php
Disallow: /history.php
Disallow: /manual.php
Disallow: /payment_address.php
Disallow: /shipping_address.php
Disallow: /order.php
Disallow: /transaction.php
Disallow: /wishlist.php
Disallow: /reward.php
Disallow: /voucher.php
Disallow: /success.php
Disallow: /pagination.php
Disallow: /password.php
Disallow: /search.php
Disallow: /upload.php
Disallow: /error_log.php
Disallow: /mail_log.php
Disallow: /quote_log.php
Disallow: /edit.php
Disallow: /not_found.php
Disallow: /permission.php
Disallow: /maintenance.php

# Sitemap

# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.

# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/robotstxt.html

Here you go

uksitebuilder wrote:Here you go

Code: Select all

User-agent: *
Disallow: /*&limit
Disallow: /*&sort
Disallow: /*&page
Disallow: /*?limit
Disallow: /*?sort
Disallow: /*?order
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Disallow: /*?route=affiliate/

Sitemap: http://www.yourdomain.com/sitemap.xml

Well that looks tidier, Always wondered why it was so long it was created by a friend years ago. Thanks for that

Well, maybe part of it was ok for a previous site, but 90% of the pages listed in that do not exist in a default OpenCart installation

Always, not good to put in sensitive folders such as admin, etc as it gives script kiddies something to play with.

Will the codes @uksitebuilder provided work for url friendly websites? Because I asked ChatGPT and it said no it will not.

parkookk wrote: ↑

Fri May 17, 2024 9:36 pm

Will the codes @uksitebuilder provided work for url friendly websites? Because I asked ChatGPT and it said no it will not.

I wouldn't always trust ChatGPT.