Post by discoverytdi » Sun Feb 05, 2017 7:42 pm

Hi guys just upgraded my site from 1.5.6.4 to 2.3.02 and just copied my old robots.txt file anyone any comments, think it's a bit long winded
User-agent: *

# Directories
Disallow: /admin/
Disallow: /download/
Disallow: /image/
Disallow: /security/
Disallow: /system/

# Files
Disallow: /php.ini
Disallow: /config.php
Disallow: /user.php
Disallow: /user_permission.php
Disallow: /address.php
Disallow: /account.php
Disallow: /cart.php
Disallow: /checkout.php
Disallow: /confirm.php
Disallow: /history.php
Disallow: /manual.php
Disallow: /payment_address.php
Disallow: /shipping_address.php
Disallow: /order.php
Disallow: /transaction.php
Disallow: /wishlist.php
Disallow: /reward.php
Disallow: /voucher.php
Disallow: /success.php
Disallow: /pagination.php
Disallow: /password.php
Disallow: /search.php
Disallow: /upload.php
Disallow: /error_log.php
Disallow: /mail_log.php
Disallow: /quote_log.php
Disallow: /edit.php
Disallow: /not_found.php
Disallow: /permission.php
Disallow: /maintenance.php

# Sitemap

# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.

# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/robotstxt.html

https://www.rvsolar.co.uk/


User avatar
New member

Posts

Joined
Fri Feb 17, 2012 6:43 am
Location - sunny lancashire UK

Post by uksitebuilder » Sun Feb 05, 2017 7:46 pm

Here you go

Code: Select all

User-agent: *
Disallow: /*&limit
Disallow: /*&sort
Disallow: /*&page
Disallow: /*?limit
Disallow: /*?sort
Disallow: /*?order
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Disallow: /*?route=affiliate/

Sitemap: http://www.yourdomain.com/sitemap.xml

User avatar
Guru Member

Posts

Joined
Thu Jun 09, 2011 11:37 pm
Location - United Kindgom

Post by discoverytdi » Sun Feb 05, 2017 11:35 pm

uksitebuilder wrote:Here you go

Code: Select all

User-agent: *
Disallow: /*&limit
Disallow: /*&sort
Disallow: /*&page
Disallow: /*?limit
Disallow: /*?sort
Disallow: /*?order
Disallow: /*?route=checkout/
Disallow: /*?route=account/
Disallow: /*?route=product/search
Disallow: /*?route=affiliate/

Sitemap: http://www.yourdomain.com/sitemap.xml
Well that looks tidier, Always wondered why it was so long it was created by a friend years ago. Thanks for that :good:

https://www.rvsolar.co.uk/


User avatar
New member

Posts

Joined
Fri Feb 17, 2012 6:43 am
Location - sunny lancashire UK

Post by uksitebuilder » Mon Feb 06, 2017 1:40 am

Well, maybe part of it was ok for a previous site, but 90% of the pages listed in that do not exist in a default OpenCart installation

Always, not good to put in sensitive folders such as admin, etc as it gives script kiddies something to play with.

User avatar
Guru Member

Posts

Joined
Thu Jun 09, 2011 11:37 pm
Location - United Kindgom

Post by parkookk » Fri May 17, 2024 9:36 pm

Will the codes @uksitebuilder provided work for url friendly websites? Because I asked ChatGPT and it said no it will not. :)

I am using English OpenCart 3.0.3.8, Default Template/Theme, php 7.4, Latest Brave Browser (No Ads-Blocker, etc.)


User avatar
Active Member

Posts

Joined
Thu Jan 17, 2013 11:56 pm
Location - london

Post by paulfeakins » Fri May 17, 2024 11:31 pm

parkookk wrote:
Fri May 17, 2024 9:36 pm
Will the codes @uksitebuilder provided work for url friendly websites? Because I asked ChatGPT and it said no it will not. :)
I wouldn't always trust ChatGPT.

UK OpenCart Hosting | OpenCart Audits | OpenCart Support - please email info@antropy.co.uk


User avatar
Guru Member
Online

Posts

Joined
Mon Aug 22, 2011 11:01 pm
Location - London Gatwick, United Kingdom
Who is online

Users browsing this forum: No registered users and 18 guests