Post by beaconoils.co.uk » Thu Mar 15, 2018 5:15 am

How can i stop Yandex from crawling my site?

New member

Posts

Joined
Sat Jan 27, 2018 8:05 am

Post by Wid » Thu Mar 15, 2018 5:57 pm

search google for this!

solution: https://yandex.com/support/webmaster/co ... ts-txt.xml
file: robots.txt

Code: Select all

User-agent: Yandex
Disallow: / # blocks access to whole site

take a visit to My Modules !


Wid
New member

Posts

Joined
Fri Feb 02, 2018 9:19 pm
Location - Athens/Greece

Post by beaconoils.co.uk » Fri Mar 16, 2018 12:38 am

thanks i will look at this

New member

Posts

Joined
Sat Jan 27, 2018 8:05 am

Post by beaconoils.co.uk » Sun Mar 18, 2018 8:04 am

i can not find robots.txt file i had one but seems to have disappeared prior to installing vqmod

New member

Posts

Joined
Sat Jan 27, 2018 8:05 am

Post by beaconoils.co.uk » Sun Mar 18, 2018 9:23 pm

re posted my robots.txt and stopped yandex from crawling my site thanks for your help

New member

Posts

Joined
Sat Jan 27, 2018 8:05 am

Post by beaconoils.co.uk » Mon Mar 19, 2018 5:28 am

I used this robots.txt file but yandex is still coming to my site can anyone correct this?

User-agent: Yandex
Disallow: / # blocks access to whole site
User-agent: *
Disallow: /admin
Disallow: /download
Disallow: /system
Disallow: /*route=account/
Disallow: /*route=affiliate/
Disallow: /*route=checkout/
Disallow: /*route=product/search
Disallow: /index.php?route=product/product*&manufacturer_id=
Disallow: /*?sort=
Disallow: /*&sort=
Disallow: /*?order=
Disallow: /*&order=
Disallow: /*?limit=
Disallow: /*&limit=
Disallow: /*?filter_name=
Disallow: /*&filter_name=
Disallow: /*?filter_sub_category=
Disallow: /*&filter_sub_category=
Disallow: /*?filter_description=
Disallow: /*&filter_description=
# Disallow: /*?tracking=
# Disallow: /*&tracking=

New member

Posts

Joined
Sat Jan 27, 2018 8:05 am
Who is online

Users browsing this forum: No registered users and 530 guests