Post by Jwestri050 » Sun Aug 23, 2015 6:49 pm

Hallo,

Ik heb wat gelezen over Robots.txt. Ik heb enkele sites al bezocht op internet maar ik kom er niet uit.
- Hoe maak ik het aan? (iemand voorbeeld? ln ieder geval voor google etc.)
- Waar zet ik het in mijn opencart php?
- Wat moet er in staan?

Ik hoop dat iemand me kan helpen :)

Newbie

Posts

Joined
Sun Aug 23, 2015 6:47 pm

Post by IP_CAM » Tue Aug 25, 2015 12:29 am

sorry, I cannot reply in dutch, but this would be a typical OC-specified robots.txt file content,
to tell, i.E. Yandex, and other Bots/Robots, what to AVOID. BUT STILL, ROBOT-TXT is ONLY just a recommandation,
no ROBOT has/need's to follow those Rules! Just to mention it.
best regards from a swiss cheese...
Ernie

Code: Select all

User-agent: *
Disallow: /*route=account/
Disallow: /*route=affiliate/
Disallow: /*route=checkout/
Disallow: /*route=product/search
Disallow: /index.php?route=product/product*&manufacturer_id=
Disallow: /admin
Disallow: /catalog
Disallow: /download
Disallow: /system
Disallow: /*?sort=
Disallow: /*&sort=
Disallow: /*?order=
Disallow: /*&order=
Disallow: /*?limit=
Disallow: /*&limit=
Disallow: /*?filter_name=
Disallow: /*&filter_name=
Disallow: /*?filter_sub_category=
Disallow: /*&filter_sub_category=
Disallow: /*?filter_description=
Disallow: /*&filter_description=
Disallow: /*?tracking=
Disallow: /*&tracking=

User-agent: Yandex
Disallow: /*route=account/
Disallow: /*route=affiliate/
Disallow: /*route=checkout/
Disallow: /*route=product/search
Disallow: /index.php?route=product/product*&manufacturer_id=
Disallow: /admin
Disallow: /catalog
Disallow: /download
Disallow: /system
Disallow: /*?sort=
Disallow: /*&sort=
Disallow: /*?order=
Disallow: /*&order=
Disallow: /*?limit=
Disallow: /*&limit=
Disallow: /*?filter_name=
Disallow: /*&filter_name=
Disallow: /*?filter_sub_category=
Disallow: /*&filter_sub_category=
Disallow: /*?filter_description=
Disallow: /*&filter_description=
Clean-param: tracking

My Github OC Site: https://github.com/IP-CAM
5'200 + FREE OC Extensions, on the World's largest private Github OC Repository Archive Site.


User avatar
Legendary Member

Posts

Joined
Tue Mar 04, 2014 1:37 am
Location - Switzerland

Post by coldrex » Wed Jan 04, 2017 2:52 am

But what if:

1) I have seo urls, not like these: /index.php?route=product/product*&manufacturer_id=

2) my admin folder called not /admin, but /secretname - if I state it in robots.txt, anyone can enter and see it ? ..

Should I just exclude the above lines?

User avatar
New member

Posts

Joined
Thu Jun 07, 2012 10:05 pm

Post by victorj » Sat Jan 07, 2017 6:18 am

robots.txt is volledig achterhaald en nutteloos, geen aandacht aan besteden.

Koeltechnische deurrubbers eenvoudig online op maat bestellen.
Alle niet stekplichtige onderdelen zoals scharnieren, sloten, randverwarming en verlichting voor alle typen koelingen en vriezers.
https://koelcel-onderdelen.com


User avatar
Expert Member

Posts

Joined
Sat Jun 25, 2011 4:09 am
Location - Alkmaar Holland

Post by Jaap » Wed Jan 11, 2017 6:50 pm

Een robot.txt bestand is slechts nodig als je wilt dat zoekmachines bepaalde pagina's niet indexeren.
Anders, zoals gezegd, nutteloos.

User avatar
Active Member

Posts

Joined
Fri Apr 15, 2011 12:10 am
Location - the Netherlands
Who is online

Users browsing this forum: No registered users and 48 guests