Page 1 of 1

[SOLVED] Rate limit site

Posted: Mon Jul 08, 2024 7:00 pm
by Joe1234
Is it possible to rate limit the whole site (How quickly an IP/session can navigate to another page) since bots don't always listen to the rules? How would I go about that?

Re: Rate limit site

Posted: Mon Jul 08, 2024 7:45 pm
by JNeuhoff
Just block all those nasty bots. A good staring point would be this list for your '.htaccess'.

Re: Rate limit site

Posted: Mon Jul 08, 2024 9:23 pm
by paulfeakins
Joe1234 wrote:
Mon Jul 08, 2024 7:00 pm
Is it possible to rate limit the whole site (How quickly an IP/session can navigate to another page) since bots don't always listen to the rules? How would I go about that?
You could use Cloudflare?

Re: Rate limit site

Posted: Tue Jul 09, 2024 2:12 pm
by nonnedelectari
JNeuhoff wrote:
Mon Jul 08, 2024 7:45 pm
Just block all those nasty bots. A good staring point would be this list for your '.htaccess'.
Which would only block those bots which actually identify themselves as such, of which 90% will never visit you site, and are therefor the least of your bot concerns. Mind, htaccess, if enabled, will be read on every request, uncached.

Re: Rate limit site

Posted: Sat Jul 13, 2024 8:53 am
by Joe1234
It seems putting together a rate limiter wasn't hard at all.

I try to shy away from using htaccess when I don't absolutely need to, especially due to its recursive nature and how it's read (or at least how I understand it is read), and with how out of control and big it can get...and putting all those agents in it is definitely one way. I used to have a large deny list of them in it, and noticed when I removed them the TTFB speed improved.

Either way, I'm testing it out now to find the best settings to see if I should keep it the way I did it, or code it a different way.

Re: [SOLVED] Rate limit site

Posted: Sun Jul 14, 2024 12:12 am
by johnp