Page 1 of 1

User Agent Indexing Issues

Posted: Mon Apr 08, 2019 12:31 pm
by kingkong001
Hi Everyone,

I'm working on the site buydirectonline.com.au, which used to be crawled with no issues. All major search engines had it indexed and screaming frog could crawl the site with no issues.

However, all of a sudden, screaming frog is returning a 403 error, and the site is being de-indexed on all major search engines.

I've checked the obvious:
- robots.txt
- search console messages/issues
- htaccess file

Nothing seems to have changed, everything seems to be how it was when the site was crawling properly.

Does anyone have any ideas on what i should try/investigate next?

Thanks in advance!

Re: User Agent Indexing Issues

Posted: Tue Apr 09, 2019 5:55 am
by IP_CAM
Well, you're trying to please a lot of Machines, but not so much Visitors. In addition
to present (default value) categories, adding ~3.6MByte Download Pages. But you
have everything built-in, what only results in negative numbers, but not one of them
will actually help to raise Customer Interest for your Site. You're doing the Job for
others, instead of getting the Job done, as it might have been, but some time ago.

It's like sitting the wrong way on a horse's Saddle. But modern search engines do no
longer honor such doings, one should be aware about such.
---
Good Luck!
Ernie
---
https://gtmetrix.com/compare/KGcAPEnK/xmSJHdIB
Image