Post by eire » Wed Jun 11, 2014 9:46 am

Hi,
I've an opencart site with 220 categories and 2,500 products. We've been having issues with Googlebot not being able to access our site (robots.txt *Never a 100% error rate) over the past few months that's killing our SEO. We've analyzed all access and log files and nothing is out of the ordinary, no 5xx or 4xx or other http status codes that would cause any issues. I use nitropack to cache our pages and generally do a stock update and cache refresh twice a week. I use xenu link sleuth to rebuild/re-cache my most popular categories (appox 100 pages) and sometimes, the URLs status come up as unavailable instead of ok. There is nothing in the access or error logs, no entries for these urls that are appearing as unavailable.

I'm thinking that maybe google is getting a similar issue when trying to contact/crawl our site. I've started using atop to monitor our server and system resources and today I ran xenu with two threads on the site after clearing all caches. CPU usage was at 100% (sys 53%, user 47%) and there were two php-cgi processes running using 85% between them and mysql was using the remaining 15%. I'm on a VPS with one vCPU and 1GB of memory, 10MB/s port speed.

Do you think the speed and memory size of my VM is causing the issues?
Are these memory usage stats unusually high or standard?
Would upping my VPS to 2 x vCPU and 2GB of RAM make much of a difference?

Thanks a lot.

New member

Posts

Joined
Sat Sep 22, 2012 5:04 am

Post by hassanbokhary » Wed Jun 11, 2014 1:26 pm

Hi,

What is your website URL?

There can many factors for high CPU Usage.

Yes better/powerful VPS will have better speed and performance.

Newbie

Posts

Joined
Tue Oct 08, 2013 6:29 pm

Post by eire » Wed Jun 11, 2014 8:06 pm

Hi Hassan,
It's www.petcara.ie. I should start having atop logs from tomorrow onwards that I can analyse for peaks in cpu and memory load.

New member

Posts

Joined
Sat Sep 22, 2012 5:04 am

rph
Expert Member

Posts

Joined
Fri Jan 08, 2010 5:05 am
Location - Lincoln, Nebraska

Post by eire » Thu Jun 12, 2014 7:44 am

Hi Ryan,
I have product counts disabled for months. I've just upgraded to 2 CPU cores and 2GB of RAM. I've atop running and logging since this afternoon. Should have some good data in a few days to analyse.

New member

Posts

Joined
Sat Sep 22, 2012 5:04 am

Post by IP_CAM » Thu Jun 12, 2014 8:11 am

#1 >> 10MB/s port speed <<
#2 >> 1GB of memory <<
#3 >> 220 categories and 2,500 products <<

#1 is this really 10 MegaBytes or just 10 Megabits , this very very old Network standard?
If so, it should be changed to minimum 100Megabits/s, better is 1 Gbit.

#2 should be changed to 2-4 GB
As more you have, the better it works.

#3 You have many Categories, compared to their Content.
This will have a measurable effect.

Ernie

Please don't send me OC Forum Personal Messages, just contact: jti@jacob.ch
---
OC 1.5.6.5 LIGHT Test Site: http://www.bigmax.ch/shop/
OC 1.5.6.5 V-PRO Test Site: http://www.openshop.li/shop/
My Github OC Site: https://github.com/IP-CAM
2'600+ FREE OC Extensions on the World's largest Github OC Repository Archive Site.


User avatar
Legendary Member

Posts

Joined
Tue Mar 04, 2014 1:37 am
Location - Switzerland

Post by eire » Thu Jun 12, 2014 8:21 am

Hi Ernie,
I upgraded the VPS today to the following:

#1 Typo :-0, It was 10 Mbps, it's now gone up to 100Mbps
#2 This has gone up to 2GB
#3 We will probable revisit the categories at some stage down the line but wouldn't envisage it getting any smaller than 180

Thanks,
Philip

New member

Posts

Joined
Sat Sep 22, 2012 5:04 am

Post by IP_CAM » Thu Jun 12, 2014 9:02 am

eire wrote:Hi Ernie,
wouldn't envisage it getting any smaller than 180..
hey, that's 20 percent less, everything counts.

A few things, I just recall to the theme:

Check on this, it may help:

https://github.com/shaman/opencart-mysql-cache

This Tool is very informative:

http://tools.pingdom.com

Not Opencart-specific, but the .htaccess info contains useful 'lines'.

http://code.tutsplus.com/tutorials/opti ... --wp-25279

And something on speeding up CSS:

http://code.tutsplus.com/tutorials/opti ... -net-24755

a few 'beginner' hints:
AVOID TO FILL UP your 'Welcome-Start-Page'. keep OUT everything, not directly related to JUST A FEW MOST FAVOURED 'Sellers' or Sections of your Program! The first LOAD, with the Category-Setup, is so 'DB' time-intensive, there is just no 'room' for many other tasks. This Page MUST BE loadable in less than 5 Seconds. If not, you're not gonna make it!

Call whatever (JS/CSS/??) Scripts, download them, whenever possible, from/to your own Server, don't rely on external Sources, not even the popular ones. (just click their Link-Line in your Browser Source View Window to get 'em.., usually)

Get all that Google/Facebook/Linkedin/Twitter/e.t.c. - Stuff, everything, requiring CONTENT, called up from another URL, out of your Start Page (incl. Header/Footer-Sections in general), most of them call their Home-URL to send back some data, this is time-intensive stuff. Place them in left/right/bottom/top-Boxes anywhere else, but not into the Main Page. Because of SOME of the above mentioned Linking, in some Cases (Seamonkey/Firebird confirmed), a never-ending Browser-pageloading can result, one more very annoying Visitor-Experience in a Shop..., since one never know's, what (seemingly) not has fully beeing loaded yet.

Whatever you do, take the LEADERS, i.E. Google, as shinig Example, on how LEAN a 'Hello/Welcome-Page' can actually look, to do it's Job! OC requires a little more, I am aware of it, but still, it's like in Reality, as less 'weight' you 'place', as faster it gets...

Good luck

Ernie

Please don't send me OC Forum Personal Messages, just contact: jti@jacob.ch
---
OC 1.5.6.5 LIGHT Test Site: http://www.bigmax.ch/shop/
OC 1.5.6.5 V-PRO Test Site: http://www.openshop.li/shop/
My Github OC Site: https://github.com/IP-CAM
2'600+ FREE OC Extensions on the World's largest Github OC Repository Archive Site.


User avatar
Legendary Member

Posts

Joined
Tue Mar 04, 2014 1:37 am
Location - Switzerland

Post by rph » Thu Jun 12, 2014 10:51 am

eire wrote:Hi Ryan,
I have product counts disabled for months. I've just upgraded to 2 CPU cores and 2GB of RAM. I've atop running and logging since this afternoon. Should have some good data in a few days to analyse.
Disabling product counts doesn't stop the queries. It just prevents them from being displayed.

-Ryan


rph
Expert Member

Posts

Joined
Fri Jan 08, 2010 5:05 am
Location - Lincoln, Nebraska

Post by IP_CAM » Thu Jun 12, 2014 10:58 am

rph wrote:Disabling product counts doesn't stop the queries. It just prevents them from being displayed.
just tell him, that you have a solution for this. (if I'm right !?)

Ernie

Please don't send me OC Forum Personal Messages, just contact: jti@jacob.ch
---
OC 1.5.6.5 LIGHT Test Site: http://www.bigmax.ch/shop/
OC 1.5.6.5 V-PRO Test Site: http://www.openshop.li/shop/
My Github OC Site: https://github.com/IP-CAM
2'600+ FREE OC Extensions on the World's largest Github OC Repository Archive Site.


User avatar
Legendary Member

Posts

Joined
Tue Mar 04, 2014 1:37 am
Location - Switzerland

Post by rph » Thu Jun 12, 2014 10:59 am

I already linked to it.

-Ryan


rph
Expert Member

Posts

Joined
Fri Jan 08, 2010 5:05 am
Location - Lincoln, Nebraska
Who is online

Users browsing this forum: No registered users and 34 guests