Opencarts use of resources

Can anyone tell me what causes the heaviest virtual memory load on opencart? I'm using version 1.5.2.1 and keep getting temporary "Resource Limit Reached". When I check the servers resource use data, the virtual memory is hitting the max most, Entry processes now and then. I only had 11 visitors on the stores when this started happening today, so I do not think it's that.

Seems it happens a lot when I'm adding products to the store - would it be better for me to ftp images instead of using the image manager? Does it use a lot of vm?

I have the Increase Page Speed v4.1 mod running and not sure if there is a setting I need to adjust or something else I need to do to minimize this problem?

I'm about to optimize my database hoping it will help a bit, and clicking around myphpadmin I found this under routines:
Could this be at least part of the issue? If so, how do I correct?

Perhaps primarily by longstanding habit and limited tolerance for my own time than should not be taken in mere file transfers, I always transfer images via ftp or scp and never by image mangers. I would guess that part of what slows image managers down is combination of http traffic with the time spent communicating with databases and caches.

Heaviest, hard to tell. But combined, apparently database interactions.

I was a trifle startled a couple of months ago when I checked an OC installation's server stats for traffic and database status, and found that in raw numbers there were far into the tenth power of ten "hits" which from traffic logs and addresses could not have been generated by ordinary numbers of (re)visitors or even of perhaps surprisingly many and unsaturated robots' visits, and had to have come from inside the server, which possibility might well have meant bit-hits or byte-hits relative to database or cache but not visitor-hits relative to address, and caused me to look at database stats and find red-flagged entries which server resettings would presumably remedy. (Short answer to latter, between the mysql and mysqli extensions to php itself, was that server staff didn't want to know anything about it even for the sake of system-wide shopping carts.)

You might try tinkering with cache settings (perhaps size, latencies, timeouts, whatever is there) as well as database settings (access, caching, whatever is there). It would probably help to notice (write it down . . .) which if any changes detectably affect virtual memory in inverse rather than direct proportion to apparent speed.

You posted while I thought mine had already posted (sometimes somehow posts don't post, even long before or after the next post).

It appears from server support's response to my inquiry about mysql vs. mysqli that some [hosts] will and some won't deal with it. As rph noted elsewhere, neither one is complete. Perhaps server people are waiting for the egg to hatch. I can relate to that in context of having no interest in Windows till at least one Service Pack has already past-tense issued (and never Windows on a public machine), but the mysql upgrade relates to a php extension and is apparently not a big deal. On the one hand a modicum of conservatism is appreciated, as where an abundance of servers still uses a completely stable even if nominally superseded version of Apache, and on the other hand a progressive progress is appreciated, as where php.exe versions finally bear retirement for upgrade.

As long as what I need works and fast enough, I'm not worrying about the host (over)loads.

I'm sorry but I have no clue where to tinker with cache settings or database settings or what I should try without breaking things. Any specific suggestions?

I ventured to guess that maybe the page-speeder had anything to poke, but evidently not. That leaves them up to the server's own people. They break enough stuff without our help.

My overall impression is not to worry about it, the server machine will use whatever memory it wishes or thinks it needs, and whatever ram ditto.

I am worried about it since it is happening every few minutes sometimes.

Anyone else have suggestions?

So talked with my hosts and unfortunately, they don't have any suggestions so still trying to work thru what I can do on a limited budget at the moment to resolve some of this. Apparently it's my cpu usage that is maxing out the most:

Current Usage CPU Usage 100% / 100%
Memory Usage 17.2M / 1.0G
Entry Processes 1 / 20

Is what is showing at the time I write this.

My hosts say its coming from index.php in my opencart route folder. Is there anything in there I can eliminate that might be causing it? For example, I only use 1 language, English, and noticed there is a large section to detect language. (using 1.5.2).

Would upgrading to 1.5.5.1 DEFINITELY solve it? I really don't want to have to redo so much and risk breaking my stores right now as I have a press release going out for selling shoes to American Idol and am afraid of messing with things too much. Are there any 1.5.5 files I can safely use in place of the 1.5.2 ones that might help? (at one time, I upgraded only the filemanager files for example which GREATLY sped up my admin and server requests). But I know that 1.5.4 and up use html5 which I do not currently use, so wondering if this or other changes in the newer version may help?

If you haven't already disable product count. It's very database intensive.

Product and category counts have always been disabled on my stores. Never use them.

In jest . . . Static discharge from your two dogs and "odd number of Ragamuffin cats" tickling the host servers?

From what you've added, the problem of soaring chip usage appears to be lack of what is now called load sharing in order to spread workload. Upgrading to any particular version will NOT DEFINITELY solve it; nor cause it. If you have too much going on at once, then sharing load one way or another will tend to reduce chip usage(s). The text ""Resource Limit Reached" is counterpart to a performance graph clinging to 100% ceiling, whether "virtual memory is hitting the max most" but "processes now and then" or the processor chip is under siege with database calls. How many language or other pure text ascii .php and .tpl files there are is not a concern. The activity is relating primarily to database connections overall, while index.php executes itself.

Host City is not altogether rather observant or helpful sometimes. Whether single or plural "hosts say its coming from index.php in my opencart route folder" you can see in your own address bar that index.php self-executed and reincarnated itself on the spot with addenda "route" and to where among the directories, the immediate point being that there is no "route/" directory unless you added one or somebody messed with the core or added a serious modification. [Edit, added:] In fairness it is possible that they intended route directory to mean the index.php self-appended route's landing directory.

For your number of 30-odd carts subserving widely varied needs without too terribly many items and with probably fewer customers but more orders than items, the problem might actually not even be arising from your own stores, since your apparently sole or primary host's "unlimited" Linux features (in both plans) tend to raise the possibility that its own combinations of shared and dedicated addresses need to be load-balanced, in-house there, among its very own accounts among its very own hard discs and motherboards. It would not be the first time that a host oversold its own machine capabiltiies or deferred catch-up maintenance till little discrepancies are noticeable. Your seat ticket and flight are ordinarily for one or three years at a time, where your host offers two options; I assume that you have the upper class.

Where those concerns raise the host's own problems, this could be a good time to ask support for upgrading from the mysql extension to the mysqli extensions to php itself, toward easing the database burdens through the processor chip(s).

Load sharing is offered in several (fully virtual) "cloud" hosts but their arrangements are generally more expensive than traditional shared or dedicated, and with as little as possible dynamic on hard discs they are disasters waiting to happen if rigorous backup schedules are not set up in advance. Much simpler than that in most regards is load sharing among traditional hardware components, where then a "cloud" just separates major functions among separate drives or boxes (e.g., web, mail). There are Stateside hosts whose capacities are not oversold.

I'll putter off and on through the day over load sharing to see what that might turn up that's new and wonderful beyond just quietly putting different carts on different machines or hosts.

Actually I had their business hosting account but switched earlier this year to the reseller hosting account thinking it would help. It's still shared, but they have less sites on each of those servers than their regular ones.

Been doing a lot of little things and it seems to be improving a bit. The cpu spikes to 100% are much briefer again which is good. Still want to figure out how to optimize everything better, but so afraid I will break something at a crucial time - like American Idol just called to buy some more boots & so glad I didn't have everything down when they were looking!

Some of the things I've done (in case anyone else needs to figure it out) that may/may not have helped were to optimize the database, update all my other software on my account, update vqmod to the latest version, uninstall all mods I wasn't using, and clean up some directories on the server. I also updated the system/database/mysql.php with the one from 1.5.5.1 and uploaded the mysqli.php as well (though I do not think 1.5.2 uses it). I already had removed category and product counts and added the database indexes suggested in other threads.

Any ideas anyone might have are greatly appreciated. I had read a thread about moving images to a subdirectory - would this be of any benefit for cpu load or not make a difference? CDN things confuse me - don't know anything about them and I know the speed up mod I have has settings for it, but not sure what they are/if it would help.

Thanks again for all the help.

On mysqli, the point was to ensure that it's php extension (of php.exe on the server itself) is enabled in your own subsidiary php.ini or better in the server's own master php.ini file(s) or both.

On image subdirectories, the nesting of an abundance of files of one kind (by extension, name, whatever) simplifies the machine's search for each one, for the same reasons that they're usually easier for you to find when grouped than when ungrouped. Traditional dictionary tabs are probably the longest-standing instance of the principle that most of us will personally remember. (Those of us who are old enough to know and remember what dictionaries are or were still outnumber the children who increasingly have no concept of dictionaries, and who partly for want of looking at dictionaries are increasingly illiterate.) The machine will read in sequence (possibly by name, timestamp, extension) every file in its way till it finds the one called. The tiny snippets of time may or may not matter. There are practical considerations in setting up nested .htaccess files (which rootward are more master and upward are more subsidiary) so as to minimize or optimize the time spent by machines wandering amongst paths to find anything. The tiny snippets of time may or may not matter.

labeshops wrote:Product and category counts have always been disabled on my stores. Never use them.

It looks like 1.5.2.1 didn't have disabling of counts (though no version actually properly removes the feature, they just hide the output). If you manually removed it did you make sure you got rid of:
also?

Will that (or similar, far enough back) work among all versions?

Yes. I have Product Count True Disabling / Speed Boost which handles it in 1.5.5.x. There are some free and commercial mods available for earlier versions though I can't speak to the quality.

I'll begin implementing the beheadings of that beast. You already convinced me in another thread a short while back that counts deploy to no useful avail (purely academic curiosities having no practical use other than code evaluations being excepted).

I didn't mention it in my list of things I did, but I did discover the files discussed in this thread http://forum.opencart.com/viewtopic.php?f=20&t=98644 in my download folder too. I deleted them and changed permissions on downloads, but have now implemented the recommendations in the above thread. Not sure if this could have been the problem all along, but so far, my resources are not reaching their limits (or not as often).

@rph don't remember which mod I used to remove the category/product counts honestly. Did it about day 1 as I never saw any reason for them to begin with.

See also:
http://forum.opencart.com/viewtopic.php ... 22#p402816
and posts following it in the thread.

Hi labeshops,

Have you solved this issue? My site with 33 categories and 700 products have this issue, especially in the admin page. I have been looking for solution for weeks but I couldn't find the solution yet.

Appreciate if you can share the solution.

Thanks~