Post by Dutch Pride Code » Thu Dec 02, 2021 7:14 pm

Hi everybody,

One of my stores running 3.0.3.2 (default theme) is taking up more and more disk space. There is still more than enough disk space available in my hosting package, but I really really really wonder what is taking up all that space. It does not seem to be the regular new product or customer that registered him-/herself. I've actually recently removed a lot of disabled and old products that are no longer in use.

Is there any way to figure out what keeps eating disk space in my store?
What extension, what part of the website, any clues?

Thanks in advance!
Last edited by Dutch Pride Code on Thu Dec 02, 2021 11:51 pm, edited 1 time in total.

User avatar
Active Member

Posts

Joined
Sun Jan 26, 2020 9:46 pm

Post by thekrotek » Thu Dec 02, 2021 7:28 pm

Use SSH to find big folders and files.

Professional OpenCart extensions, support and custom work.
Contact me via email or Skype by support@thekrotek.com


User avatar
Expert Member

Posts

Joined
Sun Jul 03, 2016 12:24 am


Post by paulfeakins » Thu Dec 02, 2021 9:02 pm

EMGX wrote:
Thu Dec 02, 2021 7:14 pm
One of my stores running 3.0.3.2 (default theme) is taking up more and more disk space.
Always the sign of a ticking time-bomb that will eventually crash your site.

Check all the log files and database tables for any that seem way too large.

If you can't find it, feel free to get in touch with a developer such as ourselves as this is a problem we solve frequently.

Paul.

UK OpenCart Hosting | OpenCart Audits | OpenCart Support - please email info@antropy.co.uk


User avatar
Guru Member
Online

Posts

Joined
Mon Aug 22, 2011 11:01 pm
Location - London Gatwick, United Kingdom

Post by xxvirusxx » Thu Dec 02, 2021 9:15 pm

thekrotek wrote:
Thu Dec 02, 2021 7:28 pm
Use SSH to find big folders and files.
Also Disk usage from Cpanel should work just fine.

Upgrade Service | OC 2.3.0.2 PHP 8 | My Custom OC 3.0.3.8 | Buy me a beer


User avatar
Expert Member

Posts

Joined
Tue Jul 17, 2012 10:35 pm
Location - România

Post by Dutch Pride Code » Thu Dec 02, 2021 10:09 pm

xxvirusxx wrote:
Thu Dec 02, 2021 9:15 pm
thekrotek wrote:
Thu Dec 02, 2021 7:28 pm
Use SSH to find big folders and files.
Also Disk usage from Cpanel should work just fine.
I could not find anything specific yet (using cPanel).
Somebody adviced me to use TreeSize on a local copy of my store.
Might try that for now.
paulfeakins wrote:
Thu Dec 02, 2021 9:02 pm
Always the sign of a ticking time-bomb that will eventually crash your site.
Could you please explain this?
I wonder what it could be that it would be this harmful...
Last edited by Dutch Pride Code on Thu Dec 02, 2021 10:16 pm, edited 1 time in total.

User avatar
Active Member

Posts

Joined
Sun Jan 26, 2020 9:46 pm

Post by OSWorX » Thu Dec 02, 2021 10:16 pm

I guess - as it is in so many cases - that an extension is either logging everything.
Or the error.log is fullfilled with error messages.
In both cases the webspace is filled with garbage.

Suggest you inspect all files under system/storage/logs.

And, OC 3.0.3.2 had some issues with the session tables (db).
If your provider is counting that also, better to check that table also.

The "time ticking bomb" is, that when your webspace is full, your Website stopps working.

Full Stack Web Developer :: Dedicated OpenCart Development & Support DACH Region
Contact for Custom Work / Fast Support.


User avatar
Guru Member

Posts

Joined
Mon Jan 11, 2010 10:52 pm
Location - Austria

Post by Dutch Pride Code » Thu Dec 02, 2021 10:30 pm

OSWorX wrote:
Thu Dec 02, 2021 10:16 pm
Suggest you inspect all files under system/storage/logs.
The files don't seem too large. Currently error.log has a size of 996 Kb
OSWorX wrote:
Thu Dec 02, 2021 10:16 pm
And, OC 3.0.3.2 had some issues with the session tables (db).
Although I am not sure when it becomes too large, the oc_session table contains over 3.4 million records and it appears to be 327 MB in size. It seems quite large to me. If I were to upgrade to a newer OC version, would this solve part of the issue?

User avatar
Active Member

Posts

Joined
Sun Jan 26, 2020 9:46 pm

Post by OSWorX » Thu Dec 02, 2021 10:44 pm

EMGX wrote:
Thu Dec 02, 2021 10:30 pm
Although I am not sure when it becomes too large, the oc_session table contains over 3.4 million records and it appears to be 327 MB in size. It seems quite large to me. If I were to upgrade to a newer OC version, would this solve part of the issue?
Not sure if the session bug is fixed in later versions - maybe one of the other users here can answer this.
But, this table should contain only a few entries at least, or do you have over 3 million visitors the last minutes?
For a first fix, delete all entries in that table and check the space afterwards.

And search the forum here, quite a few messages about that bug to find here.
GitHub has also some posts about that.

Full Stack Web Developer :: Dedicated OpenCart Development & Support DACH Region
Contact for Custom Work / Fast Support.


User avatar
Guru Member

Posts

Joined
Mon Jan 11, 2010 10:52 pm
Location - Austria

Post by Dutch Pride Code » Thu Dec 02, 2021 11:13 pm

OSWorX wrote:
Thu Dec 02, 2021 10:44 pm
do you have over 3 million visitors the last minutes?
Definitely not. It must have built up over a longer period of time.
Going to check out any bug fixes and remove those entries.

Many thanks!

User avatar
Active Member

Posts

Joined
Sun Jan 26, 2020 9:46 pm

Post by Dutch Pride Code » Thu Dec 02, 2021 11:46 pm

I think I found the big issue here...
On my local copy I discovered the image/cache directory, or more specifically the image/cache/catalog directory, is taking up more almost half of the used storage space! That's a bit surprising...

Is it safe to delete all contents, as this appears to be a cache directory?

User avatar
Active Member

Posts

Joined
Sun Jan 26, 2020 9:46 pm

Post by OSWorX » Thu Dec 02, 2021 11:53 pm

Found that old discussion about that bug, maybe it helps:
https://github.com/opencart/opencart/issues/7094

Full Stack Web Developer :: Dedicated OpenCart Development & Support DACH Region
Contact for Custom Work / Fast Support.


User avatar
Guru Member

Posts

Joined
Mon Jan 11, 2010 10:52 pm
Location - Austria

Post by OSWorX » Thu Dec 02, 2021 11:56 pm

EMGX wrote:
Thu Dec 02, 2021 11:46 pm
I think I found the big issue here...
On my local copy I discovered the image/cache directory, or more specifically the image/cache/catalog directory, is taking up more almost half of the used storage space! That's a bit surprising...

Is it safe to delete all contents, as this appears to be a cache directory?
That's another story beside your session bug.

Yes, deleting the items will force OpenCart to generate those files new on demand (when you do not use any extension which does that prior).
But once, you will have also again that sum of cached files.

You should check what cause your troubles.

And what do you mean with "local copy"?
Never mentioned that before!

Full Stack Web Developer :: Dedicated OpenCart Development & Support DACH Region
Contact for Custom Work / Fast Support.


User avatar
Guru Member

Posts

Joined
Mon Jan 11, 2010 10:52 pm
Location - Austria

Post by Dutch Pride Code » Fri Dec 03, 2021 12:02 am

OSWorX wrote:
Thu Dec 02, 2021 11:53 pm
Found that old discussion about that bug, maybe it helps:
https://github.com/opencart/opencart/issues/7094
Many thanks for sharing!
OSWorX wrote:
Thu Dec 02, 2021 11:56 pm
And what do you mean with "local copy"?
Just a localhost version of my store, so I could check the directory sizes on my PC to see what takes up so much space. That's how I discovered the cache directory being extremely large.

User avatar
Active Member

Posts

Joined
Sun Jan 26, 2020 9:46 pm

Post by OSWorX » Fri Dec 03, 2021 12:16 am

EMGX wrote:
Fri Dec 03, 2021 12:02 am
Just a localhost version of my store, ..
Local is not Live!
Except both have the same environement and configuration.

Full Stack Web Developer :: Dedicated OpenCart Development & Support DACH Region
Contact for Custom Work / Fast Support.


User avatar
Guru Member

Posts

Joined
Mon Jan 11, 2010 10:52 pm
Location - Austria

Post by Dutch Pride Code » Fri Dec 03, 2021 12:18 am

OSWorX wrote:
Fri Dec 03, 2021 12:16 am
EMGX wrote:
Fri Dec 03, 2021 12:02 am
Just a localhost version of my store, ..
Local is not Live!
Except both have the same environement and configuration.
I am aware, thanks. The image cache directory exists on both.

User avatar
Active Member

Posts

Joined
Sun Jan 26, 2020 9:46 pm

Post by OSWorX » Fri Dec 03, 2021 12:26 am

Okay now what?
How is the current status - solved or not?

Full Stack Web Developer :: Dedicated OpenCart Development & Support DACH Region
Contact for Custom Work / Fast Support.


User avatar
Guru Member

Posts

Joined
Mon Jan 11, 2010 10:52 pm
Location - Austria

Post by Dutch Pride Code » Fri Dec 03, 2021 6:18 pm

Solved. Needed to wait 24 hours for my host to update the storage data. Both the oc_session table and the image/cache directory needed to be removed made empty.
Last edited by Dutch Pride Code on Fri Dec 03, 2021 9:59 pm, edited 2 times in total.

User avatar
Active Member

Posts

Joined
Sun Jan 26, 2020 9:46 pm

Post by OSWorX » Fri Dec 03, 2021 6:46 pm

EMGX wrote:
Fri Dec 03, 2021 6:18 pm
Solved. Needed to wait 24 hours for my host to update the storage data. Both the oc_session table and the image/cache directory needed to be removed.
Just to clarify (for others):
NOT to be removed!
Only delete the entries.

Full Stack Web Developer :: Dedicated OpenCart Development & Support DACH Region
Contact for Custom Work / Fast Support.


User avatar
Guru Member

Posts

Joined
Mon Jan 11, 2010 10:52 pm
Location - Austria

Post by paulfeakins » Fri Dec 03, 2021 8:15 pm

OSWorX wrote:
Thu Dec 02, 2021 10:16 pm
The "time ticking bomb" is, that when your webspace is full, your Website stopps working.
Not just that - for PHP to write to your error log files, it must first open the files, and so your site will get slower and slower until finally PHP requires more RAM than you have available to write to the error logs and your site will crash.

UK OpenCart Hosting | OpenCart Audits | OpenCart Support - please email info@antropy.co.uk


User avatar
Guru Member
Online

Posts

Joined
Mon Aug 22, 2011 11:01 pm
Location - London Gatwick, United Kingdom

Post by Dutch Pride Code » Fri Dec 03, 2021 9:58 pm

OSWorX wrote:
Fri Dec 03, 2021 6:46 pm
Just to clarify (for others):
NOT to be removed!
Only delete the entries.
My bad, post is edited now ;)
paulfeakins wrote:
Fri Dec 03, 2021 8:15 pm
Not just that - for PHP to write to your error log files, it must first open the files, and so your site will get slower and slower until finally PHP requires more RAM than you have available to write to the error logs and your site will crash.
The error log was rather small. Image cache seems to stay relatively small this time as well, it brought the used storage space down from almost 5GB to 2.2GB now. Still the session table already has 30.000+ entries from the past < 24 hours. I think this needs a fix so I'll check this out:
I'm trying this to see if it changes anything.
As mentioned, editing php.ini

User avatar
Active Member

Posts

Joined
Sun Jan 26, 2020 9:46 pm
Who is online

Users browsing this forum: No registered users and 89 guests