Post by adambgart » Tue May 09, 2017 6:36 pm

Hi,

Hoping someone can provide some pointers or ideas on how I can improve our TTFB.
We are running opencart 2.1, we have a lot of products: 67,000+ & 4,000+ categories. The homepage loads quickly enough, but the category pages are very slow the biggest issue being TTFB of over 4 seconds...

I have installed the Nitropack Cache software from isenselabs which I think I have setup correctly and we are running the website on a dedicated VPS (6 cores / 8gb Ram)
Category page link:
http://bigboxart.com/index.php?route=pr ... &path=4592

Webpage test result:
http://www.webpagetest.org/result/17050 ... view_step1

Category count is disabled.

Any help on how I can improve the TTFB would be a great help, my knowledge of server configuration is rather weak :(

Many thanks

Newbie

Posts

Joined
Wed Apr 09, 2014 9:57 pm

Post by sculptex » Wed May 10, 2017 6:52 am

If you are using mysqli db connection, try the attached vqmod, load a page then disable the mod again so as not to generate loads of log files. Then look into the system/logs folder (or system/storage/logs) for one of the SQL_LOG files generated and post results here.

*FILE UPDATED COMPATIBLE WITH 2.3*

Attachments

Last edited by sculptex on Fri May 26, 2017 6:37 am, edited 1 time in total.

ImageImage


User avatar
Active Member

Posts

Joined
Tue Sep 13, 2011 3:07 am
Location - UK

Post by IP_CAM » Wed May 10, 2017 10:00 am

well, according to GTMETRIX Test Results, you still have a lot of ways, to make your Software work,
as it could. But for the WAIT Time of 6+ Seconds, it's your server, doing this to you.
The Test Results are commented, compressed JS+CSS Scripts are downloadable,
and so easy to rename (de-number!) again and install. And to fix missing image values, Mod exist!
But if a simple 20-product category Page already requires 1'200+ Lines or Source,
just to get it's (hidden) HEADER CONTENT, you will find sure your limits, by use of such a Theme... ;)
https://gtmetrix.com/reports/bigboxart.com/x7WgzKD4

In addition, you seem to use rather dangerous links /image titles, containing WHITE SPACE GAPS!!!:
/Artist%20M-P/Arthur%20Melville%20A%20Cabbage%20Garden-228x228.jpg
so, better don't be surprised, if you ever run into problems because of doing it against the Rules... :laugh:
Good Luck !
Ernie

My Github OC Site: https://github.com/IP-CAM
5'200 + FREE OC Extensions, on the World's largest private Github OC Repository Archive Site.


User avatar
Legendary Member

Posts

Joined
Tue Mar 04, 2014 1:37 am
Location - Switzerland

Post by sculptex » Wed May 10, 2017 3:38 pm

Ernie, the real issue is having 67,000 products and 4,000 categories, the bottleneck is pretty much the database. Although the database should scale up well and cache results automatically (in the engine) for so many products this is not feasible and increasing cache sizes etc. is not a solution as I posted about recently.
I would propose to make some index optimizations and possibly restructure some queries or add a hybrid cache system for the more time consuming queries but I could not guess these without looking at the logs. I normally only deal with stores with hundreds of products (maybe a couple of thousand). I have previously written a script to generate 20,000 products but I did not experience much slow-down but thats probably because they were artificially generated with single categories and no related products for example, so I need a real world analysis to propose real-world optimizations.
Your advice is good for general optimizations but they could spend a month stripping spaces from filenames and make no difference to TTFB! (which is the issue here).

ImageImage


User avatar
Active Member

Posts

Joined
Tue Sep 13, 2011 3:07 am
Location - UK

Post by IP_CAM » Wed May 10, 2017 8:56 pm

well, Isenselabs is selling an auto-productsgenerator, if one really cares, to test Shops, with masses of products,
and during the past 36 Months, I created more than 150 such testshops already, partly containing up to 40 K Products.
So, I am quite aware, what it takes, to streamline OC, to get top results. After all, I got my 99 out of 100 percent on
pageload results on Category Pages on GTMETRIX, and so far, I did not find any other OC Setup, coming close.
And i am not even using a dedicated Server, I just make use of the widespred free knowledge, published on
the web.

So I learned, that it makes a difference, like using a DB, full of never used Code, compared to one,
just containing, what it really requires. And keeping multi-thousands of lines, just to have 'em, would be
simply contrary to any programmer logic.

But I am not a Programmer anyway, just a Tester and Tuner, eager to find out. ;)
So, I leave it to others, to play around with cheap shared Servers, limited Memory, and Code,
full of 'brakes', and images, named contrary to rules, it's not my problem, after all... :D
Ernie

My Github OC Site: https://github.com/IP-CAM
5'200 + FREE OC Extensions, on the World's largest private Github OC Repository Archive Site.


User avatar
Legendary Member

Posts

Joined
Tue Mar 04, 2014 1:37 am
Location - Switzerland

Post by adambgart » Thu May 18, 2017 8:46 pm

Thankyou all for your responses, and apologies for my delay in responding.

I managed to reduce the TTFB considerably by simply removing the fine tune your search from the category page! it instantly dropped the TTFB from 6 seconds to 600m/s which I can deal with...

Thank you all for your suggestions, a lot more streamlining needed on my end :)

Newbie

Posts

Joined
Wed Apr 09, 2014 9:57 pm

Post by IP_CAM » Fri May 19, 2017 2:14 am

I have previously written a script to generate 20,000 products but I did not experience much slow-down
but thats probably because they were artificially generated with single categories


Well, by use of Isenselabs famous Auto-ProductsGenerator, everything is done like in the default way
of entering Products, and one can code-define just about any value, to then be part of the 'real' Game,
under real Conditions. And by use of a decent amount of images, one is likely not even beeing repeated
on the same Category Page, to possibly 'harm' real timings...

As a consequence of this, I am able to test my Sites, with any amount of products, options, images,
whatever, in order to get 'near-real' Test Results. I also invested more time on this Speed Stuff than most
everybody else here, by building near to 200 different (- ~50KProd.) setups during 3+Years, playing with OC.
I therefore know well, what it takes, and where one has to change what, to peak OC to it's max, especially,
when it comes to Category pages, since I don't just prepare a Front Page only, to make it look, on first view,
like a 'speedy' place. ::)

And it only takes 20 Bucks, and a couple thousand sample images, to auto-create 50K + Products, and so
also fill up every existing category with a just about equal amount of them. And I also leave the informative
and customerfriendly OC Product Counting switched on, in all my Test Shops, because this don't make
no difference, at least not, if those values are cached. ;)

There must be a reason, why GTMETRIX gives me a 99er, on a Desktop View Category Page, when
it comes Page Load Performance. And the still comparably 'poor' 89er Result on the YSlowValue only exists,
because I did not bother to combine Styles and Scriptings, just to get this value a few points up as well, at
least not, as long as it's still undecided, what will eventually be used in my final Thing. 8)

2'001 active Products, 671 Prod.Images, ~ OC default Category settings, Bootstrap Theme:
https://gtmetrix.com/reports/www.bigmax.ch/cfXRI8i4
---
But I made some tests tonight, by use of exactly 50'000 autogenerated Products, on this
mostly similar equipped Site, to actually finding nothing, really different from what it was before... :D
https://gtmetrix.com/compare/XDRBpN2e/4XahYJpc/RYDUPhaD

PS: Always RE_TEST, to see latest Values!
---
But it also depends HEAVY on the Theme and other Extensions used, I just tested another Bootstrap Theme
costing me 25/20 Percentage Points, just because of all that shiny crab and pic's, all over the place,
likely also produced by Scripts and styles, filling up the header sections with XX code lines, and so additionally
slowing down overall page load.... for likely then not even used/needed on/for i.E. category pages at all... :laugh:

And OC requires a well-done Server, tied to a high-speed broadbband-line, without that, it will just not work,
as it could. One has to be aware, how much CPU Power and Server Memory it takes, (I got 640MB reserved)
to produce a simple speedy oc-cat-page, but on a 'cheap' monthly Server Rent , IP-shared with at least
hundreds of others, one cannot expect usable results, when it comes to handling larger Quantities of Categories,
Products, Product Options, and so on....

And It's all 'free+open' Knowledge too, and just, to 'add' my 2 Cents of Wisdom on this.
I even use an regular IP- shared Server for my 40+ Sites, and for all that testing... ! ;)
But it's SWISS, so, like most things, known to work, as advertized.
Ernie
--
Image
---
50'000 Product Listed:
Image
---
DB View:
Image

My Github OC Site: https://github.com/IP-CAM
5'200 + FREE OC Extensions, on the World's largest private Github OC Repository Archive Site.


User avatar
Legendary Member

Posts

Joined
Tue Mar 04, 2014 1:37 am
Location - Switzerland

Post by sculptex » Mon May 22, 2017 5:24 pm

@adambgart - great that you got it down to a sensible speed, well done.

@ernie -
All this effort, have you got a real world shop that sells anything?!?
oc_product_to_colour - not familiar with that one?
As this is oc 2.x thread, so are you getting same results on oc2?
Whats the 640mb? Do you mean thats your total memory allocation of the shared server? that's not a lot but great if you are getting these results with that.
I see from the db that isenselabs product generator generates product options, attributes and related which is great for testing purposes and I should really add to mine for a better true world analysis. Could also do with (some) products also being allocated to multiple categories aswell.
I know you keep going on about gtmetrix but google is king so I always optimize for google pagespeed first then gtmetrix etc. second. I agree gtmetrix analysis is probably a better gauge of user experience than pagespeed but google pagespeed scores actually help/hinder your search engine rankings. (Also I once wasted a few hours on gtmetrix trying to get 100% and all it said was wrong was one file could be minimized by 2 more bytes even though I tried every minimizer out there, even reducing it by more than the 2 bytes it complained about yet it still complained!!!)

ImageImage


User avatar
Active Member

Posts

Joined
Tue Sep 13, 2011 3:07 am
Location - UK
Who is online

Users browsing this forum: No registered users and 262 guests