[RELEASED] Page Cache for Opencart 1.5.X

We haven't yet submitted it as an official extension, but (since updated...)

We have created a simple page level cache for opencart and posted the code on GitHub, here: https://github.com/budgetneon/pagecache.

You can also find it on the opencart extension directory, here.

It's a very simple install. One new file in your system/library directory, and 2 minor changes to your index.php file.

Update: We added an admin panel, and an auto-installer in the admin panel that will make the changes to the index.php file for you. Added a screenshot of the admin panel below...scroll down.

The upsides for this page cache:

* Simple. Easy to install and remove.
* Fast, as it skips most of the opencart code, database calls, etc, by operating/exiting at the very top of the index.php when serving from the cache.
* Has a reasonable amount of logic to determine what NOT to cache. (logged in users, https pages, etc), and has a manual list of url patterns you can edit as well.
* Open source, with the FreeBSD license. Very few restrictions on use.

The caveats:
* Tested, but not extensively. Please don't just throw this on a production site without doing your own comprehensive testing. That said, we are running a version of this (with some modifications) on our production site, budgetneon.com.
* We don't have complex logic looking at, for example, url parameters. If opencart returns an HTTP 200 code, we'll cache it (with some exceptions). Therefore, it's possible to accidentally cache a lot of pages, especially if a robot were to retrieve pages like http://site.tld?foo=1, http://site.tld?foo=2, http:///site.tld?foo=3, and so on.
* Despite our efforts to figure out what pages not to cache, we've likely missed some.
* It does not cache special http headers. So, for example, if you have logic adding X-Frame-Options headers, the cache pages won't be served with them.

Demo sites, and more info here: octurbo.com

Would welcome any feedback, bug fixes, etc. Please submit either on github, or in this thread.

Update: Screenshot of the admin panel:

Very, very interesting Solution, we used something similar, creating Category/Subcategory-Listing HTML-Pages with a Perl-based Auction System. Instead of building such Pages, everytime one called such a Routine, the 'already built' HTML Category- Pages where served, linked directly to the Script-generated Auction Pages. It saved a lot of time as well as Server Power, exactly like the one you, very generously, offer here for free.

I downloaded a product Page, created and chached by the Mod, and called it locally. Everything works fine, exept for the Product Ordering, the Add to wishlist and the Add to Compare. Just to mention this.

The logic of the generated Link-Name is still unclear to me, but possibly, this could be modified in some way, to create, i.E. 'nearer' category / product- id- type href-lines, instead of the relativly hard to understand code used, placed in, again, to me, rather unfamiliar addressed, numbered Subsections. It all may have a reason, and I'm not programmer either...

Thank you very much!

Can be tested here:

http://www.openshop.li/cart/index.php

Ernie

I downloaded a product Page, created and chached by the Mod, and called it locally. Everything works fine, exept for the Product Ordering, the Add to wishlist and the Add to Compare. Just to mention this.

Thanks for having a look. Unfortunately, you can't really test it that way. Add to wishlist, add to compare, etc, all work if you test it "live", on an opencart install. We do have a test site that is a fresh opencart install with this extension, and all the things you mention work fine. You can try it here: http://octurbo.com/pcache/ (load a page twice, to make sure it's cached, then "view source" and look at the bottom of the file).

The files it creates aren't served directly...there's logic inside the extension that's needed for all of that to work. For example, the logic won't cache a url with the string "product/compare" in it. There's also some other work going on (like setting session variables) that you won't see if you try to load the cached file locally. Consider it a 'datastore' that you aren't supposed to load in a browser.

The logic of the generated Link-Name is still unclear to me, but possibly, this could be modified in some way, to create, i.E. 'nearer' category / product- id- type href-lines, instead of the relativly hard to understand code used, placed in, again, to me, rather unfamiliar addressed, numbered Subsections.

Yes, the names of the cache files are a little odd...the long string is an MD5 hash of the url itself, and the other bits are the currency and language that were used when serving it. That allows the code to cache, separately, the same page but with different lang/currency session variables set. The directory structure is to keep the cache from putting too many files in a single directory, because some versions of linux run slow when trying to do a readdir() call on a directory with thousands of files.

You'll notice the directory structure mirrors the first two characters of the md5 hash in the file name. We're also using an MD5 hash because taking input that an outside user could create (like a url) and turning that into a filename can be dangerous. Like if I request http://yoursite.com/../../../../etc/passwd, for example, and pass that to a routine that creates files....ouch. Hashing the url ensures we get just simple letters and numbers, and no meta chars that unsafe.

None of that, however, is exposed to the outside world. You just ask for a url, like http://example.com/mycategory, and the cache extension, if the page is cached, and should be served as such, pulls the data from the respective oddly named cache file.

Very, very interesting Solution, we used something similar, creating Category/Subcategory-Listing HTML-Pages with a Perl-based Auction System.

Ah, cool. I am an old Perl hacker myself.

Can be tested here:

http://www.openshop.li/cart/index.php

FYI, the cache is running there...just fine. As long as you aren't logged in, and don't have anything in your cart, I see that it's caching pages (by the html comment it appends to the end...do a "view source" on that url).

And, "add to compare", "add to cart", etc, seem to be working just fine.

Note that once you add something to the cart, it won't serve cached pages to you, though it will serve to cached pages to others, that don't have anything in the cart. The reason for this is that the "cart" total that's on every page wouldn't work right. Since it's running on your site, you can try that. Do the "view source" thing to see you're viewing a cached page. Then, add something to the cart, and go back to the page that was cached. Do "view source" again, and you'll see you're not getting the cached page anymore. Then, empty everything from your cart, and you'll get the cached page again.

Of course, if you remove all references to "personalized data" from pages...for example, replacing the cart data with an ajax request, you could remove the logic in our extension that won't serve cached pages to an end user with something in their cart. You would just have to make sure that whatever url the ajax request was using was manually added to the skip_urls configuration setting...or it would get cached.

I probably disabled the two options, will check about tomorrow, it's getting late/early here.
Ernie
my Perl Site:
everyauction.info

Hello again,

I am fully aware of the 'meant' Purpose of the Mod, I just can't keep for guessing about, what else could be done with such goodie's, this is, why I tried to 'reproduce' all possible functions by downloading such a page and test it from my PC. Sorry, if you possibly missunderstud my remarks regarding this. I did not check, early in the morning, into the whole file-content as well, so, I was unaware of the MD5 link coding. For load-time comparence-purposes, I can also call a cached Page directly, then, all options/buttons work fine this way.

http://www.openshop.li/cart/system/cach ... acbff.html
For those checking, the pageload time will not longer change, the Source contains the saved, so fixed value.

Unfortunately, I already added a few related speed-up Mod's in the Script I presently use, so, my Shop-Site does not only reflect the implemtation of your 2 Mod's, compared with a default OC-Software. I will have to set up a new place anyway, because the latest OpenShop v.1.7.1 OC-Extension has been updated to work with VqMod v.2.5, then, I will be able to produce comparable Results.

But both Mod's look fantastic, they have, to my limited knowledge, as free OC Mod, not been done yet this way, this will take a lot of Load from a Server, when implemented. 5 Stars from me.
Good Luck

Ernie
openshop.li

PS. Check here to find some speed related info, just in case you overlooked it...:
http://forum.opencart.com/viewtopic.php?f=20&t=128378
http://forum.opencart.com/viewtopic.php?f=20&t=130433

Thank you...appreciate the feedback and you trying it out.

I did push an update to github to because it was caching captcha images.

See this for the change needed https://github.com/budgetneon/pagecache ... 5ac34d3f95

Firstly, thanks for providing a free page cache extension.

After a quick look around my test site the only problem I can see so far is the captcha. The captcha will load on first view of the Contact page but when loading the cached version it doesn't show up.

Besides this I havent noticed any other problems so far and everything is running incredibly quick!!

gtoc wrote:Firstly, thanks for providing a free page cache extension.

After a quick look around my test site the only problem I can see so far is the captcha. The captcha will load on first view of the Contact page but when loading the cached version it doesn't show up.

Besides this I havent noticed any other problems so far and everything is running incredibly quick!!

Good catch. I just made a small update on github to address contact/captcha:

https://github.com/budgetneon/pagecache ... b30a54cff6

You can make that one line change, or just pull down the new pagecache.php file.

Thats fixed it.

Any plans to make this multi currency, multilingual if a customer wants to switch during a session?

gtoc wrote:Thats fixed it.

Any plans to make this multi currency, multilingual if a customer wants to switch during a session?

That shouldn't be hard to do, I'll have a look at it this week.

A little feedback. This shouldn't be used with multi-store as it will mess with the stores.
Mike

Cue4cheap wrote:A little feedback. This shouldn't be used with multi-store as it will mess with the stores.
Mike

The cached files are stored and retrieved based on an md5 hash of the concatenation of $_SERVER['HTTP_HOST'] and http_build_query($_GET). I haven't tested multi-store, but my impression that the combination of those two should account for it.

I'll look into it, but if you can share what it specifically breaks, that would be appreciated.

gtoc wrote:Any plans to make this multi currency, multilingual if a customer wants to switch during a session?

This should be working now, checked in changes to github.

Each unique combination of hostname/currency/language/url should create (or respectively, retrieve) it's own cache file now.

It required initializing a session, which could have some side effects, so I would test this well before deploying to production.

budgetneon wrote:

Cue4cheap wrote:A little feedback. This shouldn't be used with multi-store as it will mess with the stores.
Mike

The cached files are stored and retrieved based on an md5 hash of the concatenation of $_SERVER['HTTP_HOST'] and http_build_query($_GET). I haven't tested multi-store, but my impression that the combination of those two should account for it.

I'll look into it, but if you can share what it specifically breaks, that would be appreciated.

At first it looks like things work but then after a bit, and I don't know what the trigger event is, all stores show a blank homepage. Once I noticed that I reversed course quickly without any debugging so I could ensure I didn't screw up anyone online wishing to purchase from my stores.

Mike

Cue4cheap wrote:

budgetneon wrote:

Cue4cheap wrote:A little feedback. This shouldn't be used with multi-store as it will mess with the stores.
Mike

The cached files are stored and retrieved based on an md5 hash of the concatenation of $_SERVER['HTTP_HOST'] and http_build_query($_GET). I haven't tested multi-store, but my impression that the combination of those two should account for it.

I'll look into it, but if you can share what it specifically breaks, that would be appreciated.

At first it looks like things work but then after a bit, and I don't know what the trigger event is, all stores show a blank homepage.

Mike

Interesting. I'll test it with a multi-store setup. Keep in mind, however, that it could be interacting with some other extension you're running. For example, if there's an ajax/json call that you didn't exempt from caching (via the skip_urls setting), that could be the issue. Especially if you left the add_comment setting at it's default...which would add an html comment to the json, making it invalid json.

There's nothing you've said that means it's specifically multi-store causing the issue. Did you do a "view source" on the "blank home page"? Or capture what was in the error log?

Once I noticed that I reversed course quickly without any debugging so I could ensure I didn't screw up anyone online wishing to purchase from my stores.

Hopefully I was clear with the "BETA" tag, and several warnings, both here, and on github to TEST in a test environment before installing in production

Pushed some changes today. See https://github.com/budgetneon/pagecache

• Allow it to run with PHP5.3 (Though I don't recommend it...5.4 has http_response_code(), which is needed for us to only cache pages with "200 OK" response codes)
• Fixed a problem with ':' in the filename, which Microsoft Windows doesn't like
• Added some sanitizing to $_SERVER['HTTP_HOST'], and a fallback to $_SERVER['SERVER_NAME'] if it isn't present (like for HTTP/1.0 clients)
• Fixed an errant double session check/creation. Wasn't hurting anything, but it wasted cycles.
• Only add the html comment to the end of the page if we can tell that it's really an HTML page (versus, for example, JSON).

My current TO-DO list:

• A simple admin interface, with at least a "CLEAR CACHE" button
• Verify that it works with multi-store. I believe this should already work, but I need to test it. I'm reasonably sure that the issues Cue4cheap saw were related to some other problem, like the html comment in JSON issue mentioned above (fixed today).
• Test with opencart 2.X. (It may already work, the changes to response.php don't seem extensive)
• Get more feedback. I'd like to upload it to the opencart extensions area, but I'll wait a bit to see if there are new issues to address

budgetneon wrote: Hopefully I was clear with the "BETA" tag, and several warnings, both here, and on github to TEST in a test environment before installing in production

No worries. I do not have my test site set up as multi-store because it is on my same webhost and it just causes problems because of how you need to set up multistores with the domains.

And I did also realize this could take my site down that is why I did it quickly with a ready replacement files to do it only a quick test.

No concerns I was willing to take the risk to speed things up.

Mike

Cue4cheap wrote:

budgetneon wrote: Hopefully I was clear with the "BETA" tag, and several warnings, both here, and on github to TEST in a test environment before installing in production

No worries. I do not have my test site set up as multi-store because it is on my same webhost and it just causes problems because of how you need to set up multistores with the domains.

And I did also realize this could take my site down that is why I did it quickly with a ready replacement files to do it only a quick test.

No concerns I was willing to take the risk to speed things up.

Mike

Ahh...good. It may work for you now. I pushed a change that makes it smarter about adding html comments, and not, for example, putting them into JSON content. But, I do still have testing a multi-store setup on my to-do list, so I suppose waiting is smarter.

Regarding Cue4Cheaps' issues, it must be something other than multi-store.

I have the pagecache running with multi-store here now. All looks fine.

http://octurbo.com/pcache/ (Main Store)
http://store1.octurbo.com/pcache/ (2nd store)
http://store2.octurbo.com/pcache/ (3rd store)

Load a page twice (to ensure it gets cached), then do a "view source" and look at the html comment at the bottom. You'll see it's caching each store's urls separately. Seems to work fine.

Not related to the page cache, but multi-store itself is a pain. You have to edit all layouts, products, categories, manufacturers, etc, manually to appear for each new store you add. If you see anything funny on the stores I added, it's due to this, and not the cache.