Under Development: Override system

One of the better features of zen-cart was the use of an override folder to avoid making any changes to the core files, and making adding and removing contribs very easy since you'd just remove the override file, rather than having to edit the core file to add the new code.

From the contribs I've looked at tho, there doesn't seem to be any type of override system in place. Instead, the contribs use modified core files that just get overwritten. Which means to back out any contribs, you would need to reupload the official ones. This also means that if you have multiple contribs for the same page, you are stuck manually merging them into your core file which can get messy for those that don't know php very well.

I haven't seen any kind of override system planned, so I will attempt it and see if it is something that others agree with. The overall design shouldn't be too hard. Zen-cart did it by putting the overrides into a subfolder within the root location that matched the name of the template folder:

includes/templates/default/header.php
was overridden by
includes/templates/mytemplate123/header.php

includes/languages/english.php
was overridden by
includes/languages/mytemplate123/english.php

This worked pretty good. You still run into the problem when you have 2 contribs modifying the same file, like 2 contribs that modify the header.php file. One adds some advanced search options, the other adds a random logo generator. But that may or may not be avoidable.

If there was some way that it could all be modular at an even lower level than the template name, it would be near perfect for adding multiple contribs to one section. Some kind of class extends system to each file that would load the default, but override the default with the new code where the same variable name is used.

But for now, the first step could be a template folder name override, and that is what I will be working on here.

Thoughts?

Sounds good one on on one...But how do you see the "override" function working with multiple "add-ons" affecting the same core codes?

Well... perhaps an example could be

file product.php would be expanded with the files inside the subfolder named product_override.

product.php has: product_contrib1.php changes the heading_title to something else, so it contains: and would only override the heading_title part, leaving the rest of the product.php file to load normally, unless there were more override files.

product_contrib2.php changes the product_description to add wrapper tags to it: this would override the description part of the product.php file.

So in the end the product.php file would load normally, but would have a new value for the heading_title and a new value for the description.


Something like that, tho the logisitics haven't really been worked out. This was just some half-baked idea that I came up with off the top of my head

At the very least, the template-override method would at least be a starting point.

Ok, I'm close:

In the controller.php file under
function execute($action)

I changed:

$file = $this->directory . basename($action['class']) . '.php';

to:

$template = $this->locator->get('template');
$file = $this->directory . $template->data[template] . SLASH . basename($action['class']) . '.php';
if (!file_exists($file)) {
    $file = $this->directory . basename($action['class']) . '.php';
}

This will force it to check for an override version of the file in a folder that uses the template name.

It works, but gives an error:
Warning: Cannot modify header information - headers already sent by (output started at C:\Documents and Settings\Server\Application Data\NuSphere\PhpED\projects\opencart\library\application\controller.php:70) in C:\Documents and Settings\Server\Application Data\NuSphere\PhpED\projects\opencart\library\cart\currency.php on line 43

Not sure why referencing locator would cause header output.

Also had to add the SLASH constant define to the index.php page as it was only defined in the install page, but it's needed now for the override system.

This should allow us to make changes to custom controller files without editing the core files. To go back to core, simply rename or delete the file under the template subfolder.

This will also need to be done under the extension folder as well.

Qphoria,

I get that error whenever I edit a .php/.tpl file using Dreamweaver.
To fix the error, go back and check C:\Documents and Settings\Server\Applicationdata\NuSphere\PhpED\projects\opencart\library\application\controller.php and make sure there are no extra lines that are empty at the the end of the file, if there are, delete those lines.

Regards,
Troy

Yea, that is the usual suspect and I checked that first but there aren't any this time. It seems to only happen when I add this:

$template = $this->locator->get('template');

code. Not sure if maybe locator is setting some header and I'm calling it in the wrong place or what.
Might just have to make a global for $gbltemplate or use another method of getting the template name.

thx for the help anyway

Ok, I got it:

I had to reference the directory name instead of the data array template name
So now, the override looks for a subfolder with the same name as the template folder, which should match the name of the template anyway:
Going to add this to the extension stuff too and that should get the overrides working for all controlled and extensioned code.

Ok,

I added it to:

• library/application/controller.php - To allow main controller overrides
• library/cart/module.php - To allow extension overrides

The next step will be to override template files, but relies on a structure change. Here's the idea:

Instead of having to clone the entire 'default' folder to make a new template, we can set the default to be the fallback, if and only if the custom template doesn't have an override file.

Example:

default folder has:
/content/*.*
/css/*.*
/image/*.*
/module/*.*
/layout.tpl

mytemplate folder could have:
/css/*.*
/image/*.*

Since I don't have any custom modules or content folders, it would default to the 'default' template for those files, while using my custom images and css if mytemplate is the current template. It would also be on a file-by-file basis, not folder-by-folder. So you could just change 2 images in your custom template and it will use those 2 plus all the default ones that aren't overridden.

This allows:
A) Overrides without affecting core files
B) Less disk space since you won't have to have multiple copies of the same file everywhere
C) Easier template creation
D) Backup support, in case you forgot to add a file to a template, at least the default one will take over and not leave the store owner with a broken page.
E) Some Transparency with new OpenCart versions

Thoughts?

Well there seems to be a lot of hardcoded '/' slashes

like: $language->load('extension/module/cart.php');

Oddly enough, they seem to load ok on my windows server so maybe its not worth looking into. A function to auto-convert slashes so it doesn't matter what slash you use, as it could auto-convert it for the OS type before evaluating the file existence.

I am also trying to figure the best way to check the override. I could edit each file wherever I see things like
and change it to
or it could be manipulated at the function level to always prepend the template name to the filename, and if not exist, revert to default. It would be less files to edit.. There should probably be some function that does the check on the current directory to check for a subdirectory matching the template name and check that the override file exists to cut down on code repeat.

Ok

Been working on it quite a bit, and currently have the following overriding:

• Controllers
• Extensions
• Template files

There is currently one exception at this time with the template stuff, due to the existing structure of the css path in the layout.tpl. It already includes the main path + the template name as part of the path. I will have to remove the assumed path and leave only the css tag, and move the full css path creation back to the files so that I can place logic on whether or not there is an override for each css file.

This is likely the case with javascript stuff too, tho currently it is at a higher level, but there should also likely be a per-template javascript, and the existing javascript folder should be moved into the default template folder. Different templates may have different javascript based contribs.

I've been using Nusphere PHPed to debug the code real time and that is making things much easier to work with, as well as ensure I don't overlook any files that are being loaded. 

Well I am pretty close to being complete with the override code. The last few steps is to add overrides for CSS and image files, content and module template overrides seem to work swimmingly.

I've even got a few of the released contribs working with the overrides. There are no code changes to the contribs, just a different directory structure so that it uses the new file as an override instead of a replacement.

The last step will be to figure out how to "extend" things like new menu options without replacing the complete menu file.

Following this "template-based" override, I am going to delve more addon plugins and variable overrides.

I am hoping that this will be something that Daniel might want to add to the core (maybe 1.0 ) so that it better defines a contrib standard.

I have been following your thread about the override system with interest.

Recently we had the following problem which illustrates the need for a proper override system:

A user tried to install the Export/Import module as well as the Filtering module. These 2 modules are incompatible because they both attempt to update/replace the following files:

        /admin/extension/module/menu.php
        /admin/language/english/extension/module/menu.php
        /admin/template/defaultt/module/menu.tpl

Would your override system be able to cope with such a scenario?

JNeuhoff wrote: I have been following your thread about the override system with interest.

Recently we had the following problem which illustrates the need for a proper override system:

A user tried to install the Export/Import module as well as the Filtering module. These 2 modules are incompatible because they both attempt to update/replace the following files:

        /admin/extension/module/menu.php
        /admin/language/english/extension/module/menu.php
        /admin/template/defaultt/module/menu.tpl

Would your override system be able to cope with such a scenario?

Absolutely! This is exactly the reason behind both the DB-Based Admin Menu & the Override System.

The problem above would actually be resolved with the db-based menu system. Instead of cloning the menu files and adding your one line change and putting them back up, you'd just have a simple sql statement to insert a new menu.

I have actually already been converting many 3rd party contribs, including the export/import contrib, the Generate URL Alias contrib, and the filtering module to use my admin menu. I have it working on my production store as well as my demo. I've completely removed the need for those 3 files and the menu is completely dynamic.

You can see on my demo I have installed 4 different contribs that all require changes to the menu files. All done with 1 single sql insert each:
- REPORT   ->Search History
- ADMIN     ->Generate URL Alias
- CATALOG ->Import/Export
- CATALOG ->Details (Filter module)

While this menu works perfectly, I did actually post a reply to you in the admin menu thread about your menu db structure. Currently I'm using a simple one level style and pulled the nested menus out to the top level. It might eventually be better to make one that can do nesting eventually. But if we can first get something to replace the original one, that is the first step.

Hi Qphoria,

Now that you have the category menu stuff working with the tree control, (mentioned here for interested readers), you could use a similar structure in the database for storing the menus and use a similar mechanism for reading them out and creating the menu tree.

Well, I don't really need to worry about sort in the admin menu, so the Tree class won't be necessary. But I'm thinking I'm more confused on the structure of the db for menu-to-menu vs the category table. I think maybe if I got the data out properly into a workable format it wont take much.. but i'm probably overthinking it. Tomorrow I'll have a clear head now that Subcategory sorting works (thx again for that)

My pleasure 

Remember that a menu is a tree and that structure and order are both important.

Consider your generic approach to adding new application extensions to the menu. Whatever you provide for a single extension installation will not be able to correctly decide the exact location in the existing (possibly already customised) menu structure. Hence you will have to put the new entry into a known location in the core menu hierarchy and provide a mechanism for managing the menu definitions and locations so that the installer (person) can move the new menu item to the location of their choice.

Yea, I suppose sort order could be useful, tho it would be slightly different based on whether or not 2 contrib makers want their menu to be at order 4 under the Admin heading. Then it would come down to alpha or some secondary order, which is why i figured it didn't really matter. But it couldn't hurt to add support for it anyway.

And after thinking about it, you are right.. really the admin menu is just a "category" menu for the backend. I'll have to think about that one. 

Thx!

Back on topic with the override system.. I'd like to discuss some ideas for the Phase II part of the override.

Phase I: (About 80% done)
- Override by exact match filename in an override folder.
- Override folder name based on current template.
- Every folder has override check

How does it work:
Simple!
if "folder//file.ext" exists then use that. Otherwise, use only "folder/file.ext". Thats it.

Pros:
- No longer need to touch the core files, all contribs can be "drop in" instead of doing manual edits.
- Removing contribs is as simple as removing the file in the override folder.
- Multiple templates means you can have different overrides possible per template.
- Override all file types, css, images, etc.
- Additional templates can be created with only the changed files, as it will default back to "default" for the core files. (this requires a change in the mind set of how the templates work. default should be untouched, and a new "basic" template should be created as the original template).

Cons:
- Can only override on a per-file basis. Multiple changes to the same file limitations. (e.g. If one changes product.tpl to add new div block, and another one needs to change product.tpl to add new price block, then there is conflict)



Phase II: (Still in the brainstorming stage)
Some sort of variable-level override system where at the end of the controller/extension file it does a call for any subfiles with the name as the prefix or in a matching subfolder. Inside those files will contain "only" the additional values or replacement values of existing variables.
Example:
- Contrib A wants to change the product page to change the way the options are retrieved
- Contrib B wants to change the product page to replace the current product description with a tab-based structure.
- Neither of these contribs would edit any existing code. Instead they would create files in a subfolder called "product"

1. Contrib A would then create a file named product_contrib_a.php
2. His file would only contain a clone of the existing options code, with his changes:
3. The product.php file will have to know to always check its "subfolder" for additional files at the end of its normal load process. In this case, the existing option_data array would have been reset to a different structure before displaying. Thus successfully overriding the options without having to edit the core file or affect anyone else's changes



Now Contrib B will do the same thing:

1. Contrib B would then create a file named product_contrib_b.php
2. His file would have some new code that takes the product description and puts it on a tab, takes some other modules and puts them on a tab, and then would put them into "description" variable so that when the page loads, it uses the new stuff inside
3. The product.php file will check its subfolder for additional files at the end of its normal load process and in this case, the existing description would have been reset to a different description base before displaying. Thus successfully overriding the description without having to edit the core file or affect the Contrib A's changes.

Make sense? It's still early in the thought cycle, but it would remove the con from the template override.

Either way, I think we need to get the template override system in place first so that we can start making proper drop-in contribs, then we can go from there to more advanced ideas.

I am still struggling to understand how phase I is supposed to work:

E.g. if multiple OpenCart extensions from the contributions try to change the same *.tpl or *.css files, e.g. default.css, and even worse, they try to change the same CSS-property or the same HTML-element, how will this be handled?

As regards phase II:

Again things can become quite tricky. In your example

- Contrib A wants to change the product page to change the way the options are retrieved
- Contrib B wants to change the product page to replace the current product description with a tab-based structure.
- Neither of these contribs would edit any existing code. Instead they would create files in a subfolder called "product"

it may also be possible that Contrib A wants to change variable $xyz in product.php. Now comes Contrib B and tries to change the same variable $xyz for its own purposes. Even if each contrib changes a different variable, the order in which variables are changed may be of importance.

I think it really boils down to implementing some proper code-merging: In your example, we are basically dealing with 3 different flavors of file 'product.php': The original one, the version for Contrib A, and the version for Contrib B.

Suppose we are dealing with the admin/product.php, now in 3 flavors. The original one has methods like index, insert, update, delete, validateForm, etc. If  Contrib-A wants some changes in admin/product.php for method insert, it would just extend the original product class and override the method insert. In this overriden method it has to make sure it calls the inherited parent method as needed. Now Contrib-B has the notion to change method insert, too, for its own version of admin/product.php. It should now extend the product-class from Contrib-A again, and do its own additional stuff in the overriden method insert. So far, so good. It's not quite traditional object-oriented programming, because Contrib-A is not aware of Contrib-B, and vice versa. So this business of overriding an existing method has to be handled more dynamically, and all flavors of the overriden method are to be called in the correct order, starting with the one from the most recent contribution (Contrib B), then the second-last contribution flavor (Contrib B), finally the original flavor (OpenCart).

These are just some quicks thoughts, nothing written in stone.

Perhaps a dynamic database-driven override system would make sense and could be integrated in the PHP class loader. Not sure whether PHP has APIs for this sort of thing.

JNeuhoff wrote: I am still struggling to understand how phase I is supposed to work:

E.g. if multiple OpenCart extensions from the contributions try to change the same *.tpl or *.css files, e.g. default.css, and even worse, they try to change the same CSS-property or the same HTML-element, how will this be handled?

Qphoria wrote: Cons:
- Can only override on a per-file basis. Multiple changes to the same file limitations. (e.g. If one changes product.tpl to add new div block, and another one needs to change product.tpl to add new price block, then there is conflict)

However, the nice thing about CSS is that it is already self overriding based on load order.
if default.css has a .body { background: #FF0000; }
and newbody.css is lower in the loading than the first one with .body { background: #CCCCCC; }, it will replace it.

Currently, however, the method for loading css in OpenCart is to hardcode specify the css in the tpl file. I vote that we remove that and implement an automatic css loading schema. Any file in the css folder will load. We can make it a bit more specific by prefixing the name for certain pages only based on their controller name. For example.

All files that start with "style_xxx.css" will always load
All files that start with "product_xxx.css" will only load on the product page
All files that start with "checkout_xxx.css" will only load on the checkout page.
Etc, etc.

This way, each contrib can have its own stylesheet name based automatically on the controller file name and not have to worry about merging with others. It will of course be dynamic, so if you create a new controller called "export.php" it will automatically check for any export_xxx.css files when that page loads.

JNeuhoff wrote: As regards phase II:
it may also be possible that Contrib A wants to change variable $xyz in product.php. Now comes Contrib B and tries to change the same variable $xyz for its own purposes. Even if each contrib changes a different variable, the order in which variables are changed may be of importance.

This is just a fact of science. "Two objects cannot occupy the same space". If they are replacing the same variable, it is likely that they are doing two conflicting things anyway.

JNeuhoff wrote: I think it really boils down to implementing some proper code-merging: In your example, we are basically dealing with 3 different flavors of file 'product.php': The original one, the version for Contrib A, and the version for Contrib B.

Suppose we are dealing with the admin/product.php, now in 3 flavors. The original one has methods like index, insert, update, delete, validateForm, etc. If  Contrib-A wants some changes in admin/product.php for method insert, it would just extend the original product class and override the method insert. In this overriden method it has to make sure it calls the inherited parent method as needed. Now Contrib-B has the notion to change method insert, too, for its own version of admin/product.php. It should now extend the product-class from Contrib-A again, and do its own additional stuff in the overriden method insert. So far, so good. It's not quite traditional object-oriented programming, because Contrib-A is not aware of Contrib-B, and vice versa. So this business of overriding an existing method has to be handled more dynamically, and all flavors of the overriden method are to be called in the correct order, starting with the one from the most recent contribution (Contrib B), then the second-last contribution flavor (Contrib B), finally the original flavor (OpenCart).

These are just some quicks thoughts, nothing written in stone.

Perhaps a dynamic database-driven override system would make sense and could be integrated in the PHP class loader. Not sure whether PHP has APIs for this sort of thing.

That is actually what i thought the extensions did already when i first found OpenCart. I thought the Extensions would subclass and override the main class at the function level. It is similar to Unreal Engine's "mutator" override. And each "mutator" would also call other mutators to run one after the other to include them all. So ya that would be cool to do. I am open to any suggestions. I too have only brainstormed some thoughts on paper and haven't followed the full logistics all the way through. I'm just trying to get a committee together to see if we can work as a community to hash something like this out.