Community Forums

[MOD] - VirtualQMod "vQmod" Virtual File Modification System

Community created contributions & mods for OpenCart 1.x

[MOD] - VirtualQMod "vQmod" Virtual File Modification System

Postby Qphoria » Wed Dec 22, 2010 8:44 pm

VQMOD 2.0 HAS BEEN RELEASED. MOVE TO THAT THREAD:
viewtopic.php?f=23&t=40987&p=196942#p196942
This thread will be locked and retired



VERSION 1.2.3 Includes OpenCart AutoInstaller AND support for 1.5.x! (Thanks JayG)
DO NOT RUN AUTO-INSTALLER IF YOU HAVE ALREADY MANUALLY INSTALLED!
If you get Internal Server Error 500 when trying to install. Read here

I've finally followed through with an override idea and it seems to be working quite well for me in my testing.

Based on this discussion:
viewtopic.php?f=24&t=23638#p116440

I've completed the tool to allow virtually modifying files "on-the-fly" and aptly named it "VirtualQMod™" or "vQmod™" for short

How does it work?
  • VirtualQMod (vQmod) is an innovative "virtual" override system. Instead of modifying core files like we normally do, vQmod allows you to keep all changes in an external file. This file gets parsed during page load and makes the modifications to the code as it loads. Instead of modifying actual files to add custom modifications, source files are parsed "on-the-fly" before the php include() or require() is called. The source is cloned to a temp file and modifications are made to that temp file, then substituted for the real file in the include path.

Features
  • No actual code changes are made. All changes are "virtual", hence the name.
  • Modifications are stored in their own files and applied "on-the-fly" at runtime
  • Instant Single file "plug-n-play". Add the file to apply the mod, remove the file to remove the mod.
  • No worries about losing custom core changes during upgrades
  • Multiple modifications can be made to the same file without conflict
  • Easily update or enhance customizations without having to edit any code
  • Full visual of the actual changes taking place in the generated temp files for debugging
  • Fails gracefully back to the original sourcefile if there is an error
  • Exceptional logging option to track every change made
  • Only need to modify the index.php file to add the vQmod code one time.
  • Simple structured xml format. (See readme for full breakdown of xml syntax)
  • Multiple options for find/replace, regex, positions, offsets, indexing, error handling, and more!

I've attached the zip below. Installation is easy and it should work with all 1.4.x versions. Please give it a try and let me know if you have any problems.

It includes some example scripts as well that you can use right out of the box and see results. Be sure to read the readme for the full breakdown and install steps.

This is beneficial for both developers and mod users to make mods that don't have to modify files to cut down on issues and support.

In the coming days I will create a vqmod xml script generator. This will allow you to enter basic file/search/add data and generate the formatted xml so it will be even easier to make mod scripts.

License:
* @author Qphoria <Qphoria-at-gmail.com>
* @copyright Copyright (c) 2010-2011
* @license http://opensource.org/licenses/gpl-license.php GNU Public License
* @website http://www.vQmod.com

Enjoy!
Qphoria
Attachments
VirtualQMod_v1.2.3.zip
(21.56 KiB) Downloaded 1864 times
User avatar
Qphoria
Administrator
 
Posts: 19201
Joined: Mon Jul 21, 2008 2:02 pm
Donate to Qphoria

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby JAY6390 » Wed Dec 22, 2010 8:55 pm

Sounds good, not got the chance to have a proper play with this just now but will in the next week or so :)
ImageImageImage

SEO MEGA KIT PLUS - Get your site ranking higher in the search engines
Better Product SEO URL's - Perfectly structured product links
SEO URL's Route Editor - Fix all of your index.php links


Image
User avatar
JAY6390
 
Posts: 4921
Joined: Wed May 26, 2010 10:47 am
Location: United Kingdom

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby hbuchel » Thu Dec 23, 2010 11:57 am

I haven't gotten to play with this yet either, but um, just from reading, you are amazing.
hbuchel
 
Posts: 87
Joined: Mon Aug 30, 2010 8:41 am

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby Qphoria » Thu Dec 23, 2010 12:07 pm

Updated to v1.0.1 to fix problem with relative paths when calling from a subfolder like "admin"
User avatar
Qphoria
Administrator
 
Posts: 19201
Joined: Mon Jul 21, 2008 2:02 pm
Donate to Qphoria

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby mystifier » Thu Dec 23, 2010 2:03 pm

Sounds really interesting, I'll have a play and see if I can understand it when I get chance. :-\

I missed the cited thread because I didn't see the forum for a week or so.
User avatar
mystifier
 
Posts: 774
Joined: Tue May 18, 2010 4:15 am

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby Maansy » Fri Dec 24, 2010 6:44 am

Q, this is a real innovation in php world indeed.
You know that all other sniffer carts will be jumping up in here try to get a slice.
I have a feeling that VQM is only the beginning to a new generation of handling/managing modified carts.

Great work Q, and honestly I couldn't think that there is anyone would come up with a unique idea other than you.
ALL Templates :: 1.5.1+ Templates :: 50%-75% PRICE DROP ONLY at OpencartStuff.com
User avatar
Maansy
 
Posts: 928
Joined: Wed Jun 23, 2010 5:04 pm

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby qahar » Fri Dec 24, 2010 8:54 am

I've been try and this is cool. I'm sure it will very usefull for build a theme.
Just wondering, what happen if the same source file modified by two Xml on same page? (I want to test it my self but don't have an idea how to test it) Is there conflict posibility?
My product: Opencart Blog Manager Free System Information OpenCart Shortcodes
OpencartNews - News, Tutorial n Tips - Common OpenCart Errors and How to Solve Them
Don't forget to add [SOLVED] to your Thread Title (first post), if your issue is solved.
User avatar
qahar
 
Posts: 1778
Joined: Tue Jun 29, 2010 9:24 am
Location: Indonesia

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby Xsecrets » Fri Dec 24, 2010 11:28 am

Haven't had a chance to look this over real well, but just curious on the search does it only act on the first instance of the search string. For instance I noticed in one of your mods you use a search string of <?php in that particular file it will only be in there once, but many files will have several lines like that, so I was just wondering how it handled that I would guess it would act on the first one, but you know what they say when you assume something. Which led me to my next question what if you wanted to replace all occurrences of some variable or whatever in a given file how would you handle that?
Xsecrets
 
Posts: 5026
Joined: Sat Oct 24, 2009 2:51 pm
Location: FL US

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby Qphoria » Fri Dec 24, 2010 11:46 am

qahar wrote:I've been try and this is cool. I'm sure it will very usefull for build a theme.
Just wondering, what happen if the same source file modified by two Xml on same page? (I want to test it my self but don't have an idea how to test it) Is there conflict posibility?


You can modify the same source with as many files as you like. Only in the case where 2 files modify the same exact line will there be conflict.. but that would be a conflict of mods anyway. Developers can add the error handler code to determine if the entire mod should be aborted on error or just ignore the mismatch and continue on.
User avatar
Qphoria
Administrator
 
Posts: 19201
Joined: Mon Jul 21, 2008 2:02 pm
Donate to Qphoria

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby JNeuhoff » Fri Dec 24, 2010 12:22 pm

Your VirtualQMod started me off with an old attempt to solve the problem of addons which happen to modify the same Opencart PHP core file. While your approach is interesting, there may be some performance issues, also the developer has to use a somewhat cumbersome XML syntax. So I have been playing with a system which basically extends Opencart core classes and then uses overridden methods. I am using a mechanism where the require_once is modified to allow for dynamic class extensions from an 'vmod' directory.
J.Neuhoff - MHC Web Design

Override Engine (V6.3) for OpenCart 1.5.x.x and Override Engine (V6.4) for OpenCart 2.0
allowing extensions to override and modify core methods, language files and templates (see also FAQ)

VQMod for OpenCart 2.0
a dynamic modification system for core files, controlled by XML-specifications
User avatar
JNeuhoff
 
Posts: 2593
Joined: Tue Dec 04, 2007 2:38 pm

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby Qphoria » Fri Dec 24, 2010 1:10 pm

JNeuhoff wrote:Your VirtualQMod started me off with an old attempt to solve the problem of addons which happen to modify the same Opencart PHP core file.

That is exactly what it allows now. This works perfect for adding custom menu items to the admin menu now without having to worry about someone elses menu edits replacing yours. After years of working on this there is finally a solution.

JNeuhoff wrote:also the developer has to use a somewhat cumbersome XML syntax


It's not really that cumbersome, as the only real lines you need to change is the <search> and <add> lines. The rest is just the wrapper. As I mentioned above, I am making a form for the lazy to generate the xml for them. I pondered using a simpler format, but I think a proper structure is the only way to enforce good mod and vqmod making. I have been taking some of the quickmods I've made on the forum here (1. EDIT.xxxx, 2: FIND: xxxx, 3: REPLACE :xxx ... etc) and those are simple copy paste in 3 steps. It actually takes more time to create all the steps in the forum post than it does to create the vqmod xml file.

JNeuhoff wrote:While your approach is interesting, there may be some performance issues,


Performance concerns are valid based on the principle, and they were the main thing on my mind when creating this. But I've taken major strides to improve any performance loss and I've got about 10 mods running on my demo store from 10 different files. I've added a page time to the footer and I have seen absolutely no difference between running the 10 mods or not.

The code itself is quite intelligent (modesty aside), as the __construct first scans all the xml files and creates a list array of the <file> tags that are changing. Then only those source files actually need to be processed as opposed to all the files which cuts down on about 50+ additional filesystem reads in opencart. Additionally I create an array for which source files are modified by which mod, so even when there is a file that has a mod, it only has to look in the correct mod files. Realistically its only reading and writing one additional file per mod, which is similar to the way the database cache files work and those actually help to improve performance, so there appears to be no real performance issue here so far.

Time will tell for sure, but so far there has been no noticeable hit. Also take a look at the xml examples and you will see how very little is really needed to make a mod.

The important thing is the concept of not having to modify the core files ever again. This means less versioning for developers and less worry about upgrade for store owners. That is worth the extra 10 seconds it takes to make a vqmod file. I've already started adding this to my own copy of phpbb to see how it works there. The design of this is to work with any script or platform easily while modifying very little of the platform's source.

I am using a mechanism where the require_once is modified to allow for dynamic class extensions from an 'vmod' directory.


At one point I did add some code that I wanted to use with the php PECL/APD package for "override_function" and thus override the include/requires natively without having to edit anything. But almost no webhost has that package installed.
User avatar
Qphoria
Administrator
 
Posts: 19201
Joined: Mon Jul 21, 2008 2:02 pm
Donate to Qphoria

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby xds » Sat Dec 25, 2010 11:55 pm

This is just brilliant and I would have to agree is far better than placing hooks all throughout the core. I see absolutely no reason why this can't solve everyones gripes about "this should do this instead".

A very practical application would be everyones gripes about multi-store having different functionality for each store which I have seen mentioned in damn near every post related to future updates. I myself have been trying to decide if I want to use the built in multi-store or setting up individual stores with unique db table prefixes for each install and simply removing or hardcoding prefixes to shared tables (users / admins / orders etc) but now I think we have a great way to overcome.

Well done, well done indeed.
Thanks for 1.5!
xds
 
Posts: 40
Joined: Sun Jul 11, 2010 8:53 pm

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby SapporoGuy » Sun Dec 26, 2010 2:17 am

You mentioned trying it out on phpBB, how did it go / how is it going?
930sc ... because it is fun!
User avatar
SapporoGuy
 
Posts: 1132
Joined: Mon Nov 01, 2010 6:29 am

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby Qphoria » Sun Dec 26, 2010 2:20 am

as the brits would say... "It's bloody Christmas!"
User avatar
Qphoria
Administrator
 
Posts: 19201
Joined: Mon Jul 21, 2008 2:02 pm
Donate to Qphoria

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby Qphoria » Sun Dec 26, 2010 2:03 pm

Ok i've started using it with phpbb.. have some paths to work out as phpbb uses variables for path names so I have to figure out how to handle those better but I have a small work around for it for now. Phpbb is a lot more complex with its design so there are more spots to handle but for the main common file I am able to add code from a vqmod file with no problems, so it works.

I think I will make its own website for the utility and create vqmod startups for each major php project (opencart, phpbb, smf, wordpress, etc). I'm going to try it on Website Baker next, since that is what I use for CMS
User avatar
Qphoria
Administrator
 
Posts: 19201
Joined: Mon Jul 21, 2008 2:02 pm
Donate to Qphoria

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby Qphoria » Mon Dec 27, 2010 12:33 pm

Xsecrets wrote:Haven't had a chance to look this over real well, but just curious on the search does it only act on the first instance of the search string. For instance I noticed in one of your mods you use a search string of <?php in that particular file it will only be in there once, but many files will have several lines like that, so I was just wondering how it handled that I would guess it would act on the first one, but you know what they say when you assume something. Which led me to my next question what if you wanted to replace all occurrences of some variable or whatever in a given file how would you handle that?


Seems I overlooked this question...
At this time it finds them all so it is important to find very specific strings. "<?php" maybe not be the best example, but in that file it works ok. I am going to add the ability to limit the number and maybe index them so you can set it only to change the 3rd one in the file.
User avatar
Qphoria
Administrator
 
Posts: 19201
Joined: Mon Jul 21, 2008 2:02 pm
Donate to Qphoria

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby Xsecrets » Mon Dec 27, 2010 2:58 pm

Qphoria wrote:
Xsecrets wrote:Haven't had a chance to look this over real well, but just curious on the search does it only act on the first instance of the search string. For instance I noticed in one of your mods you use a search string of <?php in that particular file it will only be in there once, but many files will have several lines like that, so I was just wondering how it handled that I would guess it would act on the first one, but you know what they say when you assume something. Which led me to my next question what if you wanted to replace all occurrences of some variable or whatever in a given file how would you handle that?


Seems I overlooked this question...
At this time it finds them all so it is important to find very specific strings. "<?php" maybe not be the best example, but in that file it works ok. I am going to add the ability to limit the number and maybe index them so you can set it only to change the 3rd one in the file.

ok that is very good to know.
Xsecrets
 
Posts: 5026
Joined: Sat Oct 24, 2009 2:51 pm
Location: FL US

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby xds » Mon Dec 27, 2010 9:47 pm

Index would probably be the most flexible way of handling such occurrences.
Thanks for 1.5!
xds
 
Posts: 40
Joined: Sun Jul 11, 2010 8:53 pm

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby Qphoria » Tue Dec 28, 2010 5:53 pm

UPDATE COMING SOON
Currently the code uses the server's "/tmp" directory to create the modified files. There is also the option of enabling debug which will create copies of those files to a "vqdbg" folder. Instead of all this, I think it would just be best to create these files in the "vqdbg" folder directly.. and rename that folder to "vqcache" or something.

This will give us 2 new features:
- The ability to always see the changed files without having to write and copy for debug.
- The ability to add a new "use cached" feature which will allow you to use the pre-modded versions of the files once they have been created once. Similar to the way images are resized in opencart. They are generated once, and then reused if they already exist. This will remove any worries of performance hit from constantly re-writing the files on every page load, which was already deemed to be a non-issue, making this tool the best thing since the internet was invented. But modesty prevents me from saying that :clown:

I'm also going to add a debug logging option to better trace what is happening as it goes to make it even easier to find mod issues. None of these changes will break any current xml mods. Stay tuned!
User avatar
Qphoria
Administrator
 
Posts: 19201
Joined: Mon Jul 21, 2008 2:02 pm
Donate to Qphoria

Re: [MOD] - VirtualQMod - Modify on the fly!

Postby JNeuhoff » Wed Dec 29, 2010 9:40 am

I played around more with your VirtualQMod and have now come to the conclusion that it is quite useful. Will test some more of my Opencart addons with it. My addons tend to modify Opencart core files, so this will be a useful test.
J.Neuhoff - MHC Web Design

Override Engine (V6.3) for OpenCart 1.5.x.x and Override Engine (V6.4) for OpenCart 2.0
allowing extensions to override and modify core methods, language files and templates (see also FAQ)

VQMod for OpenCart 2.0
a dynamic modification system for core files, controlled by XML-specifications
User avatar
JNeuhoff
 
Posts: 2593
Joined: Tue Dec 04, 2007 2:38 pm

Next

Return to Free Contributions

Who is online

Users browsing this forum: Xciso and 26 guests

Hosted by Arvixe Web Hosting