Post by Qphoria » Thu Dec 23, 2010 9:44 am

VQMOD 2.0 HAS BEEN RELEASED. MOVE TO THAT THREAD:
http://forum.opencart.com/viewtopic.php ... 42#p196942
This thread will be locked and retired



VERSION 1.2.3 Includes OpenCart AutoInstaller AND support for 1.5.x! (Thanks JayG)
DO NOT RUN AUTO-INSTALLER IF YOU HAVE ALREADY MANUALLY INSTALLED!
If you get Internal Server Error 500 when trying to install. Read here

I've finally followed through with an override idea and it seems to be working quite well for me in my testing.

Based on this discussion:
http://forum.opencart.com/viewtopic.php ... 38#p116440

I've completed the tool to allow virtually modifying files "on-the-fly" and aptly named it "VirtualQMod™" or "vQmod™" for short

How does it work?
  • VirtualQMod (vQmod) is an innovative "virtual" override system. Instead of modifying core files like we normally do, vQmod allows you to keep all changes in an external file. This file gets parsed during page load and makes the modifications to the code as it loads. Instead of modifying actual files to add custom modifications, source files are parsed "on-the-fly" before the php include() or require() is called. The source is cloned to a temp file and modifications are made to that temp file, then substituted for the real file in the include path.
Features
  • No actual code changes are made. All changes are "virtual", hence the name.
  • Modifications are stored in their own files and applied "on-the-fly" at runtime
  • Instant Single file "plug-n-play". Add the file to apply the mod, remove the file to remove the mod.
  • No worries about losing custom core changes during upgrades
  • Multiple modifications can be made to the same file without conflict
  • Easily update or enhance customizations without having to edit any code
  • Full visual of the actual changes taking place in the generated temp files for debugging
  • Fails gracefully back to the original sourcefile if there is an error
  • Exceptional logging option to track every change made
  • Only need to modify the index.php file to add the vQmod code one time.
  • Simple structured xml format. (See readme for full breakdown of xml syntax)
  • Multiple options for find/replace, regex, positions, offsets, indexing, error handling, and more!
I've attached the zip below. Installation is easy and it should work with all 1.4.x versions. Please give it a try and let me know if you have any problems.

It includes some example scripts as well that you can use right out of the box and see results. Be sure to read the readme for the full breakdown and install steps.

This is beneficial for both developers and mod users to make mods that don't have to modify files to cut down on issues and support.

In the coming days I will create a vqmod xml script generator. This will allow you to enter basic file/search/add data and generate the formatted xml so it will be even easier to make mod scripts.

License:
* @author Qphoria <Qphoria-at-gmail.com>
* @copyright Copyright (c) 2010-2011
* @license http://opensource.org/licenses/gpl-license.php GNU Public License
* @website http://www.vQmod.com

Enjoy!
Qphoria[/s]

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by JAY6390 » Thu Dec 23, 2010 9:55 am

Sounds good, not got the chance to have a proper play with this just now but will in the next week or so :)

Image


User avatar
Guru Member

Posts

Joined
Wed May 26, 2010 11:47 pm
Location - United Kingdom

Post by hbuchel » Fri Dec 24, 2010 12:57 am

I haven't gotten to play with this yet either, but um, just from reading, you are amazing.

New member

Posts

Joined
Mon Aug 30, 2010 9:41 pm

Post by Qphoria » Fri Dec 24, 2010 1:07 am

Updated to v1.0.1 to fix problem with relative paths when calling from a subfolder like "admin"

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by mystifier » Fri Dec 24, 2010 3:03 am

Sounds really interesting, I'll have a play and see if I can understand it when I get chance. :-\

I missed the cited thread because I didn't see the forum for a week or so.

Free v1.4.9 Extensions: Default Specials | Improved Search | Customer Activity Report | Customer Groups | Royal Mail With Handling | Improved Product Page | Random Products | Stock Report | All Products


User avatar
Active Member

Posts

Joined
Tue May 18, 2010 5:15 pm

Post by Maansy » Fri Dec 24, 2010 7:44 pm

Q, this is a real innovation in php world indeed.
You know that all other sniffer carts will be jumping up in here try to get a slice.
I have a feeling that VQM is only the beginning to a new generation of handling/managing modified carts.

Great work Q, and honestly I couldn't think that there is anyone would come up with a unique idea other than you.

ALL Templates :: 1.5.1+ Templates :: 50%-75% PRICE DROP ONLY at OpencartStuff.com


User avatar
Active Member

Posts

Joined
Thu Jun 24, 2010 6:04 am


Post by qahar » Fri Dec 24, 2010 9:54 pm

I've been try and this is cool. I'm sure it will very usefull for build a theme.
Just wondering, what happen if the same source file modified by two Xml on same page? (I want to test it my self but don't have an idea how to test it) Is there conflict posibility?

User avatar
Expert Member

Posts

Joined
Tue Jun 29, 2010 10:24 pm
Location - Indonesia

Post by Xsecrets » Sat Dec 25, 2010 12:28 am

Haven't had a chance to look this over real well, but just curious on the search does it only act on the first instance of the search string. For instance I noticed in one of your mods you use a search string of <?php in that particular file it will only be in there once, but many files will have several lines like that, so I was just wondering how it handled that I would guess it would act on the first one, but you know what they say when you assume something. Which led me to my next question what if you wanted to replace all occurrences of some variable or whatever in a given file how would you handle that?

OpenCart commercial mods and development http://spotonsolutions.net
Layered Navigation
Shipment Tracking
Vehicle Year/Make/Model Filter


Guru Member

Posts

Joined
Sun Oct 25, 2009 3:51 am
Location - FL US

Post by Qphoria » Sat Dec 25, 2010 12:46 am

qahar wrote:I've been try and this is cool. I'm sure it will very usefull for build a theme.
Just wondering, what happen if the same source file modified by two Xml on same page? (I want to test it my self but don't have an idea how to test it) Is there conflict posibility?
You can modify the same source with as many files as you like. Only in the case where 2 files modify the same exact line will there be conflict.. but that would be a conflict of mods anyway. Developers can add the error handler code to determine if the entire mod should be aborted on error or just ignore the mismatch and continue on.

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by JNeuhoff » Sat Dec 25, 2010 1:22 am

Your VirtualQMod started me off with an old attempt to solve the problem of addons which happen to modify the same Opencart PHP core file. While your approach is interesting, there may be some performance issues, also the developer has to use a somewhat cumbersome XML syntax. So I have been playing with a system which basically extends Opencart core classes and then uses overridden methods. I am using a mechanism where the require_once is modified to allow for dynamic class extensions from an 'vmod' directory.

Export/Import Tool * SpamBot Buster * Unused Images Manager * Instant Option Price Calculator * Number Option * Google Tag Manager * Survey Plus * OpenTwig


User avatar
Guru Member
Online

Posts

Joined
Wed Dec 05, 2007 3:38 am


Post by Qphoria » Sat Dec 25, 2010 2:10 am

JNeuhoff wrote:Your VirtualQMod started me off with an old attempt to solve the problem of addons which happen to modify the same Opencart PHP core file.
That is exactly what it allows now. This works perfect for adding custom menu items to the admin menu now without having to worry about someone elses menu edits replacing yours. After years of working on this there is finally a solution.
JNeuhoff wrote: also the developer has to use a somewhat cumbersome XML syntax
It's not really that cumbersome, as the only real lines you need to change is the <search> and <add> lines. The rest is just the wrapper. As I mentioned above, I am making a form for the lazy to generate the xml for them. I pondered using a simpler format, but I think a proper structure is the only way to enforce good mod and vqmod making. I have been taking some of the quickmods I've made on the forum here (1. EDIT.xxxx, 2: FIND: xxxx, 3: REPLACE :xxx ... etc) and those are simple copy paste in 3 steps. It actually takes more time to create all the steps in the forum post than it does to create the vqmod xml file.
JNeuhoff wrote: While your approach is interesting, there may be some performance issues,
Performance concerns are valid based on the principle, and they were the main thing on my mind when creating this. But I've taken major strides to improve any performance loss and I've got about 10 mods running on my demo store from 10 different files. I've added a page time to the footer and I have seen absolutely no difference between running the 10 mods or not.

The code itself is quite intelligent (modesty aside), as the __construct first scans all the xml files and creates a list array of the <file> tags that are changing. Then only those source files actually need to be processed as opposed to all the files which cuts down on about 50+ additional filesystem reads in opencart. Additionally I create an array for which source files are modified by which mod, so even when there is a file that has a mod, it only has to look in the correct mod files. Realistically its only reading and writing one additional file per mod, which is similar to the way the database cache files work and those actually help to improve performance, so there appears to be no real performance issue here so far.

Time will tell for sure, but so far there has been no noticeable hit. Also take a look at the xml examples and you will see how very little is really needed to make a mod.

The important thing is the concept of not having to modify the core files ever again. This means less versioning for developers and less worry about upgrade for store owners. That is worth the extra 10 seconds it takes to make a vqmod file. I've already started adding this to my own copy of phpbb to see how it works there. The design of this is to work with any script or platform easily while modifying very little of the platform's source.
I am using a mechanism where the require_once is modified to allow for dynamic class extensions from an 'vmod' directory.
At one point I did add some code that I wanted to use with the php PECL/APD package for "override_function" and thus override the include/requires natively without having to edit anything. But almost no webhost has that package installed.

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by xds » Sun Dec 26, 2010 12:55 pm

This is just brilliant and I would have to agree is far better than placing hooks all throughout the core. I see absolutely no reason why this can't solve everyones gripes about "this should do this instead".

A very practical application would be everyones gripes about multi-store having different functionality for each store which I have seen mentioned in damn near every post related to future updates. I myself have been trying to decide if I want to use the built in multi-store or setting up individual stores with unique db table prefixes for each install and simply removing or hardcoding prefixes to shared tables (users / admins / orders etc) but now I think we have a great way to overcome.

Well done, well done indeed.

Thanks for 1.5!


xds
New member

Posts

Joined
Mon Jul 12, 2010 9:53 am

Post by SapporoGuy » Sun Dec 26, 2010 3:17 pm

You mentioned trying it out on phpBB, how did it go / how is it going?

930sc ... because it is fun!


User avatar
Active Member

Posts

Joined
Mon Nov 01, 2010 7:29 pm

Post by Qphoria » Sun Dec 26, 2010 3:20 pm

as the brits would say... "It's bloody Christmas!"

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by Qphoria » Mon Dec 27, 2010 3:03 am

Ok i've started using it with phpbb.. have some paths to work out as phpbb uses variables for path names so I have to figure out how to handle those better but I have a small work around for it for now. Phpbb is a lot more complex with its design so there are more spots to handle but for the main common file I am able to add code from a vqmod file with no problems, so it works.

I think I will make its own website for the utility and create vqmod startups for each major php project (opencart, phpbb, smf, wordpress, etc). I'm going to try it on Website Baker next, since that is what I use for CMS

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by Qphoria » Tue Dec 28, 2010 1:33 am

Xsecrets wrote:Haven't had a chance to look this over real well, but just curious on the search does it only act on the first instance of the search string. For instance I noticed in one of your mods you use a search string of <?php in that particular file it will only be in there once, but many files will have several lines like that, so I was just wondering how it handled that I would guess it would act on the first one, but you know what they say when you assume something. Which led me to my next question what if you wanted to replace all occurrences of some variable or whatever in a given file how would you handle that?
Seems I overlooked this question...
At this time it finds them all so it is important to find very specific strings. "<?php" maybe not be the best example, but in that file it works ok. I am going to add the ability to limit the number and maybe index them so you can set it only to change the 3rd one in the file.

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by Xsecrets » Tue Dec 28, 2010 3:58 am

Qphoria wrote:
Xsecrets wrote:Haven't had a chance to look this over real well, but just curious on the search does it only act on the first instance of the search string. For instance I noticed in one of your mods you use a search string of <?php in that particular file it will only be in there once, but many files will have several lines like that, so I was just wondering how it handled that I would guess it would act on the first one, but you know what they say when you assume something. Which led me to my next question what if you wanted to replace all occurrences of some variable or whatever in a given file how would you handle that?
Seems I overlooked this question...
At this time it finds them all so it is important to find very specific strings. "<?php" maybe not be the best example, but in that file it works ok. I am going to add the ability to limit the number and maybe index them so you can set it only to change the 3rd one in the file.
ok that is very good to know.

OpenCart commercial mods and development http://spotonsolutions.net
Layered Navigation
Shipment Tracking
Vehicle Year/Make/Model Filter


Guru Member

Posts

Joined
Sun Oct 25, 2009 3:51 am
Location - FL US

Post by xds » Tue Dec 28, 2010 10:47 am

Index would probably be the most flexible way of handling such occurrences.

Thanks for 1.5!


xds
New member

Posts

Joined
Mon Jul 12, 2010 9:53 am

Post by Qphoria » Wed Dec 29, 2010 6:53 am

UPDATE COMING SOON
Currently the code uses the server's "/tmp" directory to create the modified files. There is also the option of enabling debug which will create copies of those files to a "vqdbg" folder. Instead of all this, I think it would just be best to create these files in the "vqdbg" folder directly.. and rename that folder to "vqcache" or something.

This will give us 2 new features:
- The ability to always see the changed files without having to write and copy for debug.
- The ability to add a new "use cached" feature which will allow you to use the pre-modded versions of the files once they have been created once. Similar to the way images are resized in opencart. They are generated once, and then reused if they already exist. This will remove any worries of performance hit from constantly re-writing the files on every page load, which was already deemed to be a non-issue, making this tool the best thing since the internet was invented. But modesty prevents me from saying that :clown:

I'm also going to add a debug logging option to better trace what is happening as it goes to make it even easier to find mod issues. None of these changes will break any current xml mods. Stay tuned!

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by JNeuhoff » Wed Dec 29, 2010 10:40 pm

I played around more with your VirtualQMod and have now come to the conclusion that it is quite useful. Will test some more of my Opencart addons with it. My addons tend to modify Opencart core files, so this will be a useful test.

Export/Import Tool * SpamBot Buster * Unused Images Manager * Instant Option Price Calculator * Number Option * Google Tag Manager * Survey Plus * OpenTwig


User avatar
Guru Member
Online

Posts

Joined
Wed Dec 05, 2007 3:38 am

Who is online

Users browsing this forum: No registered users and 22 guests