Page 1 of 2
[Solved] vQmod fails to write to vqcache directory on IIS
Posted: Sat Mar 12, 2016 11:15 pm
by chris.dempsey
Can anyone suggest why vQmod fails to write cache files and instead writes an empty file named vq2-C to /vqmod/vqcache?
Environment
Windows Server 2012
Plesk Panel 12.5
PHP 5.3.2.9
MySQL 5.6.26
OpenCart 2.1.0.1
vQmod 2.6.1
Issue
vQmod fails to save modifications to /vqmod/vqcache/vq2-*.php
On each page load it applies modifications specified in /vqmod/xml/*.xml and writes an empty file named vq2-C to /vqmod/vqcache.
Background
The affected website was migrated from another Windows box with similar configuration.
In short the old server ran Plesk Panel 12.0, the new server is Plesk 12.5 so has minor updates to software versions.
Both sites run on PHP 5.3.2.9 and the new server follows OWASP recommendations more closely so has more PHP functions disabled eg. fopen_with_path.
Investigation so far
Running the vQmod installer again reports: VQMOD ALREADY INSTALLED!
File permissions - /vqmod/logs and /vqmod/vqcache have modify permissions, files are written here. Permissions are applied through Plesk Panel, checked over remote desktop and enabling global write permissions on web root through Plesk does not change anything.
Logs
vQmod logs have no useful information, only skipped files are noted eg. VQModObject::parseMods - Could not resolve path for [ catalog/language/english/module/featured.php] (SKIPPED).
No php_error.log files are generated.
Failed Request Tracing does not pick up any issues.
Tests
All /vqmod/xml files have been removed except vqmod_opencart.xml and one that modifies column_left.tpl. These modifications are applied successfully but no cache files are generated in /vqmod/vqcache.
If I remove /vqmod/checked.cache and /vqmod/mods.cache the files are regenerated on next page load.
vQmod versions - rolled back to 2.5.1 but the issue persists.
Other considerations
When one particular vQmod modification is enabled page load time is unacceptably slow (up to 20 sec). The modification displays the first 4 products from sub categories on the parent category page. I've not gone through the code yet but assume it's hitting pretty hard on the database.
On the original server page load was sub 2 seconds. I doubt this is related to the cache issue as that seems to be a permissions problem.
Re: vQmod fails to write to vqcache directory
Posted: Sat Mar 12, 2016 11:32 pm
by IP_CAM
then, better use a UNIX based Server, MS is known crab, when it comes to handle such software.
Ernie
openshop.li
Re: vQmod fails to write to vqcache directory
Posted: Sun Mar 13, 2016 6:39 pm
by chris.dempsey
That is neither helpful nor well informed.
Did you even read past the first line of the environment details?
Or did you see Windows and think it would be a good idea to try and shut down this thread simply because my setup is not to your personal preference?
Re: vQmod fails to write to vqcache directory
Posted: Sun Mar 13, 2016 7:26 pm
by artcore
I suggest turning on php error reporting to see where it fails.
index.php right below <?php
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
Or check the plesk/apache error logs
Do you have the same issues with ocmod?
Re: vQmod fails to write to vqcache directory
Posted: Sun Mar 13, 2016 8:47 pm
by chris.dempsey
I suggest turning on php error reporting to see where it fails.
error_reporting is already on with errors logged to file. No entries are created.
index.php right below <?php
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
Doing this does not display any errors.
Do you have the same issues with ocmod?
No, ocmods are applied successfuly.
Nothing is recorded in /system/storage/logs/error.log
Update - in addition, there is nothing registered in the real time anti malware scanner logs to suggest that vqmod attempts to cache files that get blocked.
Re: vQmod fails to write to vqcache directory
Posted: Sun Mar 13, 2016 10:04 pm
by artcore
Does apache (or equivalent) has anything in the logs?
I was wondering if file_put_contents or another php function is restricted as per your OWASP settings.
Re: vQmod fails to write to vqcache directory
Posted: Sun Mar 13, 2016 10:11 pm
by chris.dempsey
Not that I can see.
I put php.ini back to default settings and there is no change to the caching issue.
Not sure how much of a clue the name of the empty file that is written is: vq2-C
Hoped it might indicate more specifically where vqmod.php is failing.
Re: vQmod fails to write to vqcache directory
Posted: Sun Mar 13, 2016 10:56 pm
by chris.dempsey
Code: Select all
file_put_contents("test.txt","Text");
Generates test.txt when executed in /vqmod/vqcache
So the function is available and file permissions correct.
Re: vQmod fails to write to vqcache directory
Posted: Sun Mar 13, 2016 11:02 pm
by budgetneon
You could add some debugging to vqmod.php to try and trace what's going on.
These seem to be the 2 areas of concern....
The name of the cache file is determined here:
Code: Select all
$stripped_filename = preg_replace('~^' . preg_quote(self::getCwd(), '~i') . '~', '', $sourcePath);
$cacheFile = self::_cacheName($stripped_filename);
The file is written here.
Code: Select all
if (sha1($fileData) != $fileHash) {
$writePath = $cacheFile;
if(!file_exists($writePath) || is_writable($writePath)) {
file_put_contents($writePath, $fileData, LOCK_EX);
$changed = true;
}
}
Re: vQmod fails to write to vqcache directory
Posted: Sun Mar 13, 2016 11:47 pm
by chris.dempsey
Getting there now...
On line #121
Code: Select all
$cacheFile = self::_cacheName($stripped_filename);
$cacheFile is set to:
C:\Net\hosts\domain\htdocs\vqmod\vqcache
\vq2-C:_Net_hosts_htdocs_domain_system_startup.php
This explains the file name of the cache file that is written: vq2-C
The _cachename function is
Code: Select all
private static function _cacheName($file) {
return self::$_cachePathFull . 'vq2-' . preg_replace('~[/\\\\]+~', '_', $file);
}
So the part that fails is: preg_replace('~[/\\\\]+~', '_', $file);
Regex isn't my strong point, will look further shortly.
Appreciate your thoughts thus far.
Re: vQmod fails to write to vqcache directory
Posted: Mon Mar 14, 2016 12:00 am
by budgetneon
chris.dempsey wrote:Getting there now...
...
$cacheFile is set to:
C:\Net\hosts\domain\htdocs\vqmod\vqcache\vq2-C:_Net_hosts_htdocs_domain_system_startup.php
That's it. Windows is unique in that ':' (a colon) is not a valid file name character. It's valid on linux/unix, and the vqmod author probably didn't test for that.
Edited:
Try changing to this:
Code: Select all
$stripped_filename = preg_replace('~^' . preg_quote(self::getCwd(), '~i') . '~', '', $sourcePath);
$stripped_filename=str_replace(':','',$stripped_filename);
That's kind of a quick/dirty fix. It doesn't account for other
non-allowed chars in filenames on windows.
Probably should open an issue on vqmod's github page.
Re: vQmod fails to write to vqcache directory
Posted: Mon Mar 14, 2016 12:20 am
by chris.dempsey
Tested but no success.
I think $cacheFile is being set to the wrong value.
Code: Select all
C:\Net\hosts\domain\htdocs\vqmod\vqcache\vq2-C:_Net_hosts_htdocs_domain_system_startup.php
should be
On the original Windows server the correct value was returned by
Code: Select all
return self::$_cachePathFull . 'vq2-' . preg_replace('~[/\\\\]+~', '_', $file);
I should have remembered that *nix systems allow colons in the filename. A client had an issue syncing PDFs through Dropbox between Windows and Mac machines. Fine on the Mac but Windows couldn't get the files.
Edit: you updated your post while I was writing this, so this may not be valid now.
Re: vQmod fails to write to vqcache directory
Posted: Mon Mar 14, 2016 12:25 am
by budgetneon
Ok...I opened
an issue on vqmod's github repo.
Sounds like you already know the areas to experiment around with, so I'm probably not much help at this point...I don't have OC running on windows myself.
Re: vQmod fails to write to vqcache directory
Posted: Mon Mar 14, 2016 12:28 am
by budgetneon
chris.dempsey wrote:Edit: you updated your post while I was writing this, so this may not be valid now.
Now that I see what's happening, I don't think my edited version will help either. You need to replace 'c:' with '\\' or '/', I think, although that's not ideal either
Re: vQmod fails to write to vqcache directory
Posted: Mon Mar 14, 2016 12:29 am
by chris.dempsey
Your quick/dirty fix corrects the filename enough that it can be written to the cache.
However it includes the full path so vQmod won't recognise the cache files representing when checking if they already exist.
ie. cache filename is
vq2-C_net_hosts_domain_htdocs_catalog_view_theme_default_template_common_column_left
And should be
vq2-catalog_view_theme_default_template_common_column_left
Re: vQmod fails to write to vqcache directory
Posted: Mon Mar 14, 2016 12:44 am
by budgetneon
chris.dempsey wrote:Your quick/dirty fix corrects the filename enough that it can be written to the cache.
However it includes the full path so vQmod won't recognise the cache files representing when checking if they already exist.
ie. cache filename is
vq2-C_net_hosts_domain_htdocs_catalog_view_theme_default_template_common_column_left
And should be
vq2-catalog_view_theme_default_template_common_column_left
It's doing that because realpath() is being called, and it seems to be on purpose.
So, my new quick/dirty fix is to find this:
Code: Select all
private static function _realpath($file) {
$path = realpath($file);
if(!$path) {
return false;
}
if(is_dir($path)) {
$path = rtrim($path, self::$directorySeparator) . self::$directorySeparator;
}
return $path;
}
And replace it with this. Put everything else back like it was.
Code: Select all
private static function _realpath($file) {
$path = realpath($file);
if(!$path) {
return false;
}
if(is_dir($path)) {
$path = rtrim($path, self::$directorySeparator) . self::$directorySeparator;
}
$path=preg_replace('/^[a-zA-Z]:/','',$path);
return $path;
}
It's wrong, as it would break if you were using the d: drive, for example, but it will suffice until a real fix is available, if you have everything on c:
Re: vQmod fails to write to vqcache directory
Posted: Mon Mar 14, 2016 12:47 am
by artcore
Nice detective skills guys!
I had a hunch it would preg_replace but for the deprecated reason. As of php 5.5+
http://php.net/manual/en/function.preg-replace.php
Maybe you can add a final str_replace on the filename result to get rid of the 'hosts' part as it's the same always.
But I'm sure Qphoria and Jay will come up with a more elegant solution
edit: budgetneon already had a better solution. I take my hat off
Re: vQmod fails to write to vqcache directory
Posted: Mon Mar 14, 2016 1:04 am
by chris.dempsey
And replace it with this. Put everything else back like it was.
Code: Select all
private static function _realpath($file) {
$path = realpath($file);
if(!$path) {
return false;
}
if(is_dir($path)) {
$path = rtrim($path, self::$directorySeparator) . self::$directorySeparator;
}
$path=preg_replace('/^[a-zA-Z]:/','',$path);
return $path;
}
If I've done this correctly it's still generating cache files named from the root of the drive.
ie. cache filename is
vq2-C_net_hosts_domain_htdocs_catalog_view_theme_default_template_common_column_left
And should be
vq2-catalog_view_theme_default_template_common_column_left
Will be later this evening before I can look at this proper again.
Thanks again.
Re: vQmod fails to write to vqcache directory
Posted: Mon Mar 14, 2016 1:07 am
by budgetneon
chris.dempsey wrote:
If I've done this correctly it's still generating cache files named from the root of the drive.
I'm pretty sure that "named from the root of the drive" is the intention. That's what realpath() does...fully resolves the entire path, and it's being called, on purpose.
Re: vQmod fails to write to vqcache directory
Posted: Mon Mar 14, 2016 1:11 am
by chris.dempsey
I'm starting to think that.
Need to wait for some peace and quiet, small child is playing loudly at my feet.