Post by chris.dempsey » Sat Mar 12, 2016 11:15 pm

Can anyone suggest why vQmod fails to write cache files and instead writes an empty file named vq2-C to /vqmod/vqcache?

Environment
Windows Server 2012
Plesk Panel 12.5
PHP 5.3.2.9
MySQL 5.6.26
OpenCart 2.1.0.1
vQmod 2.6.1

Issue
vQmod fails to save modifications to /vqmod/vqcache/vq2-*.php

On each page load it applies modifications specified in /vqmod/xml/*.xml and writes an empty file named vq2-C to /vqmod/vqcache.

Background
The affected website was migrated from another Windows box with similar configuration.

In short the old server ran Plesk Panel 12.0, the new server is Plesk 12.5 so has minor updates to software versions.

Both sites run on PHP 5.3.2.9 and the new server follows OWASP recommendations more closely so has more PHP functions disabled eg. fopen_with_path.

Investigation so far
Running the vQmod installer again reports: VQMOD ALREADY INSTALLED!

File permissions - /vqmod/logs and /vqmod/vqcache have modify permissions, files are written here. Permissions are applied through Plesk Panel, checked over remote desktop and enabling global write permissions on web root through Plesk does not change anything.

Logs
vQmod logs have no useful information, only skipped files are noted eg. VQModObject::parseMods - Could not resolve path for [ catalog/language/english/module/featured.php] (SKIPPED).

No php_error.log files are generated.

Failed Request Tracing does not pick up any issues.

Tests
All /vqmod/xml files have been removed except vqmod_opencart.xml and one that modifies column_left.tpl. These modifications are applied successfully but no cache files are generated in /vqmod/vqcache.

If I remove /vqmod/checked.cache and /vqmod/mods.cache the files are regenerated on next page load.

vQmod versions - rolled back to 2.5.1 but the issue persists.

Other considerations
When one particular vQmod modification is enabled page load time is unacceptably slow (up to 20 sec). The modification displays the first 4 products from sub categories on the parent category page. I've not gone through the code yet but assume it's hitting pretty hard on the database.

On the original server page load was sub 2 seconds. I doubt this is related to the cache issue as that seems to be a permissions problem.
Last edited by chris.dempsey on Mon Mar 14, 2016 6:17 pm, edited 2 times in total.

New member

Posts

Joined
Wed Apr 04, 2012 6:43 pm

Post by IP_CAM » Sat Mar 12, 2016 11:32 pm

then, better use a UNIX based Server, MS is known crab, when it comes to handle such software.
Ernie
openshop.li

My Github OC Site: https://github.com/IP-CAM
5'200 + FREE OC Extensions, on the World's largest private Github OC Repository Archive Site.


User avatar
Legendary Member

Posts

Joined
Tue Mar 04, 2014 1:37 am
Location - Switzerland

Post by chris.dempsey » Sun Mar 13, 2016 6:39 pm

That is neither helpful nor well informed.

Did you even read past the first line of the environment details?

Or did you see Windows and think it would be a good idea to try and shut down this thread simply because my setup is not to your personal preference?

New member

Posts

Joined
Wed Apr 04, 2012 6:43 pm

Post by artcore » Sun Mar 13, 2016 7:26 pm

I suggest turning on php error reporting to see where it fails.

index.php right below <?php

ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);

Or check the plesk/apache error logs

Do you have the same issues with ocmod?

Attn: I no longer provide OpenCart extensions, nor future support - this includes forum posts.
Reason: OpenCart version 3+ ;D

Thanks!


User avatar
Active Member

Posts

Joined
Tue Jul 09, 2013 4:13 am
Location - The Netherlands

Post by chris.dempsey » Sun Mar 13, 2016 8:47 pm

I suggest turning on php error reporting to see where it fails.
error_reporting is already on with errors logged to file. No entries are created.
index.php right below <?php

ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
Doing this does not display any errors.
Do you have the same issues with ocmod?
No, ocmods are applied successfuly.

Nothing is recorded in /system/storage/logs/error.log

Update - in addition, there is nothing registered in the real time anti malware scanner logs to suggest that vqmod attempts to cache files that get blocked.

New member

Posts

Joined
Wed Apr 04, 2012 6:43 pm

Post by artcore » Sun Mar 13, 2016 10:04 pm

Does apache (or equivalent) has anything in the logs?
I was wondering if file_put_contents or another php function is restricted as per your OWASP settings.

Attn: I no longer provide OpenCart extensions, nor future support - this includes forum posts.
Reason: OpenCart version 3+ ;D

Thanks!


User avatar
Active Member

Posts

Joined
Tue Jul 09, 2013 4:13 am
Location - The Netherlands

Post by chris.dempsey » Sun Mar 13, 2016 10:11 pm

Not that I can see.

I put php.ini back to default settings and there is no change to the caching issue.

Not sure how much of a clue the name of the empty file that is written is: vq2-C

Hoped it might indicate more specifically where vqmod.php is failing.

New member

Posts

Joined
Wed Apr 04, 2012 6:43 pm

Post by chris.dempsey » Sun Mar 13, 2016 10:56 pm

Code: Select all

file_put_contents("test.txt","Text");
Generates test.txt when executed in /vqmod/vqcache

So the function is available and file permissions correct.

New member

Posts

Joined
Wed Apr 04, 2012 6:43 pm

Post by budgetneon » Sun Mar 13, 2016 11:02 pm

You could add some debugging to vqmod.php to try and trace what's going on.

These seem to be the 2 areas of concern....

The name of the cache file is determined here:

Code: Select all

               
   $stripped_filename = preg_replace('~^' . preg_quote(self::getCwd(), '~i') . '~', '', $sourcePath);
   $cacheFile = self::_cacheName($stripped_filename);
The file is written here.

Code: Select all

if (sha1($fileData) != $fileHash) {
			$writePath = $cacheFile;
			if(!file_exists($writePath) || is_writable($writePath)) {
				file_put_contents($writePath, $fileData, LOCK_EX);
				$changed = true;
			}
}

New member

Posts

Joined
Sat Sep 20, 2014 11:32 pm


Post by chris.dempsey » Sun Mar 13, 2016 11:47 pm

Getting there now...

On line #121

Code: Select all

$cacheFile = self::_cacheName($stripped_filename);
$cacheFile is set to:
C:\Net\hosts\domain\htdocs\vqmod\vqcache\vq2-C:_Net_hosts_htdocs_domain_system_startup.php

This explains the file name of the cache file that is written: vq2-C

The _cachename function is

Code: Select all

	private static function _cacheName($file) {
		return self::$_cachePathFull . 'vq2-' . preg_replace('~[/\\\\]+~', '_', $file);
	}
So the part that fails is: preg_replace('~[/\\\\]+~', '_', $file);

Regex isn't my strong point, will look further shortly.

Appreciate your thoughts thus far.

New member

Posts

Joined
Wed Apr 04, 2012 6:43 pm

Post by budgetneon » Mon Mar 14, 2016 12:00 am

chris.dempsey wrote:Getting there now...
...
$cacheFile is set to:
C:\Net\hosts\domain\htdocs\vqmod\vqcache\vq2-C:_Net_hosts_htdocs_domain_system_startup.php
That's it. Windows is unique in that ':' (a colon) is not a valid file name character. It's valid on linux/unix, and the vqmod author probably didn't test for that.

Edited:
Try changing to this:

Code: Select all

$stripped_filename = preg_replace('~^' . preg_quote(self::getCwd(), '~i') . '~', '', $sourcePath);
$stripped_filename=str_replace(':','',$stripped_filename);
That's kind of a quick/dirty fix. It doesn't account for other non-allowed chars in filenames on windows.

Probably should open an issue on vqmod's github page.

New member

Posts

Joined
Sat Sep 20, 2014 11:32 pm


Post by chris.dempsey » Mon Mar 14, 2016 12:20 am

Tested but no success.

I think $cacheFile is being set to the wrong value.

Code: Select all

C:\Net\hosts\domain\htdocs\vqmod\vqcache\vq2-C:_Net_hosts_htdocs_domain_system_startup.php
should be

Code: Select all

vq2-system_startup.php
On the original Windows server the correct value was returned by

Code: Select all

return self::$_cachePathFull . 'vq2-' . preg_replace('~[/\\\\]+~', '_', $file);


I should have remembered that *nix systems allow colons in the filename. A client had an issue syncing PDFs through Dropbox between Windows and Mac machines. Fine on the Mac but Windows couldn't get the files.


Edit: you updated your post while I was writing this, so this may not be valid now.

New member

Posts

Joined
Wed Apr 04, 2012 6:43 pm

Post by budgetneon » Mon Mar 14, 2016 12:25 am

Ok...I opened an issue on vqmod's github repo.

Sounds like you already know the areas to experiment around with, so I'm probably not much help at this point...I don't have OC running on windows myself.

New member

Posts

Joined
Sat Sep 20, 2014 11:32 pm


Post by budgetneon » Mon Mar 14, 2016 12:28 am

chris.dempsey wrote:Edit: you updated your post while I was writing this, so this may not be valid now.
Now that I see what's happening, I don't think my edited version will help either. You need to replace 'c:' with '\\' or '/', I think, although that's not ideal either :)

New member

Posts

Joined
Sat Sep 20, 2014 11:32 pm


Post by chris.dempsey » Mon Mar 14, 2016 12:29 am

Your quick/dirty fix corrects the filename enough that it can be written to the cache.

However it includes the full path so vQmod won't recognise the cache files representing when checking if they already exist.

ie. cache filename is
vq2-C_net_hosts_domain_htdocs_catalog_view_theme_default_template_common_column_left

And should be
vq2-catalog_view_theme_default_template_common_column_left

New member

Posts

Joined
Wed Apr 04, 2012 6:43 pm

Post by budgetneon » Mon Mar 14, 2016 12:44 am

chris.dempsey wrote:Your quick/dirty fix corrects the filename enough that it can be written to the cache.

However it includes the full path so vQmod won't recognise the cache files representing when checking if they already exist.

ie. cache filename is
vq2-C_net_hosts_domain_htdocs_catalog_view_theme_default_template_common_column_left

And should be
vq2-catalog_view_theme_default_template_common_column_left
It's doing that because realpath() is being called, and it seems to be on purpose.

So, my new quick/dirty fix is to find this:

Code: Select all

private static function _realpath($file) {
		$path = realpath($file);
		if(!$path) {
			return false;
		}
		if(is_dir($path)) {
			$path = rtrim($path, self::$directorySeparator) . self::$directorySeparator;
		}
		return $path;
}
And replace it with this. Put everything else back like it was.

Code: Select all

private static function _realpath($file) {
		$path = realpath($file);
            
		if(!$path) {
			return false;
		}
		if(is_dir($path)) {
			$path = rtrim($path, self::$directorySeparator) . self::$directorySeparator;
		}
		$path=preg_replace('/^[a-zA-Z]:/','',$path);
		return $path;
}
It's wrong, as it would break if you were using the d: drive, for example, but it will suffice until a real fix is available, if you have everything on c:

New member

Posts

Joined
Sat Sep 20, 2014 11:32 pm


Post by artcore » Mon Mar 14, 2016 12:47 am

Nice detective skills guys!
I had a hunch it would preg_replace but for the deprecated reason. As of php 5.5+
http://php.net/manual/en/function.preg-replace.php

Maybe you can add a final str_replace on the filename result to get rid of the 'hosts' part as it's the same always.
But I'm sure Qphoria and Jay will come up with a more elegant solution ;D

edit: budgetneon already had a better solution. I take my hat off ;)

Attn: I no longer provide OpenCart extensions, nor future support - this includes forum posts.
Reason: OpenCart version 3+ ;D

Thanks!


User avatar
Active Member

Posts

Joined
Tue Jul 09, 2013 4:13 am
Location - The Netherlands

Post by chris.dempsey » Mon Mar 14, 2016 1:04 am

And replace it with this. Put everything else back like it was.

Code: Select all

private static function _realpath($file) {
		$path = realpath($file);
            
		if(!$path) {
			return false;
		}
		if(is_dir($path)) {
			$path = rtrim($path, self::$directorySeparator) . self::$directorySeparator;
		}
		$path=preg_replace('/^[a-zA-Z]:/','',$path);
		return $path;
}
If I've done this correctly it's still generating cache files named from the root of the drive.

ie. cache filename is
vq2-C_net_hosts_domain_htdocs_catalog_view_theme_default_template_common_column_left

And should be
vq2-catalog_view_theme_default_template_common_column_left

Will be later this evening before I can look at this proper again.

Thanks again.

New member

Posts

Joined
Wed Apr 04, 2012 6:43 pm

Post by budgetneon » Mon Mar 14, 2016 1:07 am

chris.dempsey wrote: If I've done this correctly it's still generating cache files named from the root of the drive.
I'm pretty sure that "named from the root of the drive" is the intention. That's what realpath() does...fully resolves the entire path, and it's being called, on purpose.

New member

Posts

Joined
Sat Sep 20, 2014 11:32 pm


Post by chris.dempsey » Mon Mar 14, 2016 1:11 am

I'm starting to think that.

Need to wait for some peace and quiet, small child is playing loudly at my feet.

New member

Posts

Joined
Wed Apr 04, 2012 6:43 pm
Who is online

Users browsing this forum: No registered users and 31 guests