Post by pm-netti » Mon Mar 14, 2016 10:45 pm

Qphoria wrote: I've attached the upgrade files.
Just unzip and upload the normal opencart 2.2.0.0 zip package
Then unzip my files and overwrite the 7 files in install/model/upgrade
Then run the upgrade script and it "should" work.
Be sure to follow the instructions after upgrade about user group permissions
Versions 1.4.x and 1.5.0 is not work:

Attachments

150.jpg

version 1.5.0 - 150.jpg (56.03 KiB) Viewed 4864 times

149.jpg

version 1.4.9 - 149.jpg (60.27 KiB) Viewed 4864 times


User avatar
Active Member

Posts

Joined
Sat Apr 07, 2012 11:22 pm
Location - Kittilä, Finland

Post by Qphoria » Mon Mar 14, 2016 11:21 pm

pm-netti wrote:
Qphoria wrote: I've attached the upgrade files.
Just unzip and upload the normal opencart 2.2.0.0 zip package
Then unzip my files and overwrite the 7 files in install/model/upgrade
Then run the upgrade script and it "should" work.
Be sure to follow the instructions after upgrade about user group permissions
Versions 1.4.x and 1.5.0 is not work:
Yea 1.4.x won't work.

So here's the thing.. the 2.0 and 2.1 upgrade scripts included a more dynamic design that actually parsed through every table and column in the opencart.sql file and made the database match that. That was removed in 2.2.0.0 in favor of a progressive file system using static db calls.

I had this added back for 2.2 initially (and it left it commented out in 1000.php) but it sort of conflicts with the static calls made in the progressive files. Perhaps we need to detect if versions older than 1.5.2 are the source, then use the dynamic system, otherwise use the static ones. So yea.. I will note that the upgrade scripts as defined in 2.2 are only made for coming from 1.5.3.1 or later.

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by labeshops » Tue Mar 15, 2016 12:12 am

Great. So going from 1.5.6.4 to 2.2 should not be a problem?

I'm planning to copy everything to a test folder with a copy of my db just in case, disable all extensions, then run the scripts. Assuming this is the proper way?

Running Opencart v3.0.3.2 with multi-stores and the default template from https://www.labeshops.com which has links to all my stores.


User avatar
Expert Member

Posts

Joined
Thu Aug 04, 2011 4:41 am
Location - Florida, USA

Post by pm-netti » Tue Mar 15, 2016 12:21 am

Qphoria wrote:
pm-netti wrote:
Qphoria wrote: I've attached the upgrade files.
Just unzip and upload the normal opencart 2.2.0.0 zip package
Then unzip my files and overwrite the 7 files in install/model/upgrade
Then run the upgrade script and it "should" work.
Be sure to follow the instructions after upgrade about user group permissions
Versions 1.4.x and 1.5.0 is not work:
Yea 1.4.x won't work.

So here's the thing.. the 2.0 and 2.1 upgrade scripts included a more dynamic design that actually parsed through every table and column in the opencart.sql file and made the database match that. That was removed in 2.2.0.0 in favor of a progressive file system using static db calls.

I had this added back for 2.2 initially (and it left it commented out in 1000.php) but it sort of conflicts with the static calls made in the progressive files. Perhaps we need to detect if versions older than 1.5.2 are the source, then use the dynamic system, otherwise use the static ones. So yea.. I will note that the upgrade scripts as defined in 2.2 are only made for coming from 1.5.3.1 or later.
Old upgrade script can exploit (this method newData, by line 494):
https://github.com/pekka2/OpenCart-Migr ... ucture.php
when pick up that array, can add new columns automaticilly. (ps. that free upgrade script is not in use)

User avatar
Active Member

Posts

Joined
Sat Apr 07, 2012 11:22 pm
Location - Kittilä, Finland

Post by pm-netti » Tue Mar 15, 2016 2:37 am

Ps. In file 1001.php lines 186-218 is the associated script, it can exploid and fix necessary.

User avatar
Active Member

Posts

Joined
Sat Apr 07, 2012 11:22 pm
Location - Kittilä, Finland

Post by Qphoria » Tue Mar 15, 2016 4:19 am

pm-netti wrote:Ps. In file 1001.php lines 186-218 is the associated script, it can exploid and fix necessary.
You mean 1000.php
But there is a problem with that when you hit a table that changes the primary key to a new name.

There are 2 tables
affiliate_activity
customer_activity

that have their primary keys names changed from
activity_id to affiliate_activity_id
and
activity_id to customer_activity_id

When the lines between 186-218 are uncommented, it tries to add the affiliate_activity_id to the table first because it doesn't exist. So now the table has both "affiliate_activity_id" and "activity_id" and currently activity_id is mapped to the primary key and Auto_inc. But now we are inside the column loop of the NEW columns when it removes the auto-inc from all fields in the list. But that old field "activity_id" isn't in the list of fields so it doesn't get called so then you get a crash that says there can only be one auto-inc field and it has to be the primary key.

That is easy enough to fix by doing a secondary loop through the old table fields and removing all auto-inc on any old fields as well.

But now we have a new affiliate_activity_id with a default 0 value for all records. When the code that tries to set the auto-inc on the new column comes along, it crashes because of duplicate key 0 for the primary key.

So one solution is to call the column name change BEFORE running the dynamic script. But that now adds complexity to the overall system and potential for forgetting about that in the future.

But basically for any future primary key name changes, we will have to pre-handle those before calling the dynamic script. And that breaks the design of that script and the attempt at a progressive style upgrade system.

Perhaps we can track the "old" primary key and the "new" one and if it has a different name then call a CHANGE command instead of ADD

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by Qphoria » Tue Mar 15, 2016 5:20 am

Ok I've done just that. I grabbed the extended field info from the old table and match the auto_inc. If the field doesn't exist and is autoinc in the new table, it changes the name from the old table name to the new table name. Now we can get the dynamic script back again. I'll clean it up and zip it. This should at least give us 1.5.0 support.
1.4.x technically will work except there is no code for the product option changes from 1.4.x to 1.5.x so it will lose all options. For 1.4.x you are best using the BETA upgrade script I made a few years ago.

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by pm-netti » Tue Mar 15, 2016 5:27 am

Qphoria wrote:Ok I've done just that. I grabbed the extended field info from the old table and match the auto_inc. If the field doesn't exist and is autoinc in the new table, it changes the name from the old table name to the new table name. Now we can get the dynamic script back again. I'll clean it up and zip it. This should at least give us 1.5.0 support.
1.4.x technically will work except there is no code for the product option changes from 1.4.x to 1.5.x so it will lose all options. For 1.4.x you are best using the BETA upgrade script I made a few years ago.
I am my script removed auto_increment test from loop. then Drop eg. column customer_activity_id and rename activity_id to customer_activity_id.

User avatar
Active Member

Posts

Joined
Sat Apr 07, 2012 11:22 pm
Location - Kittilä, Finland

Post by Qphoria » Tue Mar 15, 2016 10:09 am

Ok This one adds back the dynamic code and more
https://github.com/opencart/opencart/pull/4126

Features:
- Auto update config.php and admin/config.php with
DIR_UPLOAD
DIR_MODIFICATION
DB_PORT
mysql -> mysqli
- Auto convert 1.5.x modules to 2.x format, disabling any non-core modules to prevent errors.
- Auto-Convert Welcome to HTML
- Restored the dynamic update script for individual columns with fix for issue with renaming primary keys
- Added default values for API user with generated hash if not set
- Added default values for processing/complete statuses, if not set, to prevent errors in the admin
- Copied the product name to the new meta_title field, if empty, to prevent errors as it is a required field.

It would work for 1.4.x except for the product option change will not convert over.
Probably best to leave 1.4.x updates to that old BETA script I made a while back as that does have basic option conversion support already:
viewtopic.php?f=19&t=50292

To test:
1. Download OpenCart 2.2.0.0
2. Unzip and upload to your server, overwriting all existing files (yes you should have backed up first)
3. Unzip the files from my attachment to the install/model/upgrade folder, overwrite the default files.
4. Run the upgrade.

Attachments


Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by sculptex » Tue Mar 15, 2016 9:07 pm

just tried upgrading my test site from 2.0.3.1 - and I got an error when processing

actually it must be when processing 1003 as the progress bar says 1002 has been applied

(I have nearly 30,000 customers and over 500 products on there)

SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data

OK

Code: Select all

<br />
<b>Fatal error</b>:  Allowed memory size of 67108864 bytes exhausted (tried to allocate 32 bytes) in <b>/home/****/public_html/system/library/db/mysqli.php</b> on line <b>24</b><br />

ImageImage


User avatar
Active Member

Posts

Joined
Tue Sep 13, 2011 3:07 am
Location - UK

Post by pm-netti » Tue Mar 15, 2016 10:00 pm

sculptex wrote:just tried upgrading my test site from 2.0.3.1 - and I got an error when processing

actually it must be when processing 1003 as the progress bar says 1002 has been applied

(I have nearly 30,000 customers and over 500 products on there)

SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data

OK

Code: Select all

<br />
<b>Fatal error</b>:  Allowed memory size of 67108864 bytes exhausted (tried to allocate 32 bytes) in <b>/home/****/public_html/system/library/db/mysqli.php</b> on line <b>24</b><br />
How if change this lines 128-130 in file 1003.php:

Code: Select all

$data = unserialize($result['data']);

				$this->db->query("UPDATE `" . DB_PREFIX . "customer_activity` SET `data` = '" . $this->db->escape(json_encode($data)) . "' WHERE `customer_activity_id` = '" . (int)$result['customer_activity_id'] . "'");
to:

Code: Select all

$this->db->query("UPDATE `" . DB_PREFIX . "customer_activity` SET `data` = '" . $this->db->escape(json_encode(unserialize($result['data']))) . "' WHERE `customer_activity_id` = '" . (int)$result['customer_activity_id'] . "'");

User avatar
Active Member

Posts

Joined
Sat Apr 07, 2012 11:22 pm
Location - Kittilä, Finland

Post by Qphoria » Tue Mar 15, 2016 11:07 pm

sculptex wrote:just tried upgrading my test site from 2.0.3.1 - and I got an error when processing

actually it must be when processing 1003 as the progress bar says 1002 has been applied

(I have nearly 30,000 customers and over 500 products on there)

SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data

OK

Code: Select all

<br />
<b>Fatal error</b>:  Allowed memory size of 67108864 bytes exhausted (tried to allocate 32 bytes) in <b>/home/****/public_html/system/library/db/mysqli.php</b> on line <b>24</b><br />
That is unfortunately a server side issue. and likely happens during the step that processes the orders or the activity because it tries to load it all up in memory. Try editing the php.ini file in the install folder and change 64M to 1024M

We may need to alter that step and load up only a few thousand rows at a time and loop through to cut back on the overall memory.

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by Qphoria » Tue Mar 15, 2016 11:18 pm

pm-netti wrote: How if change this lines 128-130 in file 1003.php:

Code: Select all

$data = unserialize($result['data']);

				$this->db->query("UPDATE `" . DB_PREFIX . "customer_activity` SET `data` = '" . $this->db->escape(json_encode($data)) . "' WHERE `customer_activity_id` = '" . (int)$result['customer_activity_id'] . "'");
to:

Code: Select all

$this->db->query("UPDATE `" . DB_PREFIX . "customer_activity` SET `data` = '" . $this->db->escape(json_encode(unserialize($result['data']))) . "' WHERE `customer_activity_id` = '" . (int)$result['customer_activity_id'] . "'");
I guess that could be the same as just changing:
$data = unserialize($result['data']);
to
$result['data'] = unserialize($result['data']);
?
not sure if setting the variable back to itself saves the memory of creating another variable. But you may be right, it might save some memory your way. Still I think the larger problem is the select * trying to grab too much data at once. Just selecting the individual columns that need to be converted would be best. I'll clean it up a bit.

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by sculptex » Wed Mar 16, 2016 12:19 am

okay I increased memory allowance by adding

Code: Select all

ini_set('memory_limit', '2048M');
into 1003.php
after

Code: Select all

public function upgrade() {
and upgrade completed successfully.

Now I get

Notice: Error: Could not load template /home/xxx/public_html/catalog/view/theme/common/language! in /home/xxx/public_html/system/modification/system/engine/loader.php on line 45

and similar in admin

ImageImage


User avatar
Active Member

Posts

Joined
Tue Sep 13, 2011 3:07 am
Location - UK

Post by Qphoria » Wed Mar 16, 2016 2:47 am

The admin shouldn't have that same error as it doesn't use themes. But the error is weird, like it set the default theme to "language". What is the actual admin error?

Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by sculptex » Wed Mar 16, 2016 3:38 am

admin error
Notice: Error: Could not load template /home/xxx/public_html/admin/view/template/common/header! in /home/xxx/public_html/system/modification/system/engine/loader.php on line 45

All I did was re-run the upgrade after increasing the memory, would that be the problem? Do I need to restart from scratch?

ImageImage


User avatar
Active Member

Posts

Joined
Tue Sep 13, 2011 3:07 am
Location - UK

Post by OSWorX » Wed Mar 16, 2016 3:58 am

sculptex wrote:okay I increased memory allowance by adding

Code: Select all

ini_set('memory_limit', '2048M');
How many users are allowed to increase the memory to such an abstruse value??

Full Stack Web Developer :: Dedicated OpenCart Development & Support DACH Region
Contact for Custom Work / Fast Support.


User avatar
Guru Member

Posts

Joined
Mon Jan 11, 2010 10:52 pm
Location - Austria

Post by pm-netti » Wed Mar 16, 2016 4:12 am

sculptex wrote:admin error
Notice: Error: Could not load template /home/xxx/public_html/admin/view/template/common/header! in /home/xxx/public_html/system/modification/system/engine/loader.php on line 45

All I did was re-run the upgrade after increasing the memory, would that be the problem? Do I need to restart from scratch?
My estimate: your filesystem is not clean Oc 2.2.
engine/loader.php line 72:

Code: Select all

$output = $template->render($route . '.tpl');
Your missing file extension .tpl from this line?

User avatar
Active Member

Posts

Joined
Sat Apr 07, 2012 11:22 pm
Location - Kittilä, Finland

Post by sculptex » Wed Mar 16, 2016 8:50 pm

OSWorX wrote:
sculptex wrote:okay I increased memory allowance by adding

Code: Select all

ini_set('memory_limit', '2048M');
How many users are allowed to increase the memory to such an abstruse value??
LOL - okay if you have your own server I guess!
I used that figure straight from an example I found which I thought was more than adequate. The reason for doing it this way is so many servers do not allow such modifications in php.ini or htaccess anyway and as its only needed for this script it seems the more sensible method.
Of course this figure is an overkill for most and perhaps when I have time I could try various figures until I hit the point it works and maybe these could be coded into the script so <10,000 records may be fine with 256M, up to 30,000 might require 512M etc.

ImageImage


User avatar
Active Member

Posts

Joined
Tue Sep 13, 2011 3:07 am
Location - UK

Post by sculptex » Wed Mar 16, 2016 8:54 pm

pm-netti wrote:My estimate: your filesystem is not clean Oc 2.2.
engine/loader.php line 72:

Code: Select all

$output = $template->render($route . '.tpl');
Your missing file extension .tpl from this line?
no, its exactly the same.

when I get a chance I'm gonna try the whole process again from a 'clean' 2.0.3.1 with just my customer database imported to isolate it from being anything else as theres a few mods in there.
Last edited by sculptex on Wed Mar 16, 2016 8:56 pm, edited 1 time in total.

ImageImage


User avatar
Active Member

Posts

Joined
Tue Sep 13, 2011 3:07 am
Location - UK
Who is online

Users browsing this forum: No registered users and 11 guests