Post by JNeuhoff » Wed Mar 16, 2011 7:27 am

Unfortunately, even PHPExcel (used by the Export/Import tool in Opencart 1.4.9.3) is as memory hungry as the PHPExcelReader used in older versions.

I may end up using an incremental import feature in future versions. Or maybe use CSV instead of XLS, however, you'd loose a lot of the functionality coming with XLS, such as worksheets, frozen view panes, preset column widths, etc.

MHC Web Design
Override Engine * Integrated VQMod * Unused Images Manager * Instant Option Price Calculator * TrustPilot Reviews * Google Rich Snippets * Google Tag Manager * Export/Import Tool * Template Switcher PHP/Twig


User avatar
Expert Member

Posts

Joined
Wed Dec 05, 2007 3:38 am


Post by madimar » Wed Mar 16, 2011 7:08 pm

Thank you for your prompt feedback!
Some personal ideas/options :drunk: to improve things:

1) Continuing investigating with PHPExcel community if optimizations are possible for our purposes (e.g. not importing not necessary info, like cells format or similar).

2) Maintaining the complete Excel sheet, but using an incremental import approach in the script. I better explain my idea (I don't know if feasible):
a) upload the entire excel sheet on the server
b) open the file, then importing a part of it, updating the MySql DB, freeing memory (closing the file if it is necessary) and repeating iteratively partial import up to the completion of the file.
This approach could be maybe also beneficial for MySql DB, avoiding to bombarding it with massive queries.

3) Considering a mix model Excel/CSV. Starting from current situation, just add a CSV import of CSV files the user can obtain directly from Excel saving the tables separately (or via a simple Excel/VBA Macro)

For step 3, if it could be helpful and necessary, I could help preparing the Excel VBA MAcro to export CSV from your excel template :joker: . CSV files could be produced and zipped in one export archive directly from excel, then imported on the Web server. Only CSV import (after extracting from ZIP) and MySql DB update should be done so.

As you can see all these methods don't touch export process, considering it is actually not a real problem.

What do you think about?

Max

-----------------------------------------------------------------------
My last mods: Partita IVA e CF | Pro EU VAT Number | Sales Agents | Pricelist Pro
-----------------------------------------------------------------------


User avatar
Active Member

Posts

Joined
Thu Sep 24, 2009 6:27 pm


Post by asy1mpo » Thu Mar 17, 2011 7:51 am

Hi

I have followed the instructions and in the User Groups there is no checkbox for tool/export??

What have I done wrong??

Newbie

Posts

Joined
Thu Mar 17, 2011 7:50 am

Post by trapson » Thu Mar 17, 2011 10:07 pm

madimar wrote:Thank you for your prompt feedback!
Some personal ideas/options :drunk: to improve things:

1) Continuing investigating with PHPExcel community if optimizations are possible for our purposes (e.g. not importing not necessary info, like cells format or similar).

2) Maintaining the complete Excel sheet, but using an incremental import approach in the script. I better explain my idea (I don't know if feasible):
a) upload the entire excel sheet on the server
b) open the file, then importing a part of it, updating the MySql DB, freeing memory (closing the file if it is necessary) and repeating iteratively partial import up to the completion of the file.
This approach could be maybe also beneficial for MySql DB, avoiding to bombarding it with massive queries.

3) Considering a mix model Excel/CSV. Starting from current situation, just add a CSV import of CSV files the user can obtain directly from Excel saving the tables separately (or via a simple Excel/VBA Macro)

For step 3, if it could be helpful and necessary, I could help preparing the Excel VBA MAcro to export CSV from your excel template :joker: . CSV files could be produced and zipped in one export archive directly from excel, then imported on the Web server. Only CSV import (after extracting from ZIP) and MySql DB update should be done so.

As you can see all these methods don't touch export process, considering it is actually not a real problem.

What do you think about?

Max
I have also dicussed this problem with many webhosting company's and they al say that a memorylimit of 64 mb is more than enough some are willing to upgrade it to 128mb but most even wont because then the script is to much asking from the server.

I think option 2 is a good option (only incremental import is not an option with weekly changing prices for large product groups) bu i do not know if it is possible.

Opencart 1.4.9.3


New member

Posts

Joined
Fri Sep 24, 2010 7:10 pm

Post by wangxh » Fri Mar 18, 2011 3:28 pm


opencart-1-4-9-x-24-export-import




when i import my exl ,only 548kb, the info appear:

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.

Newbie

Posts

Joined
Wed Mar 09, 2011 3:56 pm

Post by JNeuhoff » Sat Mar 19, 2011 7:51 am

Internal Server Error
So why aren't you checking the server's error_log then to find out more about the nature of the error?

MHC Web Design
Override Engine * Integrated VQMod * Unused Images Manager * Instant Option Price Calculator * TrustPilot Reviews * Google Rich Snippets * Google Tag Manager * Export/Import Tool * Template Switcher PHP/Twig


User avatar
Expert Member

Posts

Joined
Wed Dec 05, 2007 3:38 am


Post by huangyuanpu » Mon Mar 21, 2011 11:51 pm

Hi JNeuhoff, I installed your Export/Import Tool, it is a great tool, thank you very much!
I just have one question, my website has 28 products, then I tested to Import a Excel (with juse one product), I can import the product, but then I found my website only has one product, the other 28 products were lost. Is there a way to solve this problem?
I only want to import the new products, but I also must import the "old" products everytime?
Please help me, thanks a lot!

Newbie

Posts

Joined
Sun Mar 20, 2011 12:37 pm

Post by klenghost » Tue Mar 22, 2011 1:59 am

thanks for improving the tool

but we are getting following error message : (while trying to import xls)

Fatal Error: Call to undefined method PHPExcel_Reader_CSV::setReadDataOnly() in /home/dynamicr/domains/domeinnaam.nl/public_html/admin/model/tool/export.php on line 1170


kind regards

Newbie

Posts

Joined
Fri Feb 04, 2011 12:20 am

Post by JNeuhoff » Tue Mar 22, 2011 2:07 am

klenghost wrote:thanks for improving the tool

but we are getting following error message : (while trying to import xls)

Fatal Error: Call to undefined method PHPExcel_Reader_CSV::setReadDataOnly() in /home/dynamicr/domains/domeinnaam.nl/public_html/admin/model/tool/export.php on line 1170


kind regards
Are you using OpenOffice Calc? or MS Excel?

If using OpenOffice try to do an explicit Save As 97/2000/XP MS Excel format.

MHC Web Design
Override Engine * Integrated VQMod * Unused Images Manager * Instant Option Price Calculator * TrustPilot Reviews * Google Rich Snippets * Google Tag Manager * Export/Import Tool * Template Switcher PHP/Twig


User avatar
Expert Member

Posts

Joined
Wed Dec 05, 2007 3:38 am


Post by JNeuhoff » Tue Mar 22, 2011 2:10 am

huangyuanpu wrote:Hi JNeuhoff, I installed your Export/Import Tool, it is a great tool, thank you very much!
I just have one question, my website has 28 products, then I tested to Import a Excel (with juse one product), I can import the product, but then I found my website only has one product, the other 28 products were lost. Is there a way to solve this problem?
I only want to import the new products, but I also must import the "old" products everytime?
Please help me, thanks a lot!
Send me a PM with your XLS file, otherwise I can't reproduce your error on my test server.

MHC Web Design
Override Engine * Integrated VQMod * Unused Images Manager * Instant Option Price Calculator * TrustPilot Reviews * Google Rich Snippets * Google Tag Manager * Export/Import Tool * Template Switcher PHP/Twig


User avatar
Expert Member

Posts

Joined
Wed Dec 05, 2007 3:38 am


Post by seasons » Tue Mar 22, 2011 3:45 am

Hello, this is the CSV file. I hope you can help us.
Thank you very much for your time.

Newbie

Posts

Joined
Tue Mar 22, 2011 3:41 am

Post by JNeuhoff » Tue Mar 22, 2011 6:48 am

seasons wrote:Hello, this is the CSV file. I hope you can help us.
Thank you very much for your time.
Sorry, can't open RAR files. Please use ZIP or TAR.GZ.

Also, The Export/Import does not support CSV files, only XLS files.

MHC Web Design
Override Engine * Integrated VQMod * Unused Images Manager * Instant Option Price Calculator * TrustPilot Reviews * Google Rich Snippets * Google Tag Manager * Export/Import Tool * Template Switcher PHP/Twig


User avatar
Expert Member

Posts

Joined
Wed Dec 05, 2007 3:38 am


Post by huangyuanpu » Tue Mar 22, 2011 7:18 am

JNeuhoff wrote:
huangyuanpu wrote:Hi JNeuhoff, I installed your Export/Import Tool, it is a great tool, thank you very much!
I just have one question, my website has 28 products, then I tested to Import a Excel (with juse one product), I can import the product, but then I found my website only has one product, the other 28 products were lost. Is there a way to solve this problem?
I only want to import the new products, but I also must import the "old" products everytime?
Please help me, thanks a lot!
Send me a PM with your XLS file, otherwise I can't reproduce your error on my test server.
Thanks, please check the zip file, when I upload it (only one product), it "overwrite" all other products on the website, I do not know why.

Newbie

Posts

Joined
Sun Mar 20, 2011 12:37 pm

Post by madimar » Tue Mar 22, 2011 7:56 am

huang, import/export tool works in that way!!! Full export and full import. If you import a file excel with only one product, your DB will be over written and it will contain only that product!

-----------------------------------------------------------------------
My last mods: Partita IVA e CF | Pro EU VAT Number | Sales Agents | Pricelist Pro
-----------------------------------------------------------------------


User avatar
Active Member

Posts

Joined
Thu Sep 24, 2009 6:27 pm


Post by huangyuanpu » Tue Mar 22, 2011 9:56 am

madimar wrote:huang, import/export tool works in that way!!! Full export and full import. If you import a file excel with only one product, your DB will be over written and it will contain only that product!
Thanks for your reply, in this case, it's a little unconvenient to use the tool. Is there any way to import the new products without the old one, but at the same time donot overwrite the products already on the website?

Thanks a lot.

Newbie

Posts

Joined
Sun Mar 20, 2011 12:37 pm

Post by madimar » Tue Mar 22, 2011 3:41 pm

At the moment it is not possible.
IMHO, generally speaking I don't see it as an inconvenient, with the only exception it becomes a serious limit when your products catalogue grows (I'm currently facing issues with 4500 products due to memory requirements, more than 64MB).
As I was saying, generally it is not an inconvenient because, you can export your db, add new products and re-import, obtaining the desired result. Working on the entire catalogue allows to batch modify fields (e.g. prices) and not only add new products.

ciao

Max

-----------------------------------------------------------------------
My last mods: Partita IVA e CF | Pro EU VAT Number | Sales Agents | Pricelist Pro
-----------------------------------------------------------------------


User avatar
Active Member

Posts

Joined
Thu Sep 24, 2009 6:27 pm


Post by madimar » Thu Mar 24, 2011 3:47 pm

JNeuhoff, any feedback on my proposal of improvements?
I appreciate much your opinion,

Max

Sent from my Nexus One using Tapatalk

-----------------------------------------------------------------------
My last mods: Partita IVA e CF | Pro EU VAT Number | Sales Agents | Pricelist Pro
-----------------------------------------------------------------------


User avatar
Active Member

Posts

Joined
Thu Sep 24, 2009 6:27 pm


Post by interaction » Sun Mar 27, 2011 8:38 pm

Hi Jneuhoff

I have use this import/export ver 1.4.9.3 module recently for my opencart 1.4.9.3 version

The export function is working well except for the import function. Whenever I import the XLS file, it will direct me to a blank page after several minutes. I have less than thousand products and I use microsoft excel for edit

Is there any way to fix this?

Regards
Johnson

Newbie

Posts

Joined
Mon Dec 06, 2010 10:53 am

Post by JNeuhoff » Sun Mar 27, 2011 9:07 pm

interaction wrote:Hi Jneuhoff

I have use this import/export ver 1.4.9.3 module recently for my opencart 1.4.9.3 version

The export function is working well except for the import function. Whenever I import the XLS file, it will direct me to a blank page after several minutes. I have less than thousand products and I use microsoft excel for edit

Is there any way to fix this?

Regards
Johnson
This is likely an issue with your PHP settings, e.g. timeout or the maximum allowed upload file size. Check your admin/php.ini fior this. Also, check your error messages in your server's error_log and Opencart's system/logs/error.txt for any hints as to why the import has problems.

User avatar
Expert Member

Posts

Joined
Wed Dec 05, 2007 3:38 am


Post by JNeuhoff » Sun Mar 27, 2011 9:11 pm

madimar wrote:JNeuhoff, any feedback on my proposal of improvements?
I appreciate much your opinion,

Max

Sent from my Nexus One using Tapatalk
A version of the Export/Import tool with support for an incremental import for adding or updating products and categories is on my to-do list, though it may take a while before I get around to implementing it because I am quite busy at the moment with some other Opencart-based projects.

MHC Web Design
Override Engine * Integrated VQMod * Unused Images Manager * Instant Option Price Calculator * TrustPilot Reviews * Google Rich Snippets * Google Tag Manager * Export/Import Tool * Template Switcher PHP/Twig


User avatar
Expert Member

Posts

Joined
Wed Dec 05, 2007 3:38 am

Who is online

Users browsing this forum: No registered users and 11 guests