Import problem

Quote

Post by passwordsas » Thu Sep 30, 2010 5:46 pm

Hi everybody.

I have a big problem.
When I try to import a file Excel bigger than 2900 Kb, the application doesn't import it.
The system doesn't "say" anything. No error messages appear, and neither appears the "successfull upload" message.

When I use a file smaller than 2900 Kb everything works good.

I have the 1.4.8b version of Open Cart.

Thanks a lot in advance for your help! :)
passwordsas
Newbie
Posts
3
Joined
Thu Sep 30, 2010 5:40 pm
Top

Re: Import problem

Quote

Post by JAY6390 » Thu Sep 30, 2010 6:10 pm

Couple of things that it could be that I can think of off the top of my head

1) The post max file size setting in php is too small for your upload
or
2) memory limit gets exceeded, so you will need to increase that

If I had to put money on it, I would say it's the first

Image

User avatar
JAY6390
Guru Member
Posts
5086
Joined
Wed May 26, 2010 11:47 pm
Location - United Kingdom
Top

Re: Import problem

Quote

Post by passwordsas » Fri Oct 01, 2010 1:23 am

First of all, thanks for your help... but... you''d better spare your money! ;D
The size is set to 40 Mb, and my file it's shorter than that.
Maybe it could be a matter of memory. I'm asking the hosting service of my site.
passwordsas
Newbie
Posts
3
Joined
Thu Sep 30, 2010 5:40 pm
Top

Re: Import problem

Quote

Post by JNeuhoff » Fri Oct 01, 2010 7:59 pm

You should still check your memory_limit. The Import from a spreadsheet file can be quite memory hungry because it first parses the whole file into memory beforing accessing the worksheets' cells.

Export/Import Tool * SpamBot Buster * Unused Images Manager * Instant Option Price Calculator * Number Option * Google Tag Manager * Survey Plus * OpenTwig

User avatar
JNeuhoff
Guru Member
Posts
7456
Joined
Wed Dec 05, 2007 3:38 am
Top

Re: Import problem

Quote

Post by passwordsas » Sat Oct 02, 2010 5:18 pm

Yes, we think it's a matter of memory, though no error message appears.
Does anyone know if it derives from my host settings or if is there any parameter that I could modify inside OpenCart?
Anyone here use the impot function to load a large amount of data?
passwordsas
Newbie
Posts
3
Joined
Thu Sep 30, 2010 5:40 pm
Top

Re: Import problem

Quote

Post by JNeuhoff » Sun Oct 03, 2010 4:34 am

passwordsas wrote:Yes, we think it's a matter of memory, though no error message appears.
Does anyone know if it derives from my host settings or if is there any parameter that I could modify inside OpenCart?
Anyone here use the impot function to load a large amount of data?
You should check your memory_limit entry in your admin/php.ini file, should be something like

memory_limit = xxxxM;

where xxxx is the amount in MBytes of your needed memory.

Note that with some web hosts, even with a higher memory_limit PHP setting, it won't accept it because they might restrict the allowed process, space especially on a shared web server.

Export/Import Tool * SpamBot Buster * Unused Images Manager * Instant Option Price Calculator * Number Option * Google Tag Manager * Survey Plus * OpenTwig

User avatar
JNeuhoff
Guru Member
Posts
7456
Joined
Wed Dec 05, 2007 3:38 am
Top

Re: Import problem

Quote

Post by EricP » Tue Oct 19, 2010 9:39 am

I'm having the same problem, if I truncate the XLS to half its size, it works. If I have the full 4,000+ items, it fails silently.

Here are the site settings:

max_execution_time = 180
memory_limit = 128M
upload_max_filesize = 10M
post_max_size = 8M

The file is only 2 meg.

I also disabled my firewall and set IE to have the keep alive timeout at 2 minutes, and it fails well before then.
EricP
Newbie
Posts
12
Joined
Tue Oct 19, 2010 9:34 am
Top

Re: Import problem

Quote

Post by JNeuhoff » Tue Oct 19, 2010 4:50 pm

Can you get it to work on a local PC with your current PHP settings?

As I said, some web hosts restrict the allowed process memory, in which case a higher PHP memory_limit would be just ignored. Also, you can check whether your PHP settings are really accepted by your remoted server by executing a phpinfo() call.

Export/Import Tool * SpamBot Buster * Unused Images Manager * Instant Option Price Calculator * Number Option * Google Tag Manager * Survey Plus * OpenTwig

User avatar
JNeuhoff
Guru Member
Posts
7456
Joined
Wed Dec 05, 2007 3:38 am
Top

Re: Import problem

Quote

Post by EricP » Wed Oct 20, 2010 11:41 am

Just for kicks I did another experiment by halving the data file. It worked the first time, and subsequent tests with the exact same file came back with nothing. If the file is the same and nothing is updated, is it supposed to tell you that nothing was changed? Or should it come back and say that the import was successful?
EricP
Newbie
Posts
12
Joined
Tue Oct 19, 2010 9:34 am
Top

Re: Import problem

Quote

Post by JNeuhoff » Wed Oct 20, 2010 4:49 pm

EricP wrote:Just for kicks I did another experiment by halving the data file. It worked the first time, and subsequent tests with the exact same file came back with nothing. If the file is the same and nothing is updated, is it supposed to tell you that nothing was changed? Or should it come back and say that the import was successful?
Please send me your XLS file, and I can test it.

Export/Import Tool * SpamBot Buster * Unused Images Manager * Instant Option Price Calculator * Number Option * Google Tag Manager * Survey Plus * OpenTwig

User avatar
JNeuhoff
Guru Member
Posts
7456
Joined
Wed Dec 05, 2007 3:38 am
Top

Re: Import problem

Quote

Post by EricP » Tue Oct 26, 2010 11:04 am

I sent it to you last week - did you get it?

I was poking around export.php and noticed the following code:

Code: Select all

	switch ($errno) {
		case E_NOTICE:
		case E_USER_NOTICE:
			$errors = "Notice";
			break;
		case E_WARNING:
		case E_USER_WARNING:
			$errors = "Warning";
			break;
		case E_ERROR:
		case E_USER_ERROR:
			$errors = "Fatal Error";
			break;
		default:
			$errors = "Unknown";
			break;
	}
		
	if (($errors=='Warning') || ($errors=='Unknown')) {
		return true;
	}
Specifically how every error not handled defaults to Unknown and if it's Unknown then you assume that the import or export succeeded. Could the problem be in there?
EricP
Newbie
Posts
12
Joined
Tue Oct 19, 2010 9:34 am
Top

Re: Import problem

Quote

Post by JNeuhoff » Wed Oct 27, 2010 6:23 am

I sent it to you last week - did you get it?
Yes, I got it and tested it. I can import and export it just fine.
Using the following PHP settings:

magic_quotes_gpc = Off;

register_globals = Off;

default_charset = UTF-8;

memory_limit = 64M;

max_execution_time = 18000;

max_upload_filesize = 999M;

safe_mode = Off;

mysql.connect_timeout = 20;

session.use_cookies = On;

session.use_trans_sid = Off;

session.gc_maxlifetime = 12000000;

Export/Import Tool * SpamBot Buster * Unused Images Manager * Instant Option Price Calculator * Number Option * Google Tag Manager * Survey Plus * OpenTwig

User avatar
JNeuhoff
Guru Member
Posts
7456
Joined
Wed Dec 05, 2007 3:38 am
Top

Re: Import problem

Quote

Post by EricP » Mon Nov 01, 2010 9:10 am

Nope, still doesn't work. When it fails, it returns back to the screen where it asks for the upload file name, and does not return a success or fail message. When I check, none of the products have been updated. It is dying right in the middle somewhere. I even shortened the file some more and it's still dying.

Is there any way to have it log every step that it does so that I can see how far it has gotten before the error? I tried modifying the code and having it write to the error log every time it reads a line but that didn't seem to work.
EricP
Newbie
Posts
12
Joined
Tue Oct 19, 2010 9:34 am
Top

Re: Import problem

Quote

Post by JNeuhoff » Mon Nov 01, 2010 11:48 pm

Does work on your local PC? Looks like there might be an issue with your remote web server's configuration of some sort.

Export/Import Tool * SpamBot Buster * Unused Images Manager * Instant Option Price Calculator * Number Option * Google Tag Manager * Survey Plus * OpenTwig

User avatar
JNeuhoff
Guru Member
Posts
7456
Joined
Wed Dec 05, 2007 3:38 am
Top

Re: Import problem

Quote

Post by EricP » Tue Nov 02, 2010 9:17 am

I don't have OpenCart locally, but you tested the same file above and said it worked.

We've had them change all the items to what you recommended, and it still just dies.

Here's where it does work. I was able to delete everything except one item, and that worked great. It seems to be fine if I remove a bunch of stuff or add a bunch of stuff. If I'm just doing a mass update of existing items, that's when it fails.

If I knew exactly where it was failing then I could tell the host what to change.
EricP
Newbie
Posts
12
Joined
Tue Oct 19, 2010 9:34 am
Top

Re: Import problem

Quote

Post by JNeuhoff » Tue Nov 02, 2010 10:46 pm

EricP wrote:I don't have OpenCart locally, but you tested the same file above and said it worked.

We've had them change all the items to what you recommended, and it still just dies.

Here's where it does work. I was able to delete everything except one item, and that worked great. It seems to be fine if I remove a bunch of stuff or add a bunch of stuff. If I'm just doing a mass update of existing items, that's when it fails.

If I knew exactly where it was failing then I could tell the host what to change.
Ask your web host whether it supports a sufficient memory_limit PHP setting. On some web hosts, even when it appears to accepts the higher memory_limit, it still won't work because of the overall PHP process memory being more limited.

Export/Import Tool * SpamBot Buster * Unused Images Manager * Instant Option Price Calculator * Number Option * Google Tag Manager * Survey Plus * OpenTwig

User avatar
JNeuhoff
Guru Member
Posts
7456
Joined
Wed Dec 05, 2007 3:38 am
Top

Re: Import problem

Quote

Post by EricP » Fri Nov 05, 2010 11:08 pm

They changed it to 128m and it's still not working.

If I import a 1 line spreadsheet, thus deleting almost everything, it works.

If I then import a 4,000 line spreadsheet, thus adding everything as new, it works.

If I then import a 4,000 line spreadsheet with the same items but with a few changes, it fails silently. To me this sounds like a problem with update statements hitting MySQL, since deletes and inserts appear to be working fine.
EricP
Newbie
Posts
12
Joined
Tue Oct 19, 2010 9:34 am
Top

Re: Import problem

Quote

Post by JNeuhoff » Sat Nov 06, 2010 12:24 am

Send me your last 2 XLS files with the 4000 products, and I can try to test them on my test server.

Export/Import Tool * SpamBot Buster * Unused Images Manager * Instant Option Price Calculator * Number Option * Google Tag Manager * Survey Plus * OpenTwig

User avatar
JNeuhoff
Guru Member
Posts
7456
Joined
Wed Dec 05, 2007 3:38 am
Top

Re: Import problem

Quote

Post by Limegreen » Sat Nov 06, 2010 12:51 am

Hi JNeuhoff,

I'm just wondering (and this is really basic I guess), how do i format the spreadsheet so it is compatible with OpenCart?

i.e. do I delete any columns that OpenCart doesn't need, do I have to give the columns the exact same title as the website, can I upload images to the spreadsheet etc...etc...

Appreciate your help as I have about 21,000 products to upload.


Kind Regards


Limegreen
http://www.limegreenofficeproducts.com
Limegreen
Newbie
Posts
14
Joined
Thu Nov 04, 2010 4:57 pm
Top

Re: Import problem

Quote

Post by EricP » Sat Nov 06, 2010 12:57 am

I'll send the files this evening.

Limegreen, I take the export that it generates, modify it, then re-upload it. Leave any unneeded columns blank. But you're better off adding a few items with your various stock statuses, quantities, descriptions, etc. to use as an example so that you can set these values appropriately for new items. Recently my client discovered that many items in his store are not to be sold online due to his contract agreements, so I had to create a new stock status called Store Pickup and apply that to the items. Also I had to hide the Add To Cart button due to specific language in those contracts stating that they must not be added to a shopping cart.
EricP
Newbie
Posts
12
Joined
Tue Oct 19, 2010 9:34 am
Top
Post Reply
Return to “Modules”
Who is online

Users browsing this forum: No registered users and 6 guests