Page 1 of 2

Importing Bulk products (import/export module)

Posted: Fri Dec 03, 2010 11:48 pm
by fishthetiger
Hi there community,

I have been having a problem for well over a month and after following advice found on this forum I have still not got much further even though it seemed like I have made slight progress.

PROBLEM, I am having a problem importing large amounts of products in one go.

Information:
Open cart: V1.4.9.1
clean install.
(mac osx)
Import/Export module

After trying to use the import export module it became apparent that the XLS file with my products was too large. From reading the forums I setup a local server with a local install of Open Cart 1.4.9.1 with the Import/Export Module.

I then started to upload my products (stripping down the xls file) in batches of around 6,000 to the local hosted shop. This worked fine. Products where showing on the store front.

Using phpMyAdmin I then exported specific tables (product, product_description, product_to_category, product_to_store) in CSV format.

I next accessed my database on my hosted (hosted shared server) site via phpMyAdmin and imported the specific data into the specific table(s) - Making sure I used CSV using LOAD DATA and changing Fields terminated by from (;) to (,).

This seemed to be working fine, The hosted database was showing signs of the records being uploaded. When I viewed on my live store the products where missing. When I accessed the admin section it confirmed that the products where not showing. When I viewed the data in the database in phpMyAdmin the records that had been added where actually blank (product id is there but everything else contains a 0 or NULL).

Im puzzled why my data shows fine locally but, once exported and imported to my live hosted shop it doesn't add the data correctly.


Does anyone have any advice? Has anyone got a different procedure I can try to have some success i uploading products? My php skills are basic to say the least. I have a friend who has created an Import script which he says should work but looking at it, it is a bit daunting and I'd rather exhaust other means which I do understand before I have to give up.

Hope I've added enough information.

Any help will be much appreciated.

Many thanks

Darren

Re: Importing Bulk products (import/export module)

Posted: Sat Dec 04, 2010 5:19 am
by justinv
When you export your products from your local server, maybe you should just export them as a mysql .sql backup file, and then you can reimport it as the same on your webserver.

I have a couple of import modules in my signature that can do straight from csv to OpenCart.

Re: Importing Bulk products (import/export module)

Posted: Sat Dec 04, 2010 6:13 pm
by fishthetiger
Cheers Justin, I shall take a look at those mods. I did try restoring from a backup sql (in open cart) but I will try importing from SQL. I have tried so many different ways I can't honestly remember if I have done this too. Thanks for your time though.

Mucho gracias :)

Re: Importing Bulk products (import/export module)

Posted: Sat Dec 04, 2010 6:19 pm
by fishthetiger
Hi Justin, I do remember seeing your CSVImport pro which I am gonna get my friend to purchase. It will save a lot of problem when he is maintaining the site himself. Keep up the good work.

Re: Importing Bulk products (import/export module)

Posted: Sun Dec 05, 2010 10:16 am
by justinv
Thanks :)

Re: Importing Bulk products (import/export module)

Posted: Tue Dec 07, 2010 1:17 am
by fishthetiger
Hi, I have been successfull in getting products into my database but they dont show in my store. Viewing the "products" in phpmyadmin confirms that the data is there. Has anybody got any pointers to nail this ever lasting problem? I am seriously thinking of having a seperate store for different manufacturers (bit drastic).

Also could anyone confirm the exact tables I need to edit to add bulk products? I just need to clarify that I have not been missing out some important tables.

Many thanks Peeps :D

Re: Importing Bulk products (import/export module)

Posted: Fri Dec 10, 2010 7:37 pm
by fishthetiger
Could any of community members confirm which tables I need to edit to add products to my store. These are the following tables that ive been editing:
products,
product_description,
product_to_category,
product_to_store

Also
has anybody got any strategies to upload lots of products. I currently have 21,000 products but need to have many more.

Thanks.

ps I am willing to buy any module or script if you can confirm that I will be able to populate my dbase with 50,000+ products.

Re: Importing Bulk products (import/export module)

Posted: Sat Dec 11, 2010 6:40 am
by justinv
My csv import PRO module will import as many as OpenCart, mysql and your server's php settings will handle/allow.

I'd be happy to support the module for 50,000 products.

Justin

Re: Importing Bulk products (import/export module)

Posted: Thu Dec 16, 2010 12:51 am
by fishthetiger
Looks like I have all the info I need cheers Justin. I shall purchase now.

Re: Importing Bulk products (import/export module)

Posted: Fri Dec 17, 2010 5:53 am
by fishthetiger
Hi there, Has anyone had success importing over 22k products and getting them into the store? I'm getting a little disheartened with the progress and even with investment of extentions I have hit a brick wall.

Has anyone had success with CSV Import Pro? I thought it would be my saviour but seeing as though its not, i'm guessing I have some major underlying problem.

Re: Importing Bulk products (import/export module)

Posted: Sat Dec 18, 2010 12:32 pm
by justinv
Maybe this will help, I've just created a screencast describing how to use csv import pro.

http://www.youtube.com/watch?v=-6k1_Qrq9_Y

Justin

Re: Importing Bulk products (import/export module)

Posted: Sun Dec 19, 2010 7:05 am
by poshook
HI justin.

your module looks very fine. I am working on webshop based on opencart for our stone shop and will need to have module which is able to transfer all product changes (in XML or CSV) from our accounting system to webshop and back in "real" time to provide our customers actual information about new products, new prices and availability.
Is this idea executable? Is Opencart able to do this with some special module?

Re: Importing Bulk products (import/export module)

Posted: Sun Dec 19, 2010 8:02 am
by justinv
Hi poshook,

you could probably get close to real time with batch runs with high frequency. I'm not sure what format your accounting system requires, but perhaps the google base feed or my anyfeed module will produce something it could use. Then you could use CSV Import PRO to import from your accounting system to OpenCart, and AnyFeed to export from OpenCart to your accounting system.

Alternatively, you could write a custom solution with our OpenCart RESTful API module as a basis, or someone from commercial section could write something for you. It certainly sounds like a task for automation.

Good luck!
Justin

Re: Importing Bulk products (import/export module)

Posted: Mon Dec 20, 2010 6:53 pm
by poshook
justinv wrote:Hi poshook,

Alternatively, you could write a custom solution with our OpenCart RESTful API module as a basis, or someone from commercial section could write something for you. It certainly sounds like a task for automation.

Good luck!
Justin
Would you able to do this automatic module?

Re: Importing Bulk products (import/export module)

Posted: Tue Dec 28, 2010 9:03 am
by oolongtea
Hi justinv,
How can I import pictures associated with the products by using your module?

thanks!

Re: Importing Bulk products (import/export module)

Posted: Tue Dec 28, 2010 3:43 pm
by justinv
If the pictures are on your server you can import them by including their path in the csv file you are importing - eg: data/image.jpg. If they are on another server you can import them using a tweak to the image.php file in OpenCart documented somewhere on this forum.

If they are on your computer the csv import module will not upload them for you.

thanks.

Re: Importing Bulk products (import/export module)

Posted: Sun Feb 20, 2011 11:10 pm
by calm
justinv wrote:If the pictures are on your server you can import them by including their path in the csv file you are importing - eg: data/image.jpg. If they are on another server you can import them using a tweak to the image.php file in OpenCart documented somewhere on this forum.

If they are on your computer the csv import module will not upload them for you.

thanks.
i m interested in your extension but i would like to know first if there is a way to upload product photos in my computer with your extension?

Re: Importing Bulk products (import/export module)

Posted: Mon Feb 21, 2011 6:05 am
by justinv
You would have to upload the product photos using your ftp server manually. The CSV Import module only updates the database to point at the image location, it does not actually fetch the image onto the server.

Re: Importing Bulk products (import/export module)

Posted: Mon Feb 21, 2011 9:27 pm
by calm
justinv wrote:You would have to upload the product photos using your ftp server manually. The CSV Import module only updates the database to point at the image location, it does not actually fetch the image onto the server.
Thanks, before buying, there is sth which i have to know, you said that we can import photos using a tweak to the image.php file in OpenCart, could you please provide the hyperlink or details? Thanks.

Re: Importing Bulk products (import/export module)

Posted: Tue Feb 22, 2011 5:21 am
by justinv
This can be done if your images are already on some server online.

Here is the thread, the third comment in is the method that I use: http://forum.opencart.com/viewtopic.php?f=20&t=23708