letxobnav wrote: ↑Sat Aug 17, 2019 10:10 amIf I need to refresh a huge amount of images in the cache because I change dimensions for some reason or the image quality setting and therefore delete the entire image cache or parts of it, I simply run a local spider on my site. That will trigger all pages and therefore all image generations as well as all other caches.
The site will take a hit for a few minutes while it is doing that but it is very easy.
Can you tell me the steps on how to run a local spider on my site to fix those issues?
There are no issues, image generation and caching is normal operations.
Bulk changing product images on a daily basis is not normal operations unless you run an online newspaper.
You can pull various opensource Spider software off the internet, install it and point it to your own url(s).
Bulk changing product images on a daily basis is not normal operations unless you run an online newspaper.
You can pull various opensource Spider software off the internet, install it and point it to your own url(s).
Crystal Light Centrum Taiwan
Extensions: MailQueue | SUKHR | VBoces
“Data security is paramount at [...], and we are committed to protecting the privacy of anyone who is associated with our [...]. We’ve made a lot of improvements and will continue to make them.”
When you know your life savings are gone.
The reason for why many hundreds of images change on a everyday basis its because I work with my computer and technology companies that change, deletes, and updates new products on a everyday basis, so when I load new updates using a thir party software that is why the images change everyday, so thats the reason why I need a software that can automatically run on my site so it can optimazise my images.
I wouldnt know which opensource spider software to choose from searching google, can you suggest one of those softwares for me?
I wouldnt know which opensource spider software to choose from searching google, can you suggest one of those softwares for me?
letxobnav wrote: ↑Sat Aug 17, 2019 11:08 amThere are no issues, image generation and caching is normal operations.
Bulk changing product images on a daily basis is not normal operations unless you run an online newspaper.
You can pull various opensource Spider software off the internet, install it and point it to your own url(s).
there are many:
https://github.com/hightman/pspider
https://github.com/mvdbos/php-spider
http://www.openwebspider.org/
https://github.com/hightman/pspider
https://github.com/mvdbos/php-spider
http://www.openwebspider.org/
Crystal Light Centrum Taiwan
Extensions: MailQueue | SUKHR | VBoces
“Data security is paramount at [...], and we are committed to protecting the privacy of anyone who is associated with our [...]. We’ve made a lot of improvements and will continue to make them.”
When you know your life savings are gone.
Thank you for your reply. I have looked at those links, but I think I will be lost if I install them, as I wouldnt know what that web spider will do for my website and how it would be useful, I think Im lost now...
letxobnav wrote: ↑Sat Aug 17, 2019 5:54 pmthere are many:
https://github.com/hightman/pspider
https://github.com/mvdbos/php-spider
http://www.openwebspider.org/
if you want to see what a spider does the easy way, install the trail version of "screaming-frog" SEO spider.
windows based with UI, it will show you exactly what info a spider can collect by simply requesting pages from your site.
https://www.screamingfrog.co.uk/seo-spider/
unfortunately the trial version only crawls 500 urls at a time and the options are disabled but you get the idea.
windows based with UI, it will show you exactly what info a spider can collect by simply requesting pages from your site.
https://www.screamingfrog.co.uk/seo-spider/
unfortunately the trial version only crawls 500 urls at a time and the options are disabled but you get the idea.
Crystal Light Centrum Taiwan
Extensions: MailQueue | SUKHR | VBoces
“Data security is paramount at [...], and we are committed to protecting the privacy of anyone who is associated with our [...]. We’ve made a lot of improvements and will continue to make them.”
When you know your life savings are gone.
Who is online
Users browsing this forum: No registered users and 49 guests
