Post by snappyfish » Mon Feb 13, 2012 5:08 am

Hi, I'm consistently receive this error:

Roboted images (135 items)
The image URLs submitted seem to be blocked by robots.txt. Google will not be able to display these images together with the products. Please change your robots.txt file to allow Google to download the image.

.. But its not blocked in robot.txt according to google web master tools.

Any help is appreciated.

Active Member

Posts

Joined
Thu Apr 14, 2011 4:36 am

Post by uksitebuilder » Tue Feb 14, 2012 1:51 am

Would need to see your robots.txt file if you have one and your .htaccess file as this could also be blocking images from being fetched

User avatar
Guru Member

Posts

Joined
Thu Jun 09, 2011 11:37 pm
Location - United Kindgom

Post by snappyfish » Tue Feb 14, 2012 2:40 am

Here they are..

Code: Select all

User-agent: *

    Disallow: /*?sort
    Disallow: /*&sort
    Disallow: /*?route=checkout/
    Disallow: /*?route=account/
    Disallow: /*?route=product/search
    Disallow: /*?page=1
    Disallow: /*&create=1
    Disallow: /*?route=affiliate/
    Disallow: /*?keyword
    Disallow: /*?av
    Disallow: /admin/
    Disallow: /system/
    Disallow: /catalog/
	Allow: /

Code: Select all

<IfModule mod_deflate.c>

############################################
## enable apache served files compression
## http://developer.yahoo.com/performance/rules.html#gzip

    # Insert filter on all content
    SetOutputFilter DEFLATE
    # Insert filter on selected content types only
    AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/javascript

    # Netscape 4.x has some problems...
    BrowserMatch ^Mozilla/4 gzip-only-text/html

    # Netscape 4.06-4.08 have some more problems
    BrowserMatch ^Mozilla/4\.0[678] no-gzip

    # MSIE masquerades as Netscape, but it is fine
    BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

    # Don't compress images
    SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary

    # Make sure proxies don't deliver the wrong content
    Header append Vary User-Agent env=!dont-vary

</IfModule>

Options +FollowSymlinks

    # Turn on Expires and set default to 0
    ExpiresActive On
    ExpiresDefault A0

    # Set up caching on media files for 5 weeks
    <FilesMatch "\.(flv|ico|pdf|avi|mov|ppt|doc|mp3|wmv|wav)$">
    ExpiresDefault A604800 
    Header append Cache-Control "public"
    </FilesMatch>

    # Set up caching on media files for 5 weeks
    <FilesMatch "\.(gif|jpg|jpeg|png|swf)$">
    ExpiresDefault A604800 
    Header append Cache-Control "public"
    </FilesMatch>

    # Set up 1 day caching on commonly updated files
    <FilesMatch "\.(xml|txt|html|js|css)$">
    ExpiresDefault A604800 
    Header append Cache-Control "proxy-revalidate"
    </FilesMatch>

    # Force no caching for dynamic files
    <FilesMatch "\.(php|cgi|pl|htm)$">
    ExpiresActive Off
    Header set Cache-Control "private, no-cache, no-store, proxy-revalidate, no-transform"
    Header set Pragma "no-cache"
    </FilesMatch>


# Prevent Directoy listing 
Options -Indexes

# Prevent Direct Access to files
<FilesMatch "\.(tpl|ini|log)">
 Order deny,allow
 Deny from all
</FilesMatch>

# SEO URL Settings
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^?]*) index.php?_route_=$1 [L,QSA]
RewriteCond %{QUERY_STRING} ^route=common/home$
RewriteCond %{REQUEST_METHOD} !^POST$
RewriteRule ^index\.php$ http://www.my_site.co.uk? [R=301,L]
RewriteCond %{http_host} ^my_site.co.uk [NC]
RewriteRule ^(.*)$ http://www.my_site.co.uk/ [R=301,NC]

### Additional Settings that may need to be enabled for some servers 
### Uncomment the commands by removing the # sign in front of it.
### If you get an "Internal Server Error 500" after enabling, then restore the # as this means your host doesn't allow that.

# 1. If your cart only allows you to add one item at a time, it is possible register_globals is on. This may work to disable it:
# php_flag register_globals off

# 2. If your cart has magic quotes enabled, This may work to disable it:
# php_flag magic_quotes_gpc Off


Active Member

Posts

Joined
Thu Apr 14, 2011 4:36 am

Post by uksitebuilder » Tue Feb 14, 2012 3:29 am

Looks fine to me.

I have just searched Google Help and seems you are not alone in his and it could well be a bug in the Google Shopping system

Are your images on the same domain or a subdomain ?

User avatar
Guru Member

Posts

Joined
Thu Jun 09, 2011 11:37 pm
Location - United Kindgom

Post by snappyfish » Tue Feb 14, 2012 3:43 am

uksitebuilder wrote:Looks fine to me.

Are your images on the same domain or a subdomain ?
There on the same domain, I have 3 websites on the same server on separate domains all been running open cart 1.5x for 6 months, This error in Google merchant does not go away for all 3 websites, on Google discussion they say its downtime or robots.txt But it's a consistent issue it seems..

Active Member

Posts

Joined
Thu Apr 14, 2011 4:36 am

Post by snappyfish » Mon Feb 20, 2012 9:12 pm

Seems a quick email to Google and things start to improve here's what they say.

"I reviewed your account and see that you previously had some crawl issues
which you have fixed and we are able to crawl your products. I also see
that you have recently submitted your data feed. Please wait for few days
for our system to re-crawl your items. Meanwhile, ensure that all your
item pages are up and active all the time and the URL's point directly to
the item."

Strange as I'm sure crawl issues dont last for 3 months.. But at least the "reboted images" percentage is dropping daily, Currently only 29% was 100% so a big improvement.

Active Member

Posts

Joined
Thu Apr 14, 2011 4:36 am

Post by OpenCode » Thu Jun 27, 2013 11:01 am

uksitebuilder wrote:Looks fine to me.

I have just searched Google Help and seems you are not alone in his and it could well be a bug in the Google Shopping system

Are your images on the same domain or a subdomain ?
I have may images in a subdomain and they are not been upload with the products I submit to google base

is there any help on how to fix this problem
OC 1.5.3.1
Thank you in advance

New member

Posts

Joined
Fri Jun 07, 2013 9:31 am

Post by gauravchawla18 » Tue Feb 18, 2014 2:52 am

I am also facing the same issue but till date there is no help.
Did anyone got the solution.

Thanks

New member

Posts

Joined
Mon May 06, 2013 2:00 am
Who is online

Users browsing this forum: No registered users and 39 guests