Post by easypeasyac » Sun Oct 06, 2024 6:13 pm

Hello;
My opencart version: 3.0.4.0 and I use SEO URL's. My store was opened at 26 August 2024. I submitted my sitemap to google on the same day. Google accepted my .xml sitemap without error.

My problem is
Google only crawl my hompage and only indexed my homepage. I manually submitted a few URLs to Google for trial purposes few time ago. Google indexed it all within a few days. However, Googlebot does not visit other pages than the homepage.

How can I get Google bot to crawl my website?

My robots.txt is:

Code: Select all

User-agent: *
Disallow: /*?page=$
Disallow: /*&page=$
Disallow: /*?sort=
Disallow: /*&sort=
Disallow: /*?order=
Disallow: /*&order=
Disallow: /*?limit=
Disallow: /*&limit=
Disallow: /*?filter_name=
Disallow: /*&filter_name=
Disallow: /*?filter_sub_category=
Disallow: /*&filter_sub_category=
Disallow: /*?filter_description=
Disallow: /*&filter_description=

Newbie

Posts

Joined
Thu Aug 22, 2024 2:51 am

Post by by mona » Sun Oct 06, 2024 10:45 pm

My store was opened at 26 August 2024
https://developers.google.com/search/do ... 20at%20all.

If you have submitted your sitemap and it is accepted then accept that you are 1 of millions of sites that were launched that day. Why would you assume it was instantaneous? Did you pay them to spend their resources on your site before anyone else's?

That said, sitemap.xml is often better for search engines than the user friendly sitemap . You have posted the robots file (which will throw lots of google notifications) this is because that file says don't crawl those pages - which is up to you but why ask google to crawl duplicate pages when it could be spending its valuable time on relevant pages?

DISCLAIMER:
You should not modify core files .. if you would like to donate a cup of coffee I will write it in a modification for you.


https://www.youtube.com/watch?v=zXIxDoCRc84


User avatar
Expert Member

Posts

Joined
Mon Jun 10, 2019 9:31 am

Post by easypeasyac » Sun Oct 06, 2024 11:06 pm

by mona wrote:
Sun Oct 06, 2024 10:45 pm
https://developers.google.com/search/do ... 20at%20all.

If you have submitted your sitemap and it is accepted then accept that you are 1 of millions of sites that were launched that day. Why would you assume it was instantaneous? Did you pay them to spend their resources on your site before anyone else's?

That said, sitemap.xml is often better for search engines than the user friendly sitemap . You have posted the robots file (which will throw lots of google notifications) this is because that file says don't crawl those pages - which is up to you but why ask google to crawl duplicate pages when it could be spending its valuable time on relevant pages?
Hello;
I have been away from SEO and website business for about 10 years. In my previous experiences, I was getting sites indexed much faster. I wondered if I was doing something wrong.

I'm sorry my English is not very good. I didn't understand exactly what you mean about my robots.txt file. Do you think it's problematic for Googlebot? This robots.txt is the original robots.txt file that comes with the Opencart installation.

Newbie

Posts

Joined
Thu Aug 22, 2024 2:51 am

Post by by mona » Sun Oct 06, 2024 11:18 pm

https://developers.google.com/search/do ... bots/intro

I presume that google has the same documentation in all languages.

You have not done "anything wrong", if your xml is accepted.
That is not to say you have done everything you can, but it can take a very long time.

Re robots.txt - https://support.bigcommerce.com/s/quest ... uage=en_US
pages is pages on for example lots of products on one category goes over more than one page
sort ie sort in category products
limit the same as sort above but the limit one 9, 12 100 etc.
filters are on the search page

DISCLAIMER:
You should not modify core files .. if you would like to donate a cup of coffee I will write it in a modification for you.


https://www.youtube.com/watch?v=zXIxDoCRc84


User avatar
Expert Member

Posts

Joined
Mon Jun 10, 2019 9:31 am

Post by nonnedelectari » Mon Oct 07, 2024 10:55 am

easypeasyac wrote:
Sun Oct 06, 2024 6:13 pm
Hello;
My opencart version: 3.0.4.0 and I use SEO URL's. My store was opened at 26 August 2024. I submitted my sitemap to google on the same day. Google accepted my .xml sitemap without error.

My problem is
Google only crawl my hompage and only indexed my homepage. I manually submitted a few URLs to Google for trial purposes few time ago. Google indexed it all within a few days. However, Googlebot does not visit other pages than the homepage.

How can I get Google bot to crawl my website?

My robots.txt is:

Code: Select all

User-agent: *
Disallow: /*?page=$
Disallow: /*&page=$
Disallow: /*?sort=
Disallow: /*&sort=
Disallow: /*?order=
Disallow: /*&order=
Disallow: /*?limit=
Disallow: /*&limit=
Disallow: /*?filter_name=
Disallow: /*&filter_name=
Disallow: /*?filter_sub_category=
Disallow: /*&filter_sub_category=
Disallow: /*?filter_description=
Disallow: /*&filter_description=
you can also add:

Sitemap: https://[YOUR-DOMAIN]/sitemap.xml
Host: [YOUR-DOMAIN]

by the way,
Google accepted my .xml sitemap without error.
but did you verify what your accepted sitemap actualy contains?

Active Member

Posts

Joined
Thu Mar 04, 2021 6:34 pm
Who is online

Users browsing this forum: No registered users and 19 guests