Post by Qphoria » Thu Aug 28, 2008 7:11 am

In the process of creating a new shipping module for USPS, It became very evident to me that there was a delay in the checkout process when using my new usps live quote shipping module.

Of course I expected a delay on the checkout_shipping page, as that has to run the getQuotes process and wait for the reponses from the usps server and any other servers that might be enabled.

After that however, the delay was just as long on the checkout_payment and checkout_confirm pages on local server.

So using Nusphere as my debugger, and switching back to zone shipping. I stepped through the pages and found the follow to be quite interesting:
  • checkout_shipping - goes through the getQuotes process one time to find ALL available mailing options and list them out for the customer to chooose. Once chosen, it then sets that to the shipping_method. That is proper behavior.
  • checkout_payment - goes through the "getQuotes" process one extra time to get the price for the shipping option that was selected on the shipping page as part of a verification step. This is completely unnecessary as the postage cost should have already been saved in the session from the last page.
  • checkout_confirm - goes through the "getQuotes" process SIX more times!! I haven't finished testing yet, but it seems to loop through once for each element of the array (id, title, cost, text, taxclass, etc). Pure craziness :P
I am looking into how to fix this and will post in in the bugs section as well, but that's pretty crazy and explains why it's causing such a performance hit.  It's easy to overlook for db-based shipping methods like flat and zone because they are using a local db to look it up, so it almost seems like there is no problem. But the debugger tells a different story, especially when using API live lookup.
       if ($cart->hasShipping()) {
        if (!$session->get('shipping_method')) {
          $response->redirect($url->ssl('checkout_shipping'));
        }

        if (!$shipping->getQuote($session->get('shipping_method'))) {
          $response->redirect($url->ssl('checkout_shipping'));
        }


        if (!$session->get('shipping_address_id')) {
          $response->redirect($url->ssl('checkout_address', 'shipping'));
        }
      } else {
        $session->delete('shipping_address_id');
        $session->delete('shipping_method');
      }
This seems to be the first culprit. Seem opencart is checking if there isn't a shipping method in the session, then go back to checkout_shipping, which is fine. But then does a redundant check if its not in the shipping class by getting the quotes again. This I believe is already being validated before leaving the checkout_shipping page.

Then later in the shipping.php library there are 3 more calls:

[php]
function getTitle($key) {
$data = $this->getQuote($key);

return (isset($data['title']) ? $data['title'] : NULL);
}

function getCost($key) {
$data = $this->getQuote($key);

return (isset($data['cost']) ? $data['cost'] : NULL);
}

function getTaxClassId($key) {
$data = $this->getQuote($key);

return (isset($data['tax_class_id']) ? $data['tax_class_id'] : NULL);
}
[/php]

This looks like it all needs to be reworked to just get it out of the existing session or another global that already has it set.

I'll post with more as I find more.
Last edited by Qphoria on Thu Aug 28, 2008 7:13 am, edited 1 time in total.

Image
Donate!|OpenCart Basics|GeoZones
Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by Luvz2drv » Thu Aug 28, 2008 10:15 am

yes if you searched this has been brought up before.. by bruce in the canada post thread

mine solved it for my module with a few checked and balances :)

Global Moderator

Posts

Joined
Fri Mar 21, 2008 10:58 am

Post by JNeuhoff » Thu Aug 28, 2008 5:21 pm

I noticed the same performance issue for the Australia Post shipping module.

I think the best option is to change the getQuote method in a way that it uses a session-cached object for the method_data it returns after an online query to the rate calculator. When I have time I'll implement the session-cache solution for my shipping module, hopefully that will resolve the performance bottleneck.

Override Engine * Integrated VQMod * Unused Images Manager * Instant Option Price Calculator * Number Option * Google Rich Snippets * Google Tag Manager * Export/Import Tool * SpamBot Buster * Survey Plus


User avatar
Expert Member

Posts

Joined
Wed Dec 05, 2007 3:38 am


Post by Qphoria » Thu Aug 28, 2008 6:59 pm

Luvz2drv wrote: yes if you searched this has been brought up before.. by bruce in the canada post thread

mine solved it for my module with a few checked and balances :)

Ah sorry about that...
I searched "getQuote" and "multiple quote" before i posted and didn't see anything.
Just searched "canada post" and saw only one comment about flat rate shipping.
I probably just missed the keywords.

Image
Donate!|OpenCart Basics|GeoZones
Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by bruce » Thu Aug 28, 2008 9:00 pm

Hi Qphoria,

The post jonathon is referring to is at here. The analysis may help you. However, I have not attempted to produce the fix mechanism suggested there as this problem currently does not (significantly) affect any of my installations.

cheers

Bruce

Active Member

Posts

Joined
Wed Dec 12, 2007 2:26 pm

Post by Qphoria » Thu Aug 28, 2008 10:39 pm

Ah thank you bruce. Your idea for having the module persist itself to the session was just the ticket too.

It's a simple workaround too:

The first step under my quote() function I added:

Code: Select all

    if ($this->session->get('usps_quotes')) {
      return $this->session->get('usps_quotes');
    }
and at the bottom I changed

Code: Select all

return $method_data;
to:

Code: Select all

$this->session->set('usps_quotes', $method_data);
return $method_data;
Works like a charm!

Thx all!
Last edited by Qphoria on Thu Aug 28, 2008 10:45 pm, edited 1 time in total.

Image
Donate!|OpenCart Basics|GeoZones
Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by Luvz2drv » Fri Aug 29, 2008 4:17 am

np, that was the easiest way i found as well was check for a session of it first. :)

sorry not gonna give all my performance tricks away :)

Jonathon

Global Moderator

Posts

Joined
Fri Mar 21, 2008 10:58 am

Post by bruce » Fri Aug 29, 2008 8:37 am

Be careful guys. Have another look over the post I linked for you. There is some discussion about persisting the parameters used to create the quotes. If they change and you do not recalculate, the persisted value returned from the session will be wrong.

Active Member

Posts

Joined
Wed Dec 12, 2007 2:26 pm

Post by Qphoria » Fri Aug 29, 2008 9:19 am

Ok.. so I got a check for:
- product change
- cart qty change
- cart weight change

Still need to add
- product option changes

Any other changes you can think of?

Image
Donate!|OpenCart Basics|GeoZones
Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by bruce » Fri Aug 29, 2008 9:55 am

Changes to the settings of your shipping extension can occur during a user session and they should be checked too.

Active Member

Posts

Joined
Wed Dec 12, 2007 2:26 pm

Post by Qphoria » Fri Aug 29, 2008 10:18 am

you mean like if a customer starts to checkout.. stops for 10 min.. and in that time the store owner changes some options?

Image
Donate!|OpenCart Basics|GeoZones
Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by JNeuhoff » Fri Aug 29, 2008 5:16 pm

you mean like if a customer starts to checkout.. stops for 10 min.. and in that time the store owner changes some options?
Yes, I think that's what he meant.

I don't think we should go overboard here.
The main settings in your shipping modules are probably the ones for the supported delivery types for which you have checkboxes in your admin area, and changing these by the admin is such a rare event that it can be neglected here. For that matter, even the USPS online API can change its delivery rates and you are still using an old session-cached value....

What's important is to make sure that the session object is uniquely identified, and the identifier for it must include cart weight and qty and whatever else is relevant for calls to the USPS online API. In the case of the Australia Post Online API the only relevant values are the cart weight and delivery type, and I always use the same default dimensions for length,width,height because OpenCart doesn't support the latter on an individual product basis anyway.

Override Engine * Integrated VQMod * Unused Images Manager * Instant Option Price Calculator * Number Option * Google Rich Snippets * Google Tag Manager * Export/Import Tool * SpamBot Buster * Survey Plus


User avatar
Expert Member

Posts

Joined
Wed Dec 05, 2007 3:38 am


Post by Qphoria » Fri Aug 29, 2008 9:16 pm

The session is also timed based as well I believe. What is the default session timeout for customers currently? From experience during I think something like 1-2 hrs. So I tend to agree that there is a limit on realistic need. But it's fun to just understand the options  :D

Image
Donate!|OpenCart Basics|GeoZones
Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by bruce » Fri Aug 29, 2008 10:53 pm

I agree with Juergen that the benefits of checking the config settings are not great and it also depends on the purpose of the settings but to be complete, they are...
  • The store owner does not have to wait for all current browsing customers to leave before setting changes are applied to a live store. The wait would be an unexpected behaviour and counter intuitive for the store owner who would most likely discover it during their own testing and think something was broken.
  • For developers and implementers, you would be setting up a situation where they have to delete all cookies etc to kill a session just to have admin settings applied while trying to test the web store.
Personally, I would go the extra yard (metre).  :D After all, this entire thread is a result of something that works ok when it is used one way and fails miserably when used another.  ::) 13 redundant executions for one order...

Active Member

Posts

Joined
Wed Dec 12, 2007 2:26 pm

Post by Qphoria » Sat Aug 30, 2008 7:35 am

Another one I just ran into was the currency switcher. The quote has a "text" field in its array that assigns the proper currency identifier ($, £, etc). So that's another one to catch, otherwise everything else can be in dollars and your quotes in euros
Last edited by Qphoria on Sat Aug 30, 2008 7:40 am, edited 1 time in total.

Image
Donate!|OpenCart Basics|GeoZones
Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by bruce » Sat Aug 30, 2008 7:41 am

8) nice catch

Active Member

Posts

Joined
Wed Dec 12, 2007 2:26 pm

Post by Qphoria » Sat Aug 30, 2008 10:43 am

Ok... without being too overboard. I've got a few more on my list:

- Module Status --  If module was disabled since last refresh, then remove the quotes
- Existing quotes
- Weight
- Product Count
- Currency change
- Language change
- Might want to add option weight support for future versions or contribs that support it.

I haven't added checks for changes on the usps side, as that would defeat the purpose.
Also haven't added checks for changes to the supported shipping types on the admin side. That one seems like overkill, though it could be a bit perplexing if you are out of express boxes and want only priority mailing choices. But I think the chances for that are minute.

You also have normal session timeout on your side.

Image
Donate!|OpenCart Basics|GeoZones
Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by bruce » Sat Aug 30, 2008 11:03 pm

LOL 13 redundant calls per page is starting to look like a streamlined design :D

Active Member

Posts

Joined
Wed Dec 12, 2007 2:26 pm

Post by Qphoria » Sat Aug 30, 2008 11:44 pm

HAHA... yea its starting to add up. Pretty much need to let everything reprocess except the actual external quote check.

Image
Donate!|OpenCart Basics|GeoZones
Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am

Post by Qphoria » Wed Sep 03, 2008 9:36 pm

Meh.. I still can't get this session business to be 100%. The coding is done, but as I test it, I keep finding new ways to break the session stuff to prevent reprocessing.
I must be going about this the wrong way. Hopefully someone can give me some pointers.

Here is what I got:

Code: Select all

function quote() {
    
    if (is_array($this->session->get('usps_quotes')) 
       && ($this->cart->getWeight() == $this->session->get('last_weight'))
       && (count($this->cart->products) == ($this->session->get('last_item_count')))
       && ($this->currency->code == ($this->session->get('last_currency')))
       && ($this->language->code == ($this->session->get('last_language')))
       && ($this->session->get('customer_id') == ($this->session->get('last_cust_id')))) {
      return $this->session->get('usps_quotes');
    }
......
......
      // Session Persistence to prevent reprocessing of quotes unless one of these have changed.
      $this->session->set('usps_quotes', $method_data); // persist existing quotes to avoid multiple loops.
      $this->session->set('last_weight', $shipping_weight); //persist cart weight to check for changes in weight.
      $this->session->set('last_item_count', count($this->cart->products)); //persist product count to check for changes in qty.
      $this->session->set('last_currency', $this->currency->code); //persist currency to properly show money descriptor.
      $this->session->set('last_language', $this->language->code); //persist language to properly show matching language.
      $this->session->set('last_cust_id', $this->session->get('customer_id')); //persist customer_id to properly get new quotes.
      //
      
      return $method_data;
  } // EOF Quote
- My first check is to see if I have any quotes stored in the session. If yes then do the next check.
- Then I check to see if the cart weight is the same as it was the last time. If yes, then do the next check.
- Then I check to see if the product count has changed. If its the same, then do the next check.
- Then I check to see if the currency or language has changed so I can recheck and use the correct money unit and language for the transit (1-3 days becomes 1-3 dias in spanish, etc)
- Then I check if the customer_id has changed, because if someone is using a public computer to shop, and they start to check out but have to leave, and immediately a new person hops on and adds the same item to their cart, then the quotes need to change. Oddly, the session doesn't dump when a person logs out.. maybe I have to add that. Not sure.

But I'm still hitting moments where it doesn't refresh because all those requirements are met, but there was something changed.

Can anyone give me some tips here?

Image
Donate!|OpenCart Basics|GeoZones
Image


User avatar
Administrator

Posts

Joined
Tue Jul 22, 2008 3:02 am
Who is online

Users browsing this forum: No registered users and 3 guests