I used the developer tools to monitor the network activity and nothing appears to happen when the button is clicked and the request only appears in the list once the page has loaded. This is not normal, I created a new store for comparative purposes and the same action is almost instant on the blank site. The slow site does have products with quite a lot of options so I can imagine that would add some time to build the product page but not like this, surely if it were the case, the front end would be comparatively slow?
Here's the network activity with the waterfall expanded, what is the possible cause for the huge gaps?

The site is running on a shared hosting server, it's using PHP 5.6 and Maria DB as the database engine