Template or Custom Design?

Hi, I am just about to develop a new website and am concerned about the speed of the site as I am planning to use a template from theme forest and was worried this option would give me a slower website than if I was to develop the template myself, could anyone shed some light on this? Should I be looking to purchase a template that runs fast out of the box or if I buy a slow template would i be able to speed it up somehow? Any advice would be much appreciated.