I checked the file system for any suspicious files, there were none. I checked the source code and templates for evidence of anything that has been added, there was nothing there. Yet all my pages were being served with the following <script> injected into them just before the closing </html> tag...
The worst part of it is GoDaddy, in their help article, admits that this could slow down or break your site! So much for a tool that is designed to improve performance and reliability!
Luckily there is a way to turn this off - by opting out. Here's how. From the Hosting console, click on the ellipsis (...) in the top right hand corner, then click 'Help Us'.
On the dialogue box that comes up, click 'Opt Out'.
By the way the issues I was having with the admin interface turned out to be unrelated to this but were rather a bug in Safari which is resolved by closing Safari down and opening it again, maybe more on that later though.