The environment I was trying to access was rarely used via SoapUI, so naturally I did not bother configuring its IP and hostname in the /etc/hosts file. So I simply used the IP address for the web service endpoint I was accessing and thought that should be sufficient. It turns out, however, that SoapUI tries to do a DNS lookup on the IP address and as this was an internal address, the network name server would not respond for a long time. This is where the delay was coming from.
The solution was to add the IP address to the hosts file. The hostname doesn't actually matter in this case, it can be anything. Once this is done, SoapUI started establishing connections to the server instantly.