If you are using DNS names and the lookup of those names take longer then they should for whatever reason this can seriously impact nginx performance. If possible use IP's to see if the delays are gone or use a debug build version to see where in the code it is waiting and for what.
If it is really a dns issue you might have to do some caching of dns lookups.
If it is really a dns issue you might have to do some caching of dns lookups.