Yes, the fact that sites 1 and 2 have different ISPs can have an affect on the speeds you get as there are likely to have different network configurations (possibly including filtering) and tier 1 providers.
The other thing I noticed is that, for site 2, you are running ROS 7.0.5 but site 1 and the client site are running ROS 6.49.2. Are you able to downgrade site 2 to 6.49.2 and test to see if this makes any difference? You might want to take a look at the reports people have posted of speed issues after upgrading to ROS 7.1 (such as this one).