Actually it is an ISP problem.
Youtube is hugely popular which for an ISP means they would have to upgrade their peering with youtube.
However many ISPs came up with the idea that charging twice for the same thing is better than charging once so they try to blackmail Youtube by saying "Either you pay us an additional fee for the traffic or your connection might start sucking...".
Google on the other hand has refused to do this so far.
What's going on there is the core of the net neutrality debate we are currently having.
Google actually has public stats about this:
http://www.youtube.com/my_speed