As Daddio said, the issue could be with the package you have been provided by your ISP. It's not because you have an adsl connection that it is a fast connection. Sometimes the "upload" speed is much lower then the advertised " download " speed. For playing games, both need to be good, as you are both sending and receiving packages of data to/from the other players in game.
Further, the issue you mentionned sometimes happens indeed when you are playing with people who have a bad or instable connection themselfs. IT is not always an issue on your end.
There are many things that could cause instability on a connection. Once a connection become's unstable then you can indeed receive the "pause" screen and when the connection does not re-establishes itself in a short time span, the game come's to an end.
To avoid this problem, you should try to play with people who do have a stable connection. In general, you can see information about this beeing displayed at the bottom of the game room near the players name. Beside the rank it will display you the ping and if there is a bad connection it will tell you "no direct connection established".
A bit more information about "ping" can be found here:
Hope that somewhat explains your problem.