Rock music was found to be particularly distracting for men.
The study, from Imperial College and the Royal College of Music, played music into headphones while people tried to play the Operation board game.
The research teams have been examining how music can alter performance.
Families might have arguments about whether teenagers should be listening to music when studying.
But there are also professional settings where music is sometimes played in the background - including operating theatres.