I know I am in the minority on watching anime with dubs instead of subs. Hell, I know some anime fans won't even want to associate with me because I ignore the subs! That's a topic we've discussed many times in the past, and I'm sure it'll be revisited. For now, let's put aside those differences and look at subtitles in general.
The anime scene is full of people that prefer subs to dubs. Outside of anime, it's hard to find an average movie-goer that doesn't mind having to read subtitles. I actually saw people walk out of Crouching Tiger, Hidden Dragon because they didn't realize it had subs. Yes, some people are that unwilling to compromise.
With that said, there are definitely some downsides to watching a movie or TV show with subtitles. Without a doubt, you're going to miss some of the action. Obviously any type of entertainment on a movie or TV screen is going to be heavily visual. When you're focused on reading through lines and understanding story, you're bound to miss some of the eye candy.
Do you find it hard to focus in on both text and visuals? Have you had issues with text scrolling through too fast? I'm sure that the more you watch subbed content, the easier it is to focus on the whole picture. Still, I'd love to hear your opinions on the constantly battling imagery and text.