I have Dutch in-laws who don't necessarily speak the best English. They're of a particularly native speaking generation, and have lived in a small 'no name' town for pretty much their entire life. I've spent a lot of time over there so I've gradually picked things up over the years. My Brother in-law's cousin speaks NO English at all.
Of course, when you head into Amsterdam it's of no practical use at all because everyone speaks English. But if you're straddling the Dutch/German border then it's extremely handy to have your Dutch down. Not even the young generation will speak English and things can get really awkward VERY fast. (What would you think if I said "slagroom" to you? LOL)
My brother in-law (sister's husband) speaks fluent English, German and Flemish. Which is handy because my German is absolutely shocking. I think my Korean is better actually.
If you stay in a country long enough (or regularly enough) you'll pick things up really fast. There's no replacement for actually being there.