You should be concerned if many people in your community aren't vaccinated. Vaccines not only protect the people that get them, but also the people who don't or can't (due to medical reasons). We often forget it wasn't so long ago that massive numbers of people would die of illnesses like the flu, and technically still could. For vaccines to effectively stop the spread of a disease in a population, enough people have to be vaccinated so that the chances of an unvaccinated person is unlikely to come into contact with someone who is sick, catch the illness, and spread it to others.
Worse is when it starts to infect animals, which often aren't vaccinated. Yes, humans can infect animals with their illnesses, it isn't a one way deal. But when animals become infected, the illness is more likely to mutate into something more dangerous.