Being privileged doesn't automatically means that you don't acknowledge it, or hate civil rights, or want to go back to slavery. You don't even have to come from a super elite background to be privileged.
Privilege is benefitting from the situation of your parents basically, and benefitting from society for no reason other than the fact that you meet certain criteria that aren't even in your control, like your race, ethnicity and gender. If you're white, if you're cis-gendered, if you're male....Those are all things that unfortunately do get somebody ahead in the world. You're more likely to get a job, you're perceived as more trustworthy, more likely to get payed more than somebody else for the same job with the same education, more likely to get promoted.
If your parents made decent money and you grew up in a decent neighborhood, that's privilege. If your parents were able to put you in a good school, if they were involved in your school and helped you do homework, if they were able to pay for your college or helped support you through college, that's privilege. If they bought you a car or helped you buy a car, that's privilege. If you were able to work as a teen or young adult for experience and save that money because your parents were otherwise supporting you, that's privilege.
Kids born into poverty have an extremely hard time getting out as adults. It's not just "Oh work hard and you'll be successful!!!". They are literally set up developmentally, culturally and financially to fail. Children do not get the kind of attention or parental involvement they need to do well in school. They're more likely to deal with teen pregnancy, more likely to get involved in selling sex, more likely to get involved in using and/or selling drugs, more likely to drop out of high school because they have to work to help support their family and their brothers and sisters. All of these things get carried with them through life and keep them from breaking out of that cycle.