On the surface this is perfectly reasonable and how, in an ideal world, it would be. The thing, though, is that benefits conferred by systemic racism exist for many of us white Americans from actions long after slavery ended and in some ways continue to this day.
Jim Crow followed the end of Slavery and showed that America was NOT going to let up on the people of color, not one whit. The 1960's saw the Civil Rights movement which definitely helped ease some of the horrors, but clearly there's something going on that is systemic and biased against black and hispanic Americans to this day. Ridiculous levels of income inequality, inequality in interactions with the justice system, etc. All of this helps ensure the "generational wealth" of white families vs the average black family.
That's the thing. We want to make TODAY more equitable and just. But we are still far away from where we need to be. Even today.