I wouldn't mind those memorials, it's worth remembering that for all our accomplishments, we have stains of barbarism. However, the attempt to remove Confederate monuments is still misguided, if not evil.
It's a common myth that the Union propagated some time after the war that the Civil War was fought over slaves, when in fact, it was over an issue much deeper, much more fundamental. An issue that plagued the United States since the initial war against Britain failed; Federalism. Should the United States be a unified, dare I say, Empire comprised of various provinces, as Greece and Rome before us? (That is the Union proposition.) Or should the United States be an alliance of States that have entered into a pact of brotherhood, an oath to support and defend one another, but never to intrude on one another by means of an Imperial authority? (That is the Confederate proposition.)
Since the South held the wrong (and incidentally, less authoritarian) philosophy, the North decided to impose laws that crippled their economy, and that's ultimately what lead to the war. The fact that one's economy depended on slavery on one's didn't was an entirely tertiary issue. In fact, around that time, the African slave trade was already practically non-existent, and machines were already replacing slave labor. Liberation would've occurred within decades at the absolute latest, no matter who won the war. But that wasn't could enough for the gracious King of the Union, the renowned Abraham Lincoln... or so we're told.
No. In reality, Lincoln, whose biographies and personal correspondences proves him to be of dubious character at best, had no interest in liberating slaves except to frighten the territory that the Union was attempting to re-conquer, and the proof is in the legendary "Emancipation Proclamation", which didn't do anything to enhance the lives of black peoples in the Union itself, but only claimed the freedom for those in the Confederacy, in the lands of which the Union had no authority.
The first successful Republic since Rome falls into the same Imperial tendencies, only this time, the people fought back! At least, that's the short version. That's the legacy of the Confederacy, a reminder of the inevitable link between power and tyranny. The rift between Left and Right, Liberalism and Imperialism. Evil and Good. Yet, as always, history was written by the victors, and the memories of this history is now boiled down to the elementary issue of "Racism vs. Equity", which couldn't be more reductionist if it tried.
Of course, I romanticized and abridged the case here, but it's important to remember what the whole issue is really about, and to recognize the propaganda associated with it.