Here are some quotes from an article I found that I believe speak to the issue at hand:
For the complete article:
http://www.jewishworldreview.com/cols/williams012600.asp
Excerpts:
"DURING OUR WAR OF 1861, ex-slave Frederick Douglass observed, "There are at the present moment, many colored men in the Confederate Army doing duty not only as cooks, servants and laborers, but as real soldiers, having muskets on their shoulders and bullets in their pockets, ready to shoot down ... and do all that soldiers may do to destroy the Federal government.""
"Horace Greeley, in pointing out some differences between the two warring armies said, "For more than two years, Negroes have been extensively employed in belligerent operations by the Confederacy. They have been embodied and drilled as rebel soldiers and had paraded with white troops at a time when this would not have been tolerated in the armies of the Union." General Nathan Bedford Forrest had both slaves and freemen serving in units under his command. After the war, General Forrest said of the black men who served under him "(T)hese boys stayed with me ... and better Confederates did not live.""
"It was not just Southern generals who owned slaves but northern generals owned them as well. General Ulysses Grant's slaves had to await the Thirteenth Amendment for freedom. When asked why he didn't free his slaves earlier, General Grant said, "Good help is so hard to come by these days.""
"The flap over the Confederate flag is not quite as simple as the nation's race experts make it. They want us to believe the flag is a symbol of racism. Yes, racists have used the Confederate flag, but racists have also used the Bible and the U.S. flag. Should we get rid of the Bible and lower the U.S. flag? Black civil rights activists and their white liberal supporters who're attacking the Confederate flag have committed a deep, despicable dishonor to our patriotic black ancestors who marched, fought and died to protect their homeland from what they saw as Northern aggression.
They don't deserve the dishonor."