If a person wants to put up signs displaying the Ten Commandments or other parts of the Bible in their front yard, that's fine. I got no problem with that, and I doubt many people would. Including atheists.
But when the local court puts a tax-payer funded piece of carved stone with the ten commandments engraved on them, then there is a problem. Why? Because it violates the First Amendment.
Specifically, the First Ammendment says, "Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances."
In other words, you can't have a state-supported religion. The state can't do anything to show it supports one religion over another. And if the court has a taxpayer funded stone with the ten commandments written on it, then that is using government money to support one religion over others.
If you don't understand it, think of it like this: If your Muslim neighbour had a bumper sticker that had a quote from the Quran, say, "And if they incline to peace, incline thou also to it. 8:61," I'm sure you would have no problem with it. But if your local court used taxpayer monies to fund a stone carving of a passage from the Quran, then I'm sure you and many Christians would complain.