When these laws are broken society breaks down. Turning a blind legal eye to some of these laws, such as adultery, doesn't mitigate the damage done by breaking them.
So what is the damage done by adultery -- or by "coveting," for that matter?
Jesus said, 'Love God, and, love your neighbor, UPON THESE TWO COMMANDS hang all the law and the prophets'. Thus the distinction.

Adultery may not the most damaging societal crime broadly speaking, but many believe that it is the most damaging offense to the individual victim, surpassing even the murder of a loved one in it's overall effect. Regarding the origin of our laws, didn't I say, 'regardless of their origin'?