Wow, separate but not divorce? Then shack up with the new girl without telling her you're still married? Who does that these days? And it surely couldn't have happened back then....
Actually that's exactly what happened and still occasionally happens today in the Jewish community (where it's called the Agunah problem).
In Malachi 2:16 God is angry with the men of Israel who were "putting away" their wives without giving them a certificate of divorce. There were 3 steps to a divorce... write a certificate of divorce, put it into her hand, and send her away. This generally necessitated the return of the bride price/dowry; which always belonged to the wife. What the men of Israel were doing were sending away their wife without the giving them a certificate of divorce. Men at that time were able to have more than one wife however, without a certificate of divorce a woman was unable to remarry leaving her without any means of support.
In current times, a Jewish man can (and sometimes do) obtain a civil divorce and then refuse to grant a 'religious' certificate of divorce that a religious Jewish woman needs to remarry. It is a way to further abuse the ex wife. Taken in context of the whole passage - what God really hates is divorce!
In light of this
Matthew 5:32 KJV But I say unto you, That whosoever shall
put away his wife, saving for the cause of fornication, causeth her to commit adultery : and whosoever shall marry her that is divorced committeth adultery.
Makes sense doesn't it. If a man puts away his wife without giving her a certificate of divorce he is causing her to commit adultery... women had no means of support other than marriage. So just as when Judah says his sin is greater than Tamar's because his sin (of not giving her to his youngest son) caused her to commit her sin of getting pregnant.
Under the law of Moses a man could put his wife away if he found some unclean thing in her. For example if it was discovered that the marriage was not legal in the first place... eg she was a close relative. The same is true in Mathew where the Greek word inappropriate contenteia is used for fornication.
inappropriate contenteia
illicit sexual intercourse adultery, fornication, homosexuality, lesbianism, intercourse with animals etc.
sexual intercourse with close relatives; Lev. 18 sexual intercourse with a divorced man or woman; Mk. 10:11, metaph. the worship of idols of the defilement of idolatry, as incurred by eating the sacrifices offered to idols.
The only reason, that one is allowed to put away his wife without issuing a certificate of divorce is when the marriage was not legal to begin with. Something more than adultery must be meant here in Matthew as adultery was punishable by death, so divorce is unnecessary.