Jesus' position was that Moses permitted divorce due to human stubbornness and weakness. It's also important to understand that in Jesus' time there were two major schools of rabbinic thought: Bet Hillel and Bet Shammai; according to Bet Hillel (supposedly) a husband could get a certificate of divorce for nearly any reason, thus even if his wife burned the afternoon meal he could divorce her; whereas Bet Shammai maintained that divorce should only be permissible in only in severe circumstances, such as infidelity. In this case Jesus comes down largely in agreement with Bet Shammai. Divorce shouldn't be something taken lightly, a husband ought to be devoted to his wife, and to love her. St. Paul goes so far as to say that a husband ought to submit to his wife in love, with the same love which Christ has for His Church.
The Christian perspective has by and large been, historically, that polygamy was tolerated in ancient times but that monogamy has always been preferable, and historically only monogamy has been accepted as valid in the eyes of the Church.
So, "legally", no, the patriarchs and kings of Israel who had multiple wives were not engaging in adultery; though an historic Christian perspective would be that this was merely tolerated and that, frequently we see in their example how polygamy was often detrimental. Solomon, for example, in spite of all his wisdom was brought down and led astray from the worship of God. Further, David's lust for Bathsheba led him to indirectly murder her husband Uriah. Augustine argues that polygamy was only acceptable in ancient times for the sake of procreation, but that such is no longer necessary; that lust is never an acceptable reason (to which we might point again to David).
If one wants to boil this all down to an easy byte of information: Polygamy was never ideal, and should never be seen as good even in those ancient times, but was merely tolerated; that monogamy is the ideal, and that in marriage there is to be fidelity toward one another.
-CryptoLutheran