Do you realize that marriage vows differ from country to country? In my language, there is no such expression of "forsaking others". Marriage vows are also not universal. At least in Finland, spouses are free to write their own vows if they want to, they don't have to use the standard ones. So, making a marriage vow which states that I promise that I'll let my wife have alot of boyfriends, would be as valid as the standard one.
Also, if we leave the mythical finality aspect out of the vow, and call it a promise (the stuff said when getting married is called "promise" in my language, not vow) there is really no point to assume that you're somehow ultimately tied to your promise if the person I made the promise to, tells me it's not against her will that we can do things differently than previously agreed.
To creatively quote someone famous: The vows are made for human, not human for the vows