I have found (with my father, at least) that it is helpful to consider such bad behavior in adults as childishness and treat them as such. A child 'acts out' but on the other hand really wants you around. They never make the connection between their bad behavior and the fact that nobody likes them or wants to be around them.
My father was pretty emotionally and physically abusive to me while I was growing up, but I still love him and recognize that he is an abused child also and (emotionally) never made it to adulthood. Regardless, he IS an adult and needs to learn that there are consequences to treating someone poorly.
I have a ZERO tolerance policy for any hurtful or unkind behavior from him. As soon as he 'acts up' the conversation/outing/dinner/whatever is over. I announce loudly, 'Okay, we're done' and leave or hang up immediately. When I'm visiting his house (he now lives several states away,) I leave the room or go out. With this sort of person, they have made a habit of malicious and manipulative interaction but they can't be malicious and manipulative to someone who isn't there.
Everyone who made a big deal out of boundaries is right. You need to clearly establish to them when they've crossed the line and enforce the consequences every time. In all likelyhood, it won't make them change, but it will put the two of you in the position of power in each situation of every time - and, let's face it, the feel of power over you is what feeds your in-laws behavior.