In the 90s, I was a card-carrying Libertarian. At that time, the Party didn't believe in moral relativism. In order to join, you had to agree to the Non-Aggression Principle, which says that it is immoral to initiate or threaten force except to protect people from violent and property crime. I left because I couldn't see how any government at all could be justified under that principle.
Since then, the party has been co-opted by progressives. For example, now we have left-libertarians, who want a free market and liberty in general, but want government to smooth over some of the negative effects of the free market. Apart from the question of how do you do that without forcible intervention, a key principle of the original libertarianism is that just because something is bad doesn't mean we're justified in using force to prevent it. You may be thinking at this point that Libertarianism isn't a coherent philosophy, and you may be right about that.
That said, our concept of subsidiarity is sufficient if put into practice to prevent the abuses of big government that Libertarianism arose in response to. None of this really works without Natural Law anyway, and while Libertarians did have a form of it, without God at the base of it, there's not a stable foundation for it, which may be as good an explanation as any of what happened.
