My mom put up with a lot of abuse (never physical though) from my dad due to his drinking. The tipping point where she finally decided to leave him for good was when I was about 7 or 8 and he was being his usual drunk mean self towards her and I told him to leave her alone. That was it, she said she wasn't going to have me start getting in the middle if it and defending her from him.
Of course, I didn't find out that that was the exact reason for many years later. She didn't want me to think it was my fault.
Point being, when the kids start to notice how things really are, that's when you know there is a problem. Now hopefully you will put put the drink down for good before it costs you your family.