You need to understand the context a bit. Back in those days, the belief among the jews was that the richer you are, the more likely you are to be saved. The poor and the sick were seen as the lowest class in terms of favor with God. Sickness and poverty was taken to mean that God was displeased with them whereas wealth means favor with God. So when Jesus said that it is hard for a rich man to enter the kingdom, the disciples were shocked because in their mind, they are the most likely to make it. So then they ask well if the rich man hardly has a chance then who on earth can. In their minds, it was impossible for anyone else to enter which is why he answered and told them that what is impossible with men (in their minds impossible) is possible with God.
So in context what it really means is that God can save anyone regardless of class or status. Everyone is equal at the foot of the cross.
They could not see how any one can be saved. Yet.. with GOD everything is possible. And in context with John 3 16. What man could not do could not see. God came down.. died for man. So the impossible was done