No, I am not saying that only married men are to be elders....but the bible does however.
1 Timothy 3:1-7
This is a faithful saying: If a man desires the position of bishop, he desires a good work. A bishop than MUST BE blameless, the husband of one wife, temperate, sober-minded, or good behavior, hospitable, able to teach; not given to wine, not violent, not greedy for money, but gentle, not quarrelsome, not covetous; one who rules his own house well, having his children in submission with all reverence (for if a man does not know how to rule his house, how will he take care of the church of God?); not a novice, lest being puffed up with pride he falls into the same condemnation as the devil. Moreover he must have a good testimony among those who are outside, lest he falls into reproach and the snare of the devil.
I honestly, don't understand how you can say that the qualification is that they should have a wife but then say that it is not a "must". It's a qualification, there are no exempts in place. And Paul encouraged those who are to be celibate to stay such, but for those who cannot, he says it is better to marry than to burn with passion.
As for Paul encouraging those who are to be celibate, well: Let's turn to the scriptures and see what it says:
1 Corinthians 7:6-9
Now I say this as a concession, not as a commandment. For I wish that all men were even as I myself. But each one has his own gift form God, one in this manner and another in that. but I say to the unmarried and to the widows: It is good for them if they remain even as I am; but if they cannot exercise self-control, let them marry. For it is better to marry than to burn with passion.
The question was arisen to Christ as well:
This verse really starts in verses 1 but I'm going to skip down to 10-12 because it asks the question, directly.
Matthew 19:10-12
His disciples said to Him, "If such is the case of the man with his wife, it is better not to marry." But He said to them, "All cannot accept this saying, but only those to whom it has been given: For there are eunuchs who were born thus from their mother's womb, and there are eunuchs who were made eunuchs by men, and there are eunuchs who have made themselves eunuchs for the kingdom of heaven's sake. he who is able to accept it, let him accept it.
In both cases of what Paul says and what Christ says, never are those eunuchs "apart" of the body and delegated to be part of a Priesthood. They simply lived their Christian life for Christ in the way that He have asked without marriage. They were not considered Priest, it was only their relationship lifestyle that was different from the other Christians who were married.