A missionary to Angola many years ago told me this story:
He came from the USA, was not well-off, did not have a regular salary, but was definitely better off than the villagers amongst whom he worked. He did have a suit, but felt that wearing it to the village church services was an arrogant display of wealth because most villagers had very poor clothing, many had second hand stuff, many had no shoes. So he stopped using his suit. He came in decent, clean, but ordinary everyday clothes.
After a couple of months the edlers of the church, all of them ordinary villagers, came to him and asked "why have you stopped coming to church in your suit? Do you no longer have respect for us or the Lord, that you don't come in the best you have? We know you have it. We feel you honour us when you use it in our church. If the Provincial Governor came to the village and wanted an audience with you, would you not wear it for him? We think it is almost an insult that you refuse to wear it in our presence. Are we not good enough for you?" He felt very challenged by their perspective and their concern. And he started using his suit again.
Hidden motives, subconscious assumptions, can work in many ways and directions. Pride can show itself in unexpected ways. The heart of man is deceitful above all things.
Underlying the whole isssue is maybe the question, why do we have "fine clothing" at all? and other questions follow... Who do we wear it for, and why? In which situations (other than church) would we wear it? what do we expect to gain (or not lose) from doing so? What do the answers to these questions say about our values, our priorities, our relationships?