Mat 13:55 Is not this the carpenter's son? is not his mother called Mary? and his brethren, James, and Joses, and Simon, and Judas?
Mar 6:3 Is not this the carpenter, the son of Mary, the brother of James, and Joses, and of Juda, and Simon? and are not his sisters here with us? And they were offended at him.
Gal 1:19 But other of the apostles saw I none, save James the Lord's brother.
I have heard the arguments about the word "brother" and "brethren", but I believe without a doubt in my heart that these were the half-brothers of Jesus and James is the same James that wrote the book of James and was the head of the Church in Jerusalem who was talked about in Acts. Joses was Jude who wrote the book of Jude.
I think it does matter if one is trying to find absolute truth in the Bible. When I read the Word, I am always trying to "connect the dots" and see how different people and places are related to each other. To me, it reveals the purity of the Word.
Now, I say that with the love for my fellow Christians who do not have the same yearning or beliefs as I do.