From reading that verse, it appears that David's older brother held David in very low esteem, and even perhaps had contempt for him. David wasn't even speaking to him, yet he imposed himself on his little brother as if David's very presence caused him to feel contempt towards him. It also possibly can be surmised that because of David's father's reaction, and his own words about David that David was sort of an "after-thought" for the entire family and that he was expected to be neither seen nor heard.
This is possibly why David was so skilled as a shepherd and why he was able to kill a bear and a lion to protect the sheep. All the time he was rejected by his family and out with the sheep, God was using as a "training ground" so to speak. And since it can be assumed that no one else paid him any heed nor talked to him, that he learned to commune with and speak to the living God. He also, during this time learned an un-faltering TRUST in God and a Godly "fear/respect" for God so that he neither feared man nor beast, which included a giant named Goliath, that the king and the entire army cowered in fear from.