I think when people refer to privilege they are talking generally about black people as a race. I doubt Obama ever has to struggle much in life with inferiority or poverty. Although he is more mixed race rather than black.
But black people who 're just ordinary working or lower middle class? Yes I think many of them have suffered and struggled to some degree due to their colour.
Once you read Obama's biography you will find that your supposition is way off base.
His father left soon after his birth, and the couple divorced two years later. In 1965, Dunham married Lolo Soetoro, a University of Hawaii student from Indonesia. A year later, the family moved to Jakarta, Indonesia, where Obama's half-sister, Maya Soetoro Ng, was born in 1970. Several incidents in Indonesia left Dunham afraid for her son's safety and education so, at the age of 10, Obama was sent back to Hawaii to live with his maternal grandparents. His mother and half-sister later joined them.
As a child, Obama did not have a relationship with his father. When his son was still an infant, Obama Sr. relocated to Massachusetts to attend Harvard University and pursue a Ph.D. Obama's parents officially separated several months later and ultimately divorced in March 1964, when their son was two. Soon after, Obama Sr. returned to Kenya.
Obama struggled with the absence of his father during his childhood, who he saw only once more after his parents divorced when Obama Sr. visited Hawaii for a short time in 1971. "[My father] had left paradise, and nothing that my mother or grandparents told me could obviate that single, unassailable fact," he later reflected. "They couldn't describe what it might have been like had he stayed."
While living with his grandparents, Obama enrolled in the esteemed Punahou Academy. He excelled in basketball and graduated with academic honors in 1979. As one of only three Black students at the school, he became conscious of racism and what it meant to be African American.
Obama later described how he struggled to reconcile social perceptions of his multiracial heritage with his own sense of self: "I noticed that there was nobody like me in the Sears, Roebuck Christmas catalog. . .and that Santa was a white man," he wrote. "I went into the bathroom and stood in front of the mirror with all my senses and limbs seemingly intact, looking as I had always looked, and wondered if something was wrong with me."
Read more at biography.com