Firstly, your listing the exceptions. Secondly, if you take a look at the list of the wealthiest, these are not the type of people that dominate that list.
http://www.forbes.com/billionaires/list/
1. Bill Gates - arguably in the category you listed, but that is debateable.
2. Carlos Slim Helu - Capitalist, investor, not inventor/businessman.
3. Warren Buffet - Capitalist, investor, not inventor/businessman.
4. Amancio Ortega - arguably in the category you described.
5. Larry Ellison - Critical in the development expansion of Oracle, check for earned his money.
6. Charles Koch - Capitalist, investor, not inventor/businessman.
7. David Koch - Capitalist, investor, not inventor/businessman.
8. Christy Walton - While she has ties to Walmart, her wealth was due to married into inherited wealth.
9. Jim Walton - inherited wealth.
10. Lilliane Bettencourt - inherited wealth.
11. Alice Walton - inhertited wealth.
12. S. Robton Walton - inherited wealth.
13. Bernard Arnault - Capitalist, investor, not inventor/businessman.
14. Michael Bloomberg - Investor/businessman. Questionably in the category you described.
15. Jeff Bezos - in the category you described.
16. Mark Zuckerburg - in the category you described.
17. Li Ka-shing - questionably in the category you described.
18. Sheldon Adelson - in the category you described.
19. Larry Page - in the category you described.
20. Sergey Brin - in the category you described.
21. Georg Schaeffler - questionably in the category you described
22. Forest Mars - inherited wealth
23. Jacqueline Mars - inherited wealth.
24. John Mars - inherited wealth.
25. David Thomson - inherited wealth.
26. Paulo Lemann - questionably in the category you described.
27. Lee Shau Kee - Capitalist/investor
28. Stefan Persson - inherited wealth.
29. George Soros - Capitalist/investor
30. Wang Jianlin - Capitalist/investor
11 of the top 30 are people who really made an invention or business plan that set them apart and brought them wealth through their innovation. The rest may have used business acumen to leverage existing wealth into more wealth, but i would argue are not people like Joe.
You can keep going down the list, but you'll find that the majority of the super wealthy are not like Joe, but are those who have leveraged wealth or managed wealth to gain wealth, rather than innovating or executing a business plan that they devised.
I'll re-iterate: i'm not arguing that those who contribute and manage capital don't deserve a share of the wealth that is created. They absolutely do. I'm arguing that the share of the created wealth they receive is disproportionate to their contribution.