It is now quite common for media to omit pertinent facts, mislead, or even deceive in order to further their agenda. I see it all the time, poor assumptions, bad data, or simply media bias. Even the alleged "fact finders" make wrong or premature conclusions at times. Today, I want to look at some reporting on polling for the U.S. Presidential race, found at Biden leads in six 2020 swing states as Trump sees no convention bounce, CNBC/Change Research poll finds
What is interesting, is this sentence. “The poll, taken Friday through Sunday, surveyed 4,143 likely voters across the six states and has a margin of error of plus or minus 1.4 percentage points.” Voters across the six states to me, means a combined number. If you polled 4,143 voters from each state you would say: The poll, taken Friday through Sunday, surveyed 4,143 likely voters from each of the six states and each state has a margin of error of plus or minus 1.4 percentage points.
The article goes on to report six battleground state polls individually. However, when it quotes the margin of error, it combines the six states and now leads the reader into deception. Here is the polling math from Sample Size Calculator - Confidence Level, Confidence Interval, Sample Size, Population Size, Relevant Population - Creative Research Systems
Using the second box, I first, put in the numbers 4,143 for the sample size. I assume a 95% confidence level and about 20,000,000 as the size of the overall population of voters in the six state survey. The percentage stays at 50%.
In doing this I get a confidence interval of 1.52 which is slightly higher than the articles, 1.48.
So I adjust the population downward, to 2,000,000. I know beforehand this will not make much of a difference because once you reach a sample size over 4,000, the population size is not going to matter too much. The results with the adjustment shows a confidence interval level of 1.52, so no change. So now let's look at why you can't use the smaller confidence interval. Because you can only count the voters you polled in Pennsylvania, if you are going to report that state separately. Initially, I was going to simply divide the 4143 by 6 to estimate how many voters in Pennsylvania were actually polled. It comes to 691, but later, I found more information on the exact poll at a 3rd party website that reports all polls. Pennsylvania Polls Here they cite the same poll for Pennsylvania to have an exact sample size of 829. This simply means that they polled more people in Pennsylvania and less in one or more of the other five states.
Again using a 95% confidence interval and the 829 figure with 2 million as the voting population of Pennsylvania, I get a confidence interval level of 3.4
If I do it with a population of 500,000 I also get 3.4
So here is the problem. CNBC reports a confidence level of 1.48 but in reality it is in the 3.4 range. Too many, that is no big deal, but in the Pennsylvania example, it is critical. Here is why.
The article reports”
“Pennsylvania: Biden 50%, Trump 46% (was Biden 49%, Trump 46%)”
What this means with the margin of error of 1.48 is that 95% of the time you will have an actual outcome between Biden 48.52% to 51.48% and Trump 44.52% to 47.52% In this scenario, we would expect Trump to lose at least 95% of the time.
Now let's use the real, larger margin of error figure of 3.4%.
Biden's range becomes 46.6% to 53.4% and Trump's range is now 42.6% to 49.4%.
This means that the poll is too close to call with any certainty. Biden might get 47 and Trump 49 as both those numbers are within the range, using the new confidence interval.
So what is wrong with this article? It combines the polling numbers of six states to lower the margin of error, though they are reporting on polling from the six states individually. To get the real confidence interval, I had to go to a third party website and calculate it myself. The real interval makes the race to close to call which is not reported. Normally, if the poll is within the margin or error, when calculated correctly, a informed reporter would state that.
I can't say if this was an omission or a blatant attempt to improve Biden's standing. If you notice the third party website gives the poll a C- grade. However, for the way they abused the confidence interval, I would give CNBC an F.
No one expects everyone to be a pollster or have that much knowledge of survey research. However, with just a few tools and a little practice anyone can become proficient. The main point of this blog entry though is to make sure you question what you are reading. Factual reporting basically states the numbers and lets the reader make their own conclusions. An article becomes an opinion piece once the author adds words that announce their own opinion. Notice that by giving the article an F, I inserted an opinion. Some articles are mainly opinion, others like this are mainly factual. May God give us all discernment, wisdom and practical knowledge to help us understand more of what we read.
What is interesting, is this sentence. “The poll, taken Friday through Sunday, surveyed 4,143 likely voters across the six states and has a margin of error of plus or minus 1.4 percentage points.” Voters across the six states to me, means a combined number. If you polled 4,143 voters from each state you would say: The poll, taken Friday through Sunday, surveyed 4,143 likely voters from each of the six states and each state has a margin of error of plus or minus 1.4 percentage points.
The article goes on to report six battleground state polls individually. However, when it quotes the margin of error, it combines the six states and now leads the reader into deception. Here is the polling math from Sample Size Calculator - Confidence Level, Confidence Interval, Sample Size, Population Size, Relevant Population - Creative Research Systems
Using the second box, I first, put in the numbers 4,143 for the sample size. I assume a 95% confidence level and about 20,000,000 as the size of the overall population of voters in the six state survey. The percentage stays at 50%.
In doing this I get a confidence interval of 1.52 which is slightly higher than the articles, 1.48.
So I adjust the population downward, to 2,000,000. I know beforehand this will not make much of a difference because once you reach a sample size over 4,000, the population size is not going to matter too much. The results with the adjustment shows a confidence interval level of 1.52, so no change. So now let's look at why you can't use the smaller confidence interval. Because you can only count the voters you polled in Pennsylvania, if you are going to report that state separately. Initially, I was going to simply divide the 4143 by 6 to estimate how many voters in Pennsylvania were actually polled. It comes to 691, but later, I found more information on the exact poll at a 3rd party website that reports all polls. Pennsylvania Polls Here they cite the same poll for Pennsylvania to have an exact sample size of 829. This simply means that they polled more people in Pennsylvania and less in one or more of the other five states.
Again using a 95% confidence interval and the 829 figure with 2 million as the voting population of Pennsylvania, I get a confidence interval level of 3.4
If I do it with a population of 500,000 I also get 3.4
So here is the problem. CNBC reports a confidence level of 1.48 but in reality it is in the 3.4 range. Too many, that is no big deal, but in the Pennsylvania example, it is critical. Here is why.
The article reports”
“Pennsylvania: Biden 50%, Trump 46% (was Biden 49%, Trump 46%)”
What this means with the margin of error of 1.48 is that 95% of the time you will have an actual outcome between Biden 48.52% to 51.48% and Trump 44.52% to 47.52% In this scenario, we would expect Trump to lose at least 95% of the time.
Now let's use the real, larger margin of error figure of 3.4%.
Biden's range becomes 46.6% to 53.4% and Trump's range is now 42.6% to 49.4%.
This means that the poll is too close to call with any certainty. Biden might get 47 and Trump 49 as both those numbers are within the range, using the new confidence interval.
So what is wrong with this article? It combines the polling numbers of six states to lower the margin of error, though they are reporting on polling from the six states individually. To get the real confidence interval, I had to go to a third party website and calculate it myself. The real interval makes the race to close to call which is not reported. Normally, if the poll is within the margin or error, when calculated correctly, a informed reporter would state that.
I can't say if this was an omission or a blatant attempt to improve Biden's standing. If you notice the third party website gives the poll a C- grade. However, for the way they abused the confidence interval, I would give CNBC an F.
No one expects everyone to be a pollster or have that much knowledge of survey research. However, with just a few tools and a little practice anyone can become proficient. The main point of this blog entry though is to make sure you question what you are reading. Factual reporting basically states the numbers and lets the reader make their own conclusions. An article becomes an opinion piece once the author adds words that announce their own opinion. Notice that by giving the article an F, I inserted an opinion. Some articles are mainly opinion, others like this are mainly factual. May God give us all discernment, wisdom and practical knowledge to help us understand more of what we read.
