Election 2016 Social Desirability Bias / Trump Shaming: Republican Primaries
The Republican primaries are a great case study for Social Desirability Bias. We have a considerable amount of data from multiple polls in multiple states for multiple candidates. Additionally, this data can be compared to actual results from the primaries.
It turns out that there was a fairly strong bias against Trump. On average, the difference between live interviewer polls (‘live’) and robocalls / IVR or internet polls (‘anonymous’) was 6.1 percentage points against Trump. In other words, people tended to answer voting intention questions very differently depending on if a live human over the phone asked the question or if they were allowed to answer in a more anonymous fashion. Additionally, there was an anti-Trump poll bias when comparing actual primary results and anonymous polls with Trump receiving on average 7.7 percentage points more in the actual primary result than in the poll.
Another interesting fact is that the Trump discount varied according to location. Most importantly, Trump’s live poll numbers suffered the most in traditionally Democratic states. In other words, it appears that Trump shaming is most effective in states that are historically dominated by Democrats.
We will also see that there was the reverse effect, though to a much lesser extent, for Kasich. As a candidate, he was very non-controversial. In an election season that was characterized by political feuds and name calling, Kasich did his best to remain positive while trying to be ‘the adult in the room’. The result appears to have been that more people were willing to support him publicly than those who actually liked and/or wanted to vote for him. In Trump’s case, live polls were considerably lower than anonymous polls creating a negative bias. In Kasich’s case, live polls were actually higher than anonymous polls creating a positive bias. In other words, it seems like in a very emotionally charged election year, some people preferred to publicly but only superficially support the plain vanilla candidate to avoid social condemnation. Social Desirability Bias tends to work both ways.
Some elements of the study:
The following table highlights the conclusions as broken down by state group.
Table 1: Comparing Actual Primary Results to Average Live Polls and Comparing Average Anonymous Polls to Average Live Polls, for Trump, Cruz and Kasich during 2016 Republican Primaries
Solid Republican States |
Flipped States, 2008 to 2012 | Traditional Swing States |
Solid Democratic States |
|
TRUMP | ||||
Actual Results – Avg Live Polls |
4.8% |
12.9% | 1.2% |
10.3% |
Avg Anon. Polls – Avg Live Polls |
6.3% |
6.3% | 2.5% |
7.1% |
CRUZ | ||||
Actual Results – Avg Live Polls |
7.3% |
8.5% | -1.2% |
0.3% |
Avg Anon. Polls – Avg Live Polls |
3.6% |
-0.8% | 2.3% |
0.4% |
KASICH | ||||
Actual Results – Avg Live Polls |
-1.4% |
-2.7% | 0.2% |
0.8% |
Avg Anon. Polls – Avg Live Polls |
-0.5% |
-0.5% | -0.2% |
-0.9% |
Source: Realclearpolitics.com, Wikipedia
Four main points jump out from the data: (1) Trump’s negative live polling bias, (2) Trump’s worst bias was in Solid Democratic States, (3) Kasich’s positive live polling bias, and (4) Traditional Swing States tend to have lower levels of bias.
Trump had by far the largest poll bias. Seven of eight of his net numbers presented in the table are the highest. The implication is that on average more people actually supporting this candidate felt that admitting to that support in public was not socially desirable than any other candidate.
Of Trump’s numbers, the most extreme appear in Solid Democratic States. The average difference between his live poll numbers and his anonymous poll numbers was an astounding 7.1 percentage points. The figure including actual results jumped to 10.3 percentage points. In other words, the anonymous polls were much closer to the actual but actual results were still better. These results make sense in that a Social Desirability Bias against Trump in traditionally Democratic states seems obvious. Comparing the same figures to those of Cruz and Kasich for these states and it looks like Trump shaming was extremely effective in Democratic states.
Kasich appears to have the opposite bias in that he tended to do better across the board in live polls. The net difference between his anonymous polls and live polls is actually negative. In other words, people seem to have seen Kasich as a socially acceptable choice among Republican candidates and used Kasich as a public choice but in reality supported a different candidate. The caveat here is that these figures are very low. However, consistency across the groups of states supports its existence.
As a group, the figures from Swing States are noticeably lower. Though there is not enough data to actually prove this theory, it could be that VA, FL, and OH have received so much scrutiny and polled so often that the populace is more at ease with answering polls and with political divergence. Put another way, because they have been in the political spotlight for a considerable period of time, they might not feel as constrained about answering political questions and might not feel as much social pressure to answer one way or the other. This would reduce the impact of Social Desirability Bias.
We will revisit these concepts in other posts so this is not the final conclusion. For now, there are some basic takeaways, especially as they relate to the general election: