Election 2016 Social Desirability Bias / Republican Shaming: Congress Approval Rating
The idea of individuals hiding their true voting intentions when responding in polls seems to have come to the forefront with Trump’s candidacy. In reality, however, similar things have occurred for the Republican Party before Trump even announced his candidacy. This does not fit into many of the current talking points in that it shows that Trump is not a unique case. It also highlights the general social undesirableness of the Republican Party. Although not very surprising it might not make conservatives very happy to see evidence that Trump does not appear to have brought the negative onslaught of bias on the party. In fact, looking at the numbers objectively, the negative bias shown in this case study against the Republican Congress far exceeds anything that has occurred to Trump.
For national politics, as we have explained elsewhere, there needs to be an efficient transference mechanism for essentially ‘teaching’ the public what should be considered socially acceptable in terms of party, candidate, and policy. In this case, the strongest mechanism is media including content-producing TV, internet, and print publications.
Social Desirability Bias can be created in politics when the media takes a significantly excessive stance in comparison to the populace’s average opinion. Greater excess and more intense news coverage will result in higher levels of Social Desirability Bias. So if the media, on average, takes a strong position against a policy and make it clear that not agreeing with them is morally reprehensible and repeat this general idea frequently, many people will begin to agree with that position in public but not in private. The difference between the public and private (or superficial and real) responses is the bias.
We can measure Social Desirability Bias in politics by looking at the difference between live and anonymous polls. Live polls use live interviewers to collect data whereas anonymous polls use such things as robocalls and internet collection (not completely anonymous but much more so than speaking with a live person).
This phenomenon exists in multiple political examples. There has been speculation over a Trump discount or the fact that some people tend to not admit to supporting him in polls and in public. We have shown elsewhere that the same type of bias occurs in Obama’s approval rating. He currently receives a significant positive bias thanks to general media support and his aggressive attacks on Trump, someone who has been tagged as socially undesirable by the media. These examples are very Trump-centric and centered more or less on the current election cycle.
The topic of this post is Social Desirability Bias as it related to the 2014 mid-term elections. These took place well before Trump announced so we can assume they are completely independent of any influence related to him.
In the mid-term elections, the Republicans unexpectedly won control of the House of Representatives. This came as a huge surprise. Many believed that the Democrats were ushering in an age of dominance. Obama was reelected in 2012, the country culturally was moving in the direction of Democrats, long-term demographics were clearly in the Democrats favor, and Hillary Clinton was surely expected to win in 2016. For many, the Republican’s strong performance in 2014 was a shocker. The general media reacted negatively and commenced an attack on the Republican House.
Mid-terms are normally not that newsworthy but this time was different. Before any real disputes began, the media speculated on the difficult situation that a Republican Congress would mean for Obama – in short the media did an excellent job juxtaposing the socially acceptable Obama with a socially unacceptable roadblock for Obama, a Congress controlled by the opposing party.
The immediate result was the increasing of the bias against Congress. We can measure this bias by looking at how live and anonymous polls compare for the ‘Congress Approval Rating’ which is a fairly standard poll taken by a variety of entities. As in other examples, this net bias trends over time and shows influence of events if longer term trends shift around those events. As we see in the following chart, the bias against Congress increased dramatically from the election of the Republican majority in 2014.
Chart 1: Net Bias against US Congress, Live minus Anonymous Poll results for net approval rating of Congress, 6 month moving average
Source: Realclearpolitics
Prior to the mid-term elections, the net bias between live and anonymous polls for the Congress Approval Rating was slightly negative, but nothing exaggerated. This level is about normal. If the net bias for a topic moves between slightly positive and slightly negative without any discernible longer term trends, then we can more or less assume that Social Desirability Bias does not exist. We can assume this was the case just prior to the mid-term elections.
Mid-terms were held in November 2014 – this month was the worst month for negative bias from the start of the sample data. From that point, the obvious trend is not only down but a collapse. Rarely do you see such an obvious trend for this type of analysis. Recall that this is not measuring a change in opinion but the degree to which people feel uncomfortable telling their true opinions in front of another human. The low point for the 6 month moving average occurred in September of 2015 at 10.7%. The low point for the raw monthly data was 16.7% in November of 2015. These figures are very high for a bias rating. The fact that during one month the assumed error contained in live poll data was approximately 5 times the average margin of error is astounding and frankly makes us question much of the assumed best-practices for data collection and statistical analysis if based on faulty assumptions.
You should also note that such polls are conducted by extremely well reputed entities. A partial list includes CBS, New York Times, Monmouth University, Fox News, and Gallup. As these entities are so well reputed their polls get news attention and are cited as reliable sources. Having monthly biases that can reach over 10 percentage points makes many of these polls simply unreliable and draws into question analysis based off of them. This goes beyond the topic of Congress and into every political topic where the media holds a strong bias that is not equally held by the populace.
Another important point to highlight is how the percent of undecideds fluctuates during rapid changes in bias trends. After analyzing multiple datasets, it seems like many people when faced with a situation that forces them to decide between supporting a seemingly socially unacceptable topic or not will chose to ‘skip’ the question – or they will declare themselves neutral, undecided, don’t recall, or similar. This is a fairly obvious way to avoid social pressure all together.
In another post, we saw that the number of ‘undecideds’ in the 2016 US Presidential race appears to be at the highest on record. This makes sense given the assumed extremely high Social Desirability Bias working in Clinton’s favor and to Trump’s detriment.
Similarly, we see the percent of undecideds increase as the negative bias against the Congress increases, as can be seen in the following chart.
Chart 2: Net Bias against US Congress compared with Average Percent Undecideds and Neutrals of Live and Anonymous Polls, 6 month moving average
Source: Realclearpolitics
The percent of undecideds tends to move in-line with the net bias rating. Note that the right axis is inverted, so while the net bias worsens for Congress on the left axis, the percent of undecideds is increasing on the right axis. This is exactly what is expected to occur.
A very strong specific and unexpected event, the Republicans gaining control of the Congress, occurred which was covered by the media as a negative event. Many understood quickly that approving of Congress in such an environment would be questionable socially, especially as this topic was a major news item. The news changed over time from shock of Republicans gaining control to their defiance of Obama, who was covered in a positive light by the media. This topic of general disapproval of Congress was therefore continued as a news story far into 2015 as Congress and Obama fought over a variety of issues including Cuba and Iran.
In such an environment, approval of Congress collapsed in live polls by an approximate margin of 10 percentage points ahead of anonymous polls and the percent of undecideds and neutrals increased by approximately 2 percentage points. These rapid changes which can be verified in the previous chart show that many people temporarily ‘hid’ from socially awkward situations in undecideds and by changing their public responses. Then as the news item of Congress not being a socially acceptable entity worthy of support began to fade, perhaps being dislodged by Fall 2015 Republican and Democratic debates as a news focus, we see the net bias and percent of undecideds begin to revert towards their previous norms.