SMI Track-Record Making Correct Forecasts in Difficult Times
Analyzing social media and/or on-line activity to measure influence and intent is still relatively new. It has been used with success in finance and product releases, but its application to elections was unheard of until 2015. Since that time, a number of out-of-consensus forecasts have been proven correct and thereby establishing a new field. In this post, we highlight three forecasts based on Social Media Influence (SMI) and how they have ‘performed’ versus expectations and betting markets.
The first example is the 2016 US Presidential Election. From very early, the trifecta of polls, betting markets, and pundits declared that Hillary Clinton would be the next president. The consensus in favor of Clinton was, frankly, overwhelming. Betting markets and statisticians were placing the probability of her winning from around 65% to 99%. The day before the election, the probability of a Clinton victory was estimated at 99% at the Princeton Election Consortium, 98.1% at the Huffington Post, 89% at PredictWise, 88% at Daily Kos, 84% at the New York Times, and 68.5% at 538.com. These forecasts were done by well-respected publications and/or professional forecasters (and yet got it terribly wrong).
In this environment, we conducted our SMI analysis and found that Trump would not only win but do so by a relatively surprising margin with 306 electoral votes including wins in Michigan, Pennsylvania, Florida and Ohio. Our SMI analysis stood in complete contrast to most, which called for a very comfortable victory for Clinton. Yet, Trump won with 306 electoral votes and ‘stunned’ almost every forecaster and pundit.
Chart 1: Betting Market Based Probability of a Trump Victory in the 2016 US Presidential Race versus timing of SMI Forecast of 306 Electoral Vote Victory for Trump
Source: ZettaCap and PredictIt (for election probability graph data)
At the time, before election-day, calling for a Trump victory was essentially seen as extremely poor forecasting. However, by following what the populace was doing on-line in social media and not what political analysts thought or what polls declared, the SMI was able to cut through a tremendous amount of bias and determine true underlying support or at least intention of support. To-date, the SMI still seems to be the only method to have correctly called the election to the electoral vote.
In a second example, we then applied SMI analysis to the 2017 French Presidential Election. In December 2016, the SMI forecasted that a relatively under-reported and inexperienced candidate, Emmanuel Macron, would win the election due to his extremely strong SMI. This forecast at the time was seen as more out-of-consensus than a forecast calling for a significant Trump victory in the US, in other words it was seen as being extremely low probability by the consensus.
Two main parties, the Republicans and Socialists, have dominated French national politics for a generation. Plus, if any individual could successfully challenge these establishment parties, according to the general consensus, it would be Le Pen from the National Front Party, or the populist party in France. In a period when many see populism and problems with immigration policy on the rise, Le Pen was seen as a strong challenger to establishment politics.
Again, according to consensus, Macron did not fit the bill of a high probability candidate. He has very little political experience and is surprisingly young (39). He started a new political party, En Marche!, less than a year ago which by definition is untested. He is running as an Independent for the presidency, a very unusual strategy in France. In short, pundits and betting markets were not giving him much of a chance when the SMI forecast was made.
SMI analysis, however, did not take into consideration such traditional factors but only social media interactions. Macron consistently performed exceptionally well in social media in comparison to other candidates, who were apparently more suited for the job, more well-known, better financed and had stronger organizations around them.
Chart 2: Betting Market Based Probability of a Macron Victory in the 2017 French Presidential Race versus timing of SMI Forecast of Macron Win
Source: ZettaCap and PredictIt (for election probability graph data)
When we published our forecast of Macron winning the French election in December 2016, the probability of him winning was around 9%. It is currently around 36%, placing him in a close second place just behind Le Pen, though he was the clear leader for much of Februrary before this last week. In short, Macron went from a highly unlikely candidate to fighting for leadership in less than two months from the time that the SMI made a completely out-of-consensus forecast. To our knowledge, no other forecaster, method or analyst had called for his victory prior to the SMI, though most consider him a front-runner now.
To be fair, the French election is still months away and anything can happen. We cannot fully determine the performance of the SMI until ballots are cast. However, the mere fact that the SMI was able to identify which candidate would surge to become the leader from a position of an also-ran is impressive.
A third example of using SMI to forecast election results was recently conducted on the 2017 Dutch General Election which will take place within a month’s time. The pundits and betting markets have focused on two parties – the PVV (populist) and VVD (establishment). The general consensus is that one of these two will win and attempt to lead the next coalition government. The chances of a different party winning the most seats in parliament during the election fall dramatically after these two, as can be seen in this betting market odds quote.
Chart 3: Betting Market Quotes for 2017 Dutch General Election of which Party Will Win the most Seats
Source: Ladbrokers
SMI analysis, in contrast, shows that D66, a somewhat forgotten party which currently has the sixth most number of seats in parliament and which is struggling in the polls, has the best chances at winning the most seats in the upcoming election. This might be the most out-of-consensus forecast to-date based on the SMI.
The caveat here is that Dutch politics is very fluid, seemingly more so than in other countries. There are 28 parties on the March 15 ballot nearly half of which have been founded within the last three years, so there will likely be considerable froth in the election process. The SMI could change fairly quickly with so many parties and with a populist push both in the Netherlands and in Europe. Having given this disclaimer, the SMI of D66 is impressively strong especially considering its relative lack of media attention. And, at roughly 100 to 1 odds, it seems like a very attractive, though low probability, bet.
Lastly, we should point out that the SMI highlights those candidates who have the most influence on social media – there is no attempt to pick underdogs or long-shots. It just so turns out, however, that the global political environment has been extremely unusual in the last few years in that often the most traditional candidates, those preferred by political analysts, pundits, and polls, have ended up not being as popular on election-day. Additionally, the ‘surprise’ candidates who won and/or greatly exceeded expectations were exceptionally influential on social media and gave signals of such influence well before election-day (on social media), thereby offering a window when they could be identified as outperformers ahead of time.