After Hillary Clinton had led throughout most of the campaign, she was also ahead in the BBC poll of polls on Tuesday with 48% of the votes to Donald Trump’s 44%.
Number cruncher Nate Silver, of statistical analysis website FiveThirtyEight, wrote that morning that Mrs Clinton had a 71.4% chance of winning. The results, of course, were quite different.
Whilst pollsters’ reputations may have taken another hit, newer methods of prediction based on data from social media appear to have had more success.
BrandsEye, a tool that looks at people’s tweets, correctly predicted both the vote to leave the EU in June’s referendum and a Trump victory in the US election, but how did it do this?
They compared the volume of pro-Trump and pro-Clinton tweets in key battleground states. As the chart above shows, between 1 October and 7 November Trump’s popularity in key states on Twitter was greater than most of the traditional polls were suggesting.
It is easy to count the volume of tweets but how do you measure sentiment? The answer is using lots and lots of people.
The company took a sample of 200,000 tweets. These tweets were then checked by a crowd-sourced army logging in from their homes. Their task was simple; when presented with a tweet, decide if it was pro-Trump or pro-Clinton.
Using vast numbers of people is expensive but Jean Pierre Kloppers, the founder of DataEQ, insists it is a better option than using an algorithm, saying: “Algorithms find it hard to see how people feel. Using humans to read the tweets makes it far more accurate.”
For example, says Kloppers, an algorithm would find it harder to understand the meaning of a tweet like “I’m supporting Clinton to see her wearing striped pyjamas”, whereas a human can detect the irony or sarcasm.
To view the full article on BBC, click HERE