In late 2017, Transport for London announced that it was revoking Uber’s operating licence in the city. The ban made headlines in number of international papers. One article, in the Daily Telegraph, reported on the results of a YouGov poll which found fewer than one in three Londoners thought that the ban on the ride-hailing app was wrong.
The Telegraph article noted that despite a social media “outcry” only 31% of Londoners felt that that Transport for London had taken the wrong decision. That’s all the mention that social media received. The traditional polling methodology was taken as the final word. Londoners, it seemed, were OK with the ban.
But a sentiment analysis conducted by DataEQ from 22 to 25 September (The ban was announced on 22 September) on Twitter, found that of the sentiment-bearing conversation, emanating from London about the ban, some 48%, were against it. And crucially, only 8% expressed support for it. Social media was telling a significantly different story to the YouGov results that so many papers ran with.
In recent years, the rise of populist movements has been energised by the notion that ruling elites have lost touch with ordinary people. However, critics of this notion have dismissed the so-called gap between politicians and the people as the reason for their success, often pointing to campaigns being fronted by a billionaires and Oxbridge alumni. But despite this irony, leaders, be it in politics or business, ought to remember that what they read in the newspapers or watch on TV may not reflect public sentiment, on a particular policy issue or company product.
Many critics have also scoffed at social media, probably unaware that Twitter and Facebook are goldmines of public opinion. People take to these platforms, usually without prompting, to volunteer their opinion on everything from the price of their broadband service to their criticism of a political candidate running for office. And unlike traditional polling methods, public opinion on social media is not framed by a questionnaire or limited to a small number of participants. Everyone with a mobile phone or internet connection is now able to have their say.
One reason why so many people have ignored social media as a source of public opinion may have something to do with the task of converting large amounts of unstructured data into decipherable and valuable insights. Even the technology that can assist in this process, natural language processing algorithms, has its own limitations. With an accuracy of at best 60%, these tools regularly fail to capture the nuances of our online posts which include sarcasm, slang, and emojis. Consequently, the data they provide is not accurate enough to inform key decisions in business and government.
The answer to this problem lies in combining algorithmic and human analysis. This allows one to process vast amounts of data, while the human integration allows for verification of the data. At the end of the day, despite impressive advancements in AI, when it comes to analysing human conversations, computers still struggle.
By using integrating humans into the process, a sentiment accuracy of 95% can be achieved. At DataEQ we do this by distributing a relevant sample of social media posts to a team of human contributors that we call ‘the Crowd’. The Crowd code the posts for sentiment and tag the topics contained in each posts. By doing this, one is able to ascertain not only how people feel about a particular issue, but more importantly what is driving them to feel this way.
Increasingly, businesses will look to social media sentiment data to compliment, and in some cases, replace, the traditional polling methods they’ve respected for so long.
Had Uber known that only 8% of Londoners supported the ban, and that such a gap existed between local government and their constituents, their interaction, both with the public and the City of London would have been bolstered by a compelling case.
–
DataEQ CEO JP Kloppers will be speaking at Fund Forum Berlin on 12 June on - Social sentiment, governance and measuring the expectation gap between what politicians, companies and CEOs say and do - and what people think.
This article was originally published here