How Crawled Data Gave One News Outlet the Edge in the Israeli Election

In the spring of 2015, as Israel prepared for general elections, virtually all of the mainstream media analysts believed that change was in the air.

Conventional wisdom at that time had it that the Israeli populace was ready to turn its back on Prime Minister Benjamin Netanyahu and the government led by his Likud Party since 2009.

Why Crawled Data Is Better for Gaging Sentiment

If only more news outlets had crawled the web, they would have seen – as the Globes newspaper did – that in the conversations surrounding the candidates, Netanyahu was receiving far more positive sentiment from the citizenry than any of the official polls showed.

Polls are only as good as their data samples reflect the whole, of course. Today there are many factors that can render “authoritative” political surveys inaccurate. With the rise of unlisted mobile phone numbers, “do not call” lists and the like, people are becoming harder to reach. And regardless, any time you’re dealing with responses to questions, you’re going to encounter biases that can skew the results.

To learn what the populace really thinks, professional researchers are increasingly turning to the open web, where we all volunteer our true feelings via comment threads, forum discussions and blog posts.

Discovering a Contrarian Story Line

Following several years of declining approval ratings for the hawkish incumbent, as the March 17 election neared, left-wing candidate Isaac Herzog was predicted to unseat Netanyahu. As CNN noted at the time, “Herzog’s Zionist Union party is in the lead, holding a four-seat edge over Prime Minister Benjamin Netanyahu’s Likud party.”

Suspecting that a surprise Netanyahu victory might be in the works, research analysis platform Buzzilla, a sister company of Webz.io, extracted and structured data from online discussions to provide a series of reports to Globes, a leading financial newspaper that also covers politics. Sure enough, the data showed Netanyahu pulling ahead as the election approached.

When applied to Webz.io’s crawled data, Buzzilla’s Brand Positioning Index (BPI) analysis model showed a solid lead for Netanyahu.

Webz.io’s data-as-a-service (DaaS) platform provides direct access to real time and historic data from millions of sources across the web, including discussions, forums, news items, comments, reviews and blog posts. Webz.io merges the data and delivers it in a structured format, either via query-specific API (News API) or via a firehose stream for processing upon import.

Buzzilla utilizes Webz.io’s comprehensive firehose to extract all of the data related to its social media research and customer listening needs. After indexing and filtering the firehose data, the Buzzilla team is able to focus on analysis and insights rather than sourcing data.

Examples of Israeli political content as extracted and structured by Webz.io

Visualizing Webz.io data using Buzzilla’s “Chatter Volume” module

The World Plays Catch-Up

In the days following the election, the media scrambled to answer the question of how Netanyahu pulled off his upset victory. “The result was a dramatic and unexpected victory for Netanyahu,” Reuters commented at the time. “Netanyahu’s move to the right, playing up fears of the spread of Islamist groups, promising no concessions to the Palestinians and raising alarm about growing support for Arab-Israeli parties, looks to have spurred his base into action.”

But thanks to Buzzilla’s shrewd use of Webz.io’s tools, the story was already out there on Globes (their coverage, in Hebrew, appears on these three pages).

How to Unearth Exceptional Insights

Here’s a breakdown of how Buzzilla performed the crawl and analysis that informed the coverage in Globes.

• Step one – Using the Webz.io query builder, Buzzilla’s analysts extracted thousands of posts on comment threads, forum discussions and blog posts.
• Step two – With the cleaned and structured data provided by Webz.io, the Buzzilla team tagged the conversations to enrich the data with key divisive political issues, audience segments and sentiment towards the candidates.
• Step three – Buzzilla applied the Brand Positioning Index (BPI) analysis model to the data and then used its proprietary tools to generate automated visualizations in the form of graphs, volume charts and maps.
• Step four – Buzzilla coordinated with the editorial team at Globes, submitting analyses and visualizations on a weekly basis to inform news coverage.

Ultimately, thanks to the data crawled and extracted using Webz.io’s tools, Globes was one of the only media outlets in the world to publish content questioning the polls that predicted a Netanyahu loss. Thanks to its relationship with Buzzilla, Globes was able to reinforce its place as a go-to, authoritative news source.

Ready to learn more? Check out Webz.io’s News API now.