Month: June 2017

The Attention Index — An open source algorithm for ranking premium media

At Kaleida we’ve been developing a new method for measuring media — an algorithm that values quality and impact. We’re opening up the project with a Creative Commons license today so we can start collaborating with other orgs interested in using the data.

The Attention Index algorithm considers editorial decisions and social activity to derive attention scores.

The idea for Kaleida when we started the company a year ago was to reinvent the way media is measured.

The media ecosystem has been defined by reach, impressions, and clicks for about 20 years. From our experience much of the value that goes into producing quality journalism gets lost by those types of metrics, and the effect has been disastrous for the industry.

Yet it seems completely counterintuitive that media orgs would struggle given the massive opportunity the Internet offers. Over 3.5B people are connected now. Fast mobile networks reach 84% of the global population. And digital ad spending is expected to reach $224 billion globally in 2017.

If quality journalism outlets could demonstrate how much impact they have in the world then lucrative business models would surely find them.

So, we collected tons of data about the media and began analysing it. You can see our machine doing its magic throughout the day every day on kaleida.com, creating a map of interests in the world. The data shows what matters to whom, when, where and how much. That was the first phase of our plan to reinvent media measurement.

The second phase is beginning to unfold now — The Attention Index.

The Attention Index puts all this activity into context. It scores and ranks behaviours in relative terms so it’s easy to compare what’s working vs what’s not working. For example, it considers whether a publisher thinks something is particularly important and uses reader response to see if the public agrees.

https://kaleida.github.io/attention-index/

We’ve started publishing raw data for people to download and explore. The May 2017 CSV includes 128,000 article headlines, URLs, editorial promotion data, social data and attention scores. Every month we’ll publish the data we’ve collected and scored with the algorithm.

The math and methodology is public, too. Kaleida CTO Graham Tackley has documented step-by-step how he applied various statistics and arrived at these results.

And, of course, everyone is invited to join the community and to contribute. The Attention Index is published with a Creative Commons license with support from Google’s DNI fund.

The Attention Index algo — Full exploration and sample data is available via a Jupyter Notebook.

There’s a lot more to do. We started by sharing our work with several media academics in the US and the UK. The data and the algorithm will require more scrutiny, more contributors, more data and more use cases.

Opening it up for public use will help us get that kind of engagement and, hopefully, make the Attention Index an industry standard.

Everyone we talk to in the media business is frustrated with many of the same things. Fighting to lead on metrics you don’t believe in and that don’t really serve your business goals is right near the top of the list.

The lower-end commodity metrics will always be useful, but it’s time to recognize things that matter and to talk about them with shared language.

Keep an eye on the project and be sure to get in touch if you want to get involved or if you have questions about it.

UK election attention was far stronger on the left than the right

The data suggests mainstream and alt-media coverage and the amount of sharing of that coverage on Facebook may have played a role in the outcome.

We’re finding some fascinating connections between media attention and realworld events. That doesn’t mean one causes the other, necessarily, but sometimes it might.

Last week’s UK election outcome was yet another example.

Source: Kaleida

Early campaign reports from Conservative-led voices that Theresa May would win a landslide victory were not apparent in our data. Actually, the race looked very close in terms of media attention.

After an initial focus on May when she announced the snap election in mid-April the amount of coverage and the volume of social activity in response to that coverage appeared to even out. Some days Corbyn took the lead. Some days May was winning.

The interesting trend we found in our data was the scale of attention across the political spectrum. Attention tipped to the left far more than the right.

Source: Kaleida Data, 2017

In some research we did with The Guardian we scanned a selection of alt-media sources and compared how well they performed against mainstream publishers. Articles posted by activist blogs like Another Angry Voice often outperformed UK election coverage by the BBC, The Guardian, The Telegraph and even the big US publishers like NYT, CNN and Fox News.

They must have had an impact. But there’s another interesting story in the data.

The Independent and The Guardian influenced significantly more attention than The Telegraph, the Daily Mail and The Times. They even beat out the BBC, Buzzfeed and HuffPost. It was the left-leaning mainstream national news orgs that produced the most articles. And it was those articles that received the most shares on Facebook compared to coverage by right-leaning publishers.

Similarly, the left-leaning alt-media sources dominated the right. Westmonster, Breitbart, Guido Fawkes and Your News Wire were not as successful in their UK election coverage compared to their left-leaning counterparts, by these measures.

Now, we didn’t make a prediction prior to the UK election. We learned from our French election predictions that we needed more data, particularly when a race is close. In that case we overestimated the power of English-language media attention on the outcome of the French vote.

In the case of the UK election we not only needed more alt-media sources to measure attention, we knew nuance in how we measure attention was required, too. There are things that need to be taken into account such as the impact of a publisher’s coverage in proportion to the size of their readership and the number of likes they have on Facebook.

If you’re interested in this data we will make it all public soon, or you can get in touch. We’re happy to share it.

The impact of alt-media on elections

In a collaboration with The Guardian published today we looked at several alt-media sources that are having an impact on the UK’s general election debate.


We found some remarkable figures. Many of the articles published by individuals with strong left or right political views are being shared considerably more than articles by the biggest national news orgs in the UK, including The Guardian, BBC, and The Independent. Sources ranged across the spectrum from The Canary and Skwawkbox to Westmonster and Breitbart.

Source: Kaleida Data, 2017

Another Angry Voice is a Yorkshire-based site published by Thomas Clark. He holds 3 of the Top 10 most-shared articles about the election. The Independent and indy100 perform well compared to others, but most of their most successful articles have earned less than half of Clark’s top-performing post at over 100,000 shares.

Guardian journalist Robert Booth spoke to the university student behind Evolve Politics, Matthew Turner. He captured what we’re seeing brilliantly,

“This kind of dynamic activism is new to the media,” Turner said. “Stories that go viral are stories that you can rally around. I think the vast majority of readers like us because we light a fire in their belly.

Readers of the mainstream media tend to not get that nowadays. We are the ones offering the fight.”

That sentiment was echoed by both the left and the right alt-media sources.

The data behind The Guardian’s report can be found in our UK election coverage explorer. If you are interested in a full set of results to explore on your own, please get in touch. We would be happy to share more data.