Category: Data Journalism

Nuanced communities: Mapping ISIS support on Twitter

As every content marketer knows, creating resonant narratives requires intimate knowledge of the audience in question.

Nowhere is this more true than in attempts to counter the potent messaging of ISIS. The terrorist group is infamous for its ability to attract recruits from across the world to commit violence in the name of the ‘caliphate.’

ISIS has been a fixture in the global public consciousness for over two years, from its dramatic emergence in summer 2014 to facing near-decline earlier this year, followed by resurgence with its latest attack on Berlin just weeks ago. Long before Berlin, the group had already become notorious for the quality and power of its social media messaging, professionally produced videos and slick English-language print publications.

Concerned national governments and civil society groups have made numerous attempts to counter the ISIS narrative in various ways, ranging from shutting down followers’ Twitter accounts en masse to creating alternative narratives that aim to discredit the group, its ideology and its actions. But despite all these attempts, attacks against European cities remain a very real threat.

As another gloomy and blood-soaked year of ISIS activity comes to an end, the group shows no sign of fading away. Although it has lost physical territory in Iraq and Syria, the ongoing risk of the ISIS virtual caliphate persists.

A whole range of diverse factors determine an individual’s likelihood to become radicalised, many of which have been studied in significant depth elsewhere. Social media is not necessarily the most influential factor, but it undoubtedly plays a role.

RAND, a US-based think-tank, conducted a detailed research study, published in 2016, to examine ISIS support and opposition networks on Twitter, aiming to gather insights that could inform future counter-messaging efforts.

The study used a mixed-method analytics approach to map publicly available Twitter data from across the Arabic-speaking Twitter-verse. Specific techniques used were community detection algorithms to detect links between Twitter users that could signify the presence of interactive communities, along with social network analysis and lexical analysis to draw out key themes from among the chatter.

Research goals were to learn how to differentiate between ISIS opponents and supporters; to understand who they are and what they are saying; and to understand the connections between them while identifying the influencers.

Lexical analysis uncovered four major groups, or ‘meta-communities’ among the Arabic-speaking ISIS conversation on Twitter. These were Shia, Sunni, Syrian Mujahideen, and ISIS Supporters. They are characterised by certain distinct patterns in their tweets. Shia tend to condemn ISIS and hold positive views of Christians/the West/the international coalition fighting ISIS. This is unsurprising considering the long-standing hostility between Sunni and Shia Muslims and the fact that ISIS is a Sunni group.

The Syrian Mujahideen group is anti-Assad, holds mixed views of ISIS, and negative views of the coalition. ISIS supporters talk positively in bombastic overblown language about ISIS and the caliphate. They insult Shia, the Assad regime, and the West. Notably, their approach to social media strategy is by far the most sophisticated of the lot. And finally, the Sunni group is heavily divided along nationalistic lines, which includes most countries of the Arab world.

Key findings of interest

1. Unique audiences, essential nuance

Telling the difference in large datasets between ISIS supporters and opponents was key for this study. RAND researchers chose an easy way; Twitter users who tweeted the Arabic word for ‘Islamic State’ (الدولة ا س مية ) were considered to be supporters, while those who used the acronym ‘DAESH’ (داعش ) were opponents. This dividing line isn’t foolproof but, based on what’s known about the significance of these two Arabic terms, it seems a valid way to approach the task. Research discovered that although opponents outnumbered supporters six to one, the supporters were far more active, producing 50 % more tweets daily.

This could point to a couple of things. Firstly the outnumbering suggests that the majority of the Arab world (or at least the Twitter sphere) is anti-ISIS; while the volume of pro-ISIS tweets could suggest passionate support for the group, or on the other hand could point to the presence of armies of pro-ISIS bots or perhaps the use of astro-turfing. The latter two could be an interesting case for new research, especially in the present climate where the curtain has been lifted on use of social media bots, astro-turfing armies and persona management software.

2. Jordanian pilot, Turkish soldiers

The researchers also plotted Twitter activity levels for all four groups, between July 2014 (when ISIS emerged and announced itself to the world), to May 2015. Notable findings were firstly that both the anti-ISIS groups (Shia and Sunni States) showed similar activity patterns, suggesting that both were responding to the same ISIS-related events. All four groups experienced a large spike in activity in early February 2015, when ISIS released a video showing Jordanian pilot Moath al-Kasasbeh being burned alive.

After this event, the ISIS supporters activity decreased sharply, while the Syrian Mujahideen’s grew to almost match the Shia and Sunni States groups. Possible explanations (assuming the ISIS supporters are not bots) could include outrage at the murder of a fellow Muslim, and/or outrage at the way he was killed, burning, which is forbidden in the Qur’an. It would be interesting to compare the Twitter response to al-Kasasbeh’s murder with the response to another ISIS burning video, released last week, where two Turkish soldiers were killed.

This comparison could reveal further insights about the nature of the original 2015 spike; or reveal changing attitudes towards Turkey, which has started fighting against ISIS in recent months and has most likely become hated among the group’s supporters as a result.

3. Social media mavens

The ISIS supporters Twitter community analysed in the study showed particular features that made it distinct from the other groups. The supporters group members were more active than the other three groups (despite smaller numbers overall). They tweeted a lot of pro-ISIS terms and phrases, predictably. But most notable about this group was their fluency and command of advanced social media strategy, as shown by their use of certain terms on Twitter. In the study, the supporters group used disproportionately high levels of terms such as spread, link, breaking news, media office, and pictorial evidence.

In general, ISIS has always been exceptionally conversant with social media marketing tools and techniques, in fact far superior to the efforts of many national governments. I would be very interested to see a study that uncovers who exactly is responsible for the ISIS propaganda, what their backgrounds are, and how they were recruited and trained (if indeed they weren’t already expert in this area).

4. CVE insights from Twitter data

Finally, the report offers insights for policy-makers and for those engaged in online CVE efforts across the Arab world. The most important of these is a reiteration of the need for counter-messaging that’s not just tailored, but that shows deep levels of insight into the mindsets of its target audiences. Research like this can help reveal useful themes and connections to build upon.

Also, the ongoing efforts by Twitter to ban pro-ISIS accounts has undoubtedly driven many of them to other channels, most notoriously Telegram. Analysing activity on new channels would be of great use in revealing any shifts in ISIS supporters focus or mindset. Much in the landscape has changed since this report was released, and continues to do so at a rapid rate.

The data journalism landscape worldwide

What does data journalism look like the world over?

Finding data to work with is fairly straightforward in many Western countries, with increasing numbers of resources (such as the Guardian data blog) dedicated to assembling and showcasing varied datasets. No matter what topics you’re interested in, you will probably find relevant data to explore. But what about beyond, in countries where the value of open data has yet to make an impact? Or in countries where the leadership prefers to keep information under wraps. How do we begin digging it out?

In Turkey for example, the situation feels very different. Just recently I was working on a piece for for Middle East Eye about the trials, tribulations and triumphs of Turkish female software developers. I wanted to add an extra layer to the story by integrating a simple data visualisation. It would show the numbers of Turkish women in software careers.

My early efforts to find suitable data didn’t get very far. I Googled, asked fellow journalists (including Turkish speakers), and contacted organisations such as Kadin Yazilimci (Women Developers). The latter told me that they’re also looking for this data – and haven’t found it yet. I was at a dead end, running out of leads to find the information I wanted.

During a tech-related procrastination session, I discovered a fascinating blog called Source. It focuses on stories from the intersection of journalism and code. While browsing the articles, I came across one in particular that resonated in light of my earlier problems. The opening paragraph, with the irresistible words “Yangon tech hub” – couldn’t fail to hook me. I’ve been intrigued with Myanmar since visiting in 2012, just as the country was starting to open up.

Eva Constantaras, who wrote the piece, is a data journalist who also trains journalists in developing countries. She reflects on her experiences covering the November 2015 general elections in Myanmar; elections that represented its biggest leap towards democracy for decades.  The article highlights the substantial quantity of election coverage, but points out that complexity remains lacking. This is unsurprising as Myanmar only gained widespread internet access in the last couple of years.

Cross-border partnerships to collect and manage open data are a ‘natural next step’, according to Constantaras. In this kind of collaboration, local journalists work with international news teams to provide deeper coverage of stories, including those involving data. Local topics of interest would benefit from global visibility with coverage on international platforms. These partnerships could obviously benefit from more open access to local data.

I faced the same problem when I visited Jamaica last summer. The only data sources I could find were from external organisations like the UN or World Bank, or a few rusty spreadsheets from local online sources. But they weren’t up to date enough to be really useful.

It would be great to have a one-stop online portal for all existing global data sources, from inside countries as well as from international organisations. I don’t know if anything like this already exists, or is being developed, but if so, I’d like to hear about it.

UPDATE: Since writing this post, I found Dag Medya, Turkey’s first data journalism portal. It just got shortlisted for the 2016 Data Journalism awards, so is definitely worth keeping an eye on.



Mapping global passport power


Passport power can shape your life. A strong passport allows the holder to move smoothly through the world, breezing through borders with ease and opening doors for new opportunities in travel, work and investment.

But a weak passport spells endless rounds of visa applications, being denied, being treated with suspicion (in case you’re an economic migrant masquerading as a tourist), having to stump up large sums of money just so they can be sure you’ll go home again.

Around 200 passports exist today. All offer varying levels of travel flexibility. Some, like the German passport, can take their holders to 177 countries visa-free or visa on arrival. Others, like the Afghan passport, are practically useless, allowing entry across a mere 25 national borders.

[embeddoc url=”” download=”all” viewer=”microsoft”]

The aim of this project is to map global passport power using CartoDB. The original data came from a study called the Visa Restrictions Index, created by Henley & Partners. It was embedded in a PDF, making it impossible to access by scraping or copy-pasting. This was a chance for me to try out Tabula, an invaluable tool that extracts raw data in table form from within PDFs. Tabula worked really well in this case. It pulled the data neatly into Excel format, with the minimum of additional tweaking needed. Now I had a nicely formatted Excel sheet ready to work with.

The dataset contains three columns: Rank, Country Name, and Score (i.e. how many countries the passport holder can enter visa-free). I can imagine the final visualisation involving a world map, with the ranks perhaps mapped as pins (or circles) of increasing sizes (smallest for weakest passports, biggest for strongest). Circles would be geographically located, with the individual score showing up inside each circle.

Here’s how it ended up. I like how the chloropleth feature looks; it seems to work well for this particular map. From looking at this map I can see some starting points for potential stories involving national image, economic power, public diplomacy and more.

Discovering CartoDB

I’m in Marrakesh, Morocco this week. While here, I’m taking a break from D3. I’ve been trying out the mapping tool CartoDB. It’s sophisticated and full of interesting features that do much of the heavy lifting on the user’s behalf.

Here’s a quick and easy map I made just now. It uses a CartoDB dataset that shows the most and least populated places across the globe. I chose the intensity visualisation to display this data. That seemed to be the most intuitive way to show population density. It’s quite easy to change the colours, background layers and other features for this map.

I’m pretty excited about exploring CartoDB in more detail. For my next project, I’m going to use it to map global passport power. Watch this space!

Super-simple D3 bar chart

No, it’s not quite a visually stunning world map or amazing graphic artistry, sadly.

BUT nevertheless –  here’s a little data visualisation that I wrote today with D3:


d3 bar chart data



It doesn’t look like much but took a ton of code to produce (well probably not, but it certainly felt like it). I had fun playing with the pretty colours, finally settling on a blue/purple scheme. The bar colours turn more blue the larger the number and more purple the smaller the number.

The dataset is just a simple array of numbers that I threw in. There’s a touch of CSS added in the document head to make the bits play nicely together. Other than that, this was an exercise in learning how to use SVGs, how to bind data onto elements in the DOM, and how to style the results in a very basic way. This is fascinating stuff. I love the way it can give me the potential to mingle coding with journalism and tell stronger, deeper stories.

I can see those gorgeous interactive maps awaiting me in the not-so-distant future… 😀