Gullibility, squared

Infographics are all the rage on Twitter (well, the pol-sci segment of Twitter, anyways). They can be attached to a single tweet and provide dramatic information in a compact, visually appealing package.

Whether the information is precise, applicable, or even remotely true, is often a secondary consideration. At best, the tweet will also include a link to an article explaining the source of the graphic – but as often as not, all you get is a vague reference in the fine print at the bottom, which may or may not enable you to track down the source.

And yet, people who preach skepticism towards Internet rumours, will frequently share these “fascinating” pictures with nary a second thought (and yes, I’ve been guilty of this myself) – presumably on the unpoken assumption that the information must surely be credible if someone put that much effort into presenting it in the form of a purty picture.

One graphic making the rounds today – particularly after being shared by the Independent’s i100 site – is a map of “Where Isis supporters tweet from“. Not surprisingly, many of the people sharing it point out that the US is as high as fourth in the table, with the UK ninth.

However, it’s enough to read the title of the actual graphic to spot a tell-tale word that was omitted from the article title: “Top locations CLAIMED by Twitter users supporting ISIS in 2015.

Because, as the actual report points out:

[T]he only totally reliable method of geo-locating users is to obtain coordinates provided when a user has enabled the location feature on his or her smartphone. (…) Unsurprisingly, very few users in the dataset opted to enable coordinates. (…) Out of the 20,000 users in the Demographics Dataset, 292 had enabled location on at least one tweet out of their last 200, or 1.5 percent.

In other words, this report has identified less than 300 ISIS-supporting accounts* worldwide (in a dataset which they estimate to contain about 40% of all ISIS accounts) where the account’s location can be determined with any degree of certainty. And how many of these were in the US or the UK?

None of the location-enabled users were based in the United States; [while there was] one in the United Kingdom,

Well, where did this number of 404 US supporters come from, then? Well, this is simply the number of ISIS-supporting accounts* that have entered “United States” (or variations thereof, or a city in the US) in their profile.
It feels embarassingly unnecessary to even have to point out how ridiculously useless this is as any sort of even remotely reliable indicator of a person’s true location, when anyone is perfectly free to enter “Buckingham Palace”, “Ruritania”, or “the planet Tralfamadore” as their location.
The report laconically admits that:

We are reasonably certain some ISIS supporters deceptively listed locations in the United States in order to create the appearance of a homeland threat.

which, yeah, you’d think – not to mention the fact that a lot of the locations may be from foreign fighters that have travelled to Iraq/Syria while deliberately or accidentally retaining their old location text.

However, the report then blithely goes on to state that:

Nevertheless, the location field was the only method that produced a confidence-inspiring result.

(which, of course, is statistician-speak for “These data are largely useless, but they’re the best we’ve got, so we’re going to use them anyway and hope nobody notices”).
And just in case uncritical belief in the publically distributed information from terrorist supporters wasn’t inaccurate enough, we get this extra little gem:

Users who listed “Islamic State” as their location were considered to be in either Syria or Iraq, and we assigned them to one or the other following the two-to-one distribution noted in the location-enabled findings.

despite the pretty obvious fact that a lot of ISIS supporters will list their location as “Islamic State” on purely ideological og metaphorical grounds, regardless of where in the world they’re located.

So the moral of the story is, yet again, to not believe things people tell you on the Internet: Even if it comes from a so-called ‘reputable source’, it may be based solely on what other people told them on the Internet.

*) And no, I haven’t checked the validity of their methods to actually identify an account as “ISIS-supporting”. There’s a limit to how many levels of debunking is necessary, after all.

(Thanks to Hans-Petter Halvorsen for the tip about this report’s hopelessly inadequate methodology.)