A picture of me standing at a lectern, working on a laptop computer, on the stage of the FWD50 digital government conference

Hi! I’m Alistair. I write surprisingly useful books, run unexpectedly interesting events, & build things humans need for the future.

Policing the mental genome

Netflix has always made recommendations based on categories (“upbeat romantic comedies”) but this April 1st, the company decided to take things to a whole new level with its overly precise categorizations.


I was discussing this prank over lunch this week, and folks I was with pointed out that the company has a “movie genome” model similar to “music genomes” of Spotify or Pandora. Netflix categorizes its movies by specific “genes” that can be used to define a movie—not just genres like “action” and “horror” but also things like “a strong female lead.”

Clearly, Netflix knows a lot about what you like, which is good for finding more you like. This might also be useful elsewhere: If you only enjoy films with happy endings, maybe you’re not well suited for a telemarketing job that has a lot of rejection. If you stop movies half-way through watching them all the time, maybe you have trouble focusing.

Can we detect bad mental genomes?

There’s a more serious aspect to this. I’d been thinking about it for a while, and was going to write something last year, and then the horrible events of Sandy Hook happened. Not wanting to jump on the bandwagon, I put the thoughts aside and chose to hug my daughter a little tighter instead.

Just to be clear: I am not advocating any of this. I’m just proposing a thought experiment because I think it needs to be discussed.

Yesterday’s conversation made me think about it again. Imagine that Newton gunman Adam Lanza had been a Netflix user. Imagine that, as a result, Netflix has his movie viewing history, and that this was a relatively distinctive mental genome. And imagine—no mean stretch of the imagination, this—that the company can find other viewers with similar “genes.”

Suppose Netflix’s query returns a few thousand people who are a close match. What would we do with the results? There are 525,000 police officers in the US. There are a decent number of social workers, too. It wouldn’t even need to come to the pre-emptive arrests of a Minority Report. Just a note to a student counsellor, or casual surveillance, or maybe even enough to warrant a surveillance of their purchases, emails, and phone calls.

Unthinkable, and a clear violation of our 9th amendment right to privacy? Maybe. Certainly the weak link in my thought experiment is that Lanza’s movie-viewing habits are somehow a fingerprint of his mental state. There’s good evidence that, given enough data, we’re easily predictable—but data can’t fulfil its prophecies.

Here’s the real, awkward, I-have-a-bad-feeling question: when does our desire to save a schoolful of definitely-innocents from possibly-death override the privacy of a maybe-guilty?

I worded it that way precisely because this is a game of probability. Lawyers have a name for coincidences that lack causal proof. They call it circumstantial evidence. But the bar for a jury is “reasonable doubt.” Reasonableness is a probability too. Juries weigh probabilities.

Guilty until proven innocent

Pulp cover by Marxchivist on Flickr. Used under a creative commons licenseLet’s take another example. Imagine that, twenty years from now, everyone has a mobile device that records their location. This isn’t a big leap of faith: Soon, it will be nearly impossible to function in society without one.

And then someone is robbed.

There’s no data on a person having been at the scene of the crime, but one person in the whole country happened to disable their device during the hours of the theft.

Are they guilty? That’s certainly a higher probability—one in 300 million—than the kind of evidence by which many juries convict people. It’s definitely enough to haul them in for questioning. Is lack of data grounds for suspicion?

Both prediction-of-crime and absence-of-alibi challenge the fundamental notion of “innocent until proven guilty,” a basic right we’ve had for a thousand years, since the Magna Carta.

Only disconnect

One approach to avoiding all of this analysis is to simply “disconnect.”  But that, too, might backfire. Following shootings in Norway and Colorado, reporters speculated that the shooters’ lack of presence on social platforms was an indicator of their intentions.

The German magazine Der Taggspiegel went so far as to point out that accused theater shooter James Holmes and Norwegian mass murder Anders Behring Breivik have common ground in their lack of Facebook profiles.

Vaccination by USACE Europe District on Flickr. Used under a creative commons license.The simple act of disconnecting becomes another gene in the mental genome, another piece of gristle for the mill of data.

Unless, that is, we do it in large numbers—letting us remain somewhat anonymous.

It’s a kind of “herd immunity” for online privacy.

Herd immunity (or community immunity) describes a form of immunity that occurs when the vaccination of a significant portion of a population (or herd) provides a measure of protection for individuals who have not developed immunity.

Maybe going off the grid is a kind of privacy vaccination; those of us who don’t need to, do so to protect those who need anonymity. It only works when enough people do it to avoid the nightmare scenario of guilty-by-lack-of-information.

At the same time, maybe it gives the bad guys somewhere to hide. As another mental experiment, imagine a town where every action by every citizen—including the act of accessing others’ logs—is logged, and open to the world. Would that be a better or worse world? Would it be free of crimes, since everyone would know they’re being watched? Or would it soon turn into a eugenic nightmare, a digital Lebensunwertes, where increasingly small deviations from societal norms are frowned upon, even excised?

There’s no right answer. But what’s clear is that we’re quickly becoming a new species, one half of which has a hive mind. The legal system of a hive mind would be vastly different from ours, governing things like encryption, public shaming, tolerance, prediction, freedom to speculate, and amnesty.

In discussing whether the US constitution should include an exhaustive list of rights afforded to citizens, Dr. Benjamin Rush said, “our Rights are not yet all known, why should we attempt to enumerate them?” I’m no constitutional scholar—I’m not even an American—but it seems to me that as we start to sequence our mental genomes, it’s time to enumerate some of their rights.

Some credits

Hat tip to Chris Taylor for the title of this post; he’s written about data privacy eloquently before and is good at picking the right titles, despite what he claims.

And a quick thanks to my trust T-Mobile Mifi giving me better bandwidth in a taxicab than I enjoy at home in Canada.


(If you wonder why I care about such things, you should read this post about onerous roaming fees for Canadians.)






3 responses to “Policing the mental genome”

  1. […] week at Cloud Connect in Santa Clara, I had a chance to sit down with Alistair Croll, founder of Solve for Interesting. I had been vaguely aware of the site 23andMe that tests and individual’s spit to help […]

  2. Chris Taylor Avatar

    Alistair, thanks for the title props. Really enjoyed reading it and actually wrote one in the same vein on Netflix, big data, privacy and our mental genome.

    This is really fascinating stuff that I don’ think most people are ‘connecting’ to their own lives. They should!

  3. Esteban Avatar

    What about Bonferroni’s principle? The probability of targeting mostly “false positives” is quite high. The implications are just terrifying. Being sent to a counselor just because I happen to like horror films (just an example) and that coincides with what the algorithm thinks are potential mass murderers is quite medieval.

    The danger lies on trying to build a “norm” for something that is definitely **not** normal (grabbing guns, walking into a school or other public space and opening fire) based on other behaviors that DO NOT imply THAT deviation from the norm.

    Also, for every measure there is a counter-measure, and if, just to follow your examples, the police start paying special attention to “people with no social network profiles” or “people who watch n type of movies” (or “people who read ‘the catcher in the rye’”, remember that? 😉 ), there are good chances that wrong doers will start acting normal in order not to b picked up by digital surveillance.