I’m in Prague this week for WebExpo. Being largely detached from the constant interruptions of a North American timezone has given me a chance to think about big data from a somewhat more detached perspective.
I’ve got three things I want to spend some time writing about in the coming weeks, and probably touching on at Strata in New York later next month:
- The increased predictive accuracy that comes from data will run head-on into businesses that try to amortize risk, because more prediction means less risk.
- The fact that a massive amount of correlation can sometimes be taken as a substitute for causality is at the root of many moral issues with technology, because we infringe on the negative rights of the individual (i.e. to not be imprisoned) in favor of the positive rights of the many (i.e. to incarcerate someone who is likely a bad person.)
- The confusion between privacy, freedom to act, and other terms that often get commingled. Is privacy simply the right to not have people know what we’re doing or saying? And what should we demand of those who use our private data? (I think it’s that the use be necessary; commensurate; and intended; more on that later.)
Anyway, I’ve got a lot of hastily-scribbled thoughts and rough-shod ideas I need to flesh out over beers with smart people—always the best way to validate or repudiate things, IMHO. Much of this has been prompted by e-mail discussions with folks including Chris Taylor, who’s been writing interesting things about the subject and has been nice enough to share some of his stuff with me for comment beforehand.
I’ve written up some initial thoughts on the first one: prediction running headlong into amortization, how that changes economics, and why big data might make us all Tea Partiers in the future. It’s unminted and unpolished, so I’m hoping for a lot of feedback or push-back. But as a friend of mine used to say, “there’s a pony in here somewhere.”