the dark side of marketing and analytics

too perfect for words..

“Data abstracted from the banks of every computer in the human system. Unthinkable complexity. Lines of light ranged in the nonspace of the mind, clusters and constellations of data. Like city lights, receding.” Neuromancer: William Gibson

As I’ve mentioned before, once you get enough data in one place you can throw processor cycles at and reach some interesting conclusions pretty fast. What’s stopped us from doing this in the past was two simple factors, not enough accessible data and the price of processor cycles.
Well thanks to the mindless data collection of various systems, and the self provided data-hoards of places like Facebook we’re not short of readily accessible data about people and what they get up to. Thanks to the diligent efforts of processor designers everywhere and the easy availability of cloud compute stacks, processor cycles are as cheap as chips.

This confluence of data and subject is real tempting to some sorts of minds. From the relatively benign commercially motivated and the more risky law enforcement/public order focus groups to the mildly terrifying lets-manipulate-the-public-gestalt-and-get-our-evil-asses-elected crowd who we *all* know are out there.

Marketing and commerce are easy, we know what they want, our money. We can work out how they get it too, they sell us something. Now we have the tools to model and personify us to such an extent that they know who we are, in an abstract way, and can make a good stab at what we want and what the consequences of our choices will be (see posts passim). To be honest, whilst this feels creepy, it’s probably not that evil. Being shown an advert for exactly what I want when I want it does short circuit a lot of shopping based crap when I look at things which aren’t quite right or I don’t know what I want. When buying presents, being shown things that the person I'm buying for might like is a fucking god send to be honest. It’s either that or I buy them what *I* might want and that’s not the same thing at all. These pure commercial motives apparently become a lot creepier when they try to manipulate what I want and when I want it. To trigger a buying event for something I don’t really want but that the seller has loads left of and wants to shift them is part of the raison d’être of retail since the dawn of shopping* and the idea of being shown things I might like at a great price is both a benefit and a risk of the fusion of data and content in the new retail “free money machine” which is shaping up out there. 

Law enforcement/public order motives are a lot more complex and can be a lot more scary. There is a lot to be scared about, in the UK there are over four million CCTV cameras out there and there have been proposals kicking around for years to link them into one god like CCTV database so they can track people around the place via facial recognition and store the images for evidential purposes. I used to joke that these proposals where all sponsored by storage manufacturers since that is a whole lot of disc space but who knows. On top of that in the UK there are automatic number plate recognition systems (APNR) plugged into every police car which tool around and they already are used to track people around the place (and on the plus side of things, haul idiots without insurance and tax off the roads but I digress) and now that movement into our online lives is becoming more apparent with the FBI announcing that they are paying for tools to monitor Facebook for criminal activity which is logical but more sinister. CCTV and APNR systems can at best only work out who and I and where I am/where I’ve been. They don’t know where I'm going and they don’t know what I'm doing. But analysis of Facebook and other sources may be able to bridge that gap particularly if you link my online data with those other systems (and my ISP logs, and my telco records and my TV choices and my credit card bills etc etc) to build very detailed profiles of who, what, where and why I'm doing stuff and make law enforcement decisions, which in some cases can be deadly decisions based on analysis.

Oh there are always the “nowt to hide lad, nowt to fear” bridge out there (see posts passim) but we’ve all got things we’d like to hide and frankly I don’t trust government/police/intelligence agencies/the military as far as I could throw them. Given their track record on protecting our data I wouldn’t trust them with string or a Sudoku book but that’s a given for most of us.

No matter how scary the law enforcement/public order market for big data is, at least it’s understandable and in the vast majority of cases they do mean well, their road to over analytics hell is genuinely paved with good intentions and you can at least understand most of their motives even if you vehemently disagree with their methods and risk assessments.

The ones who scare the living crap out of me are those who want to change public opinion by analysis of the who/where/what/why and tailoring messages designed to pervert opinion by providing seemingly inconsequential facts or statements which alter facets of our own view of the world to a wider effect. I'm probably not explaining myself very well here but think of it this way. Imagine you wanted to effect a wide societal change. You wanted people to feel positively about a person or a statement or idea. Previously the amount of effort required to change the mind of the whole public about something was utterly massive, requiring mass media bombardment, messaging in all channels, high visibility and persistent and pervasive effort over years or decades, think smoking or drink driving or war time propaganda. It’s not subtle but it does work, for good or for bad, but at least everyone knows you are doing it and it costs a bomb. Because it’s slow people can develop their own opinions on the subject and a bit like a mental immune response, a least some degree of value judgement and debate can take place. This industry has been speeding up since Vance Packard cracked a bit of a wedge in the door with his book the people shapers in 1977 and we’ve all become more accustomed to it since then. Exposes on the black propaganda departments running in WW2 and the collusion between media organisations and government during the cold war etc are common tropes in many a cheap thriller and pub based paranoid conversation. But it’s still not cheap, it’s still not effective, it’s still crude and it’s still not guaranteed. It’s mass market for the masses or high price lobbying/blackmail behind closed doors. 

What if that were to change, what if you could combine the expensive bespoke nature of lobbying/blackmail with the mass market reach of modern digital media? Imagine you now want to effect a wide scale societal change. You have access to everyone’s patterns of behaviours, you can break your message into factoids, opinionettes, small scale but outright lies, fake personal experiences and targetted adverts. You can place them to maximum effect to very tightly defined ranges of digital personas, you can storyboard the progression of the change you want to make and what reinforcing messages you need from your bought-and-paid-for public figures and when. Best of all, you can now do this automatically. You can insert side bar adverts with sinister messages where and when you want them, trusting on the new phenomena of “advert blindness” to let the message slide past (oh how we laughed about subliminal messages in images, it doesn’t work like that) and back it up with relevant op-ed, tailored news articles, single message blog posting and confirmatory feedback loops. All automatic, all costed to the penny and as near guaranteed as any other commercial proposition. What would that concept be worth to the lets-manipulate-the-public-gestalt-and-get-our-evil-asses-elected lot? What would it be worth to those with motives less benign even than them? Hell what’s the dark side of marketing technology worth to those with a huge pile of unsellable widgets they need to shift?

It’s happening now, it’s always been happening to one degree or another, all that’s changing is that technology is joining up all the dots and making it more realistic, more cost effective and more achievable than ever before. It's important for us as technologists to understand the implications of what we do and the unintended consequences of our capabilities. We can design things which whilst on the surface are harmless or even benificial but that our innovations can be misued. This is true of every field of endeavour of course and in due course effective "counters" will be developed or the mass indifference of the public will prove yet again to be harder to shift or damage than anyone believed. However in this case technology is moving very fast and the usual immune systems don't seem to have caught up yet. Prepare for interesting times.

* The most visible example of these in modern real world shopping are the end of aisle counters in supermarkets, known as FMTC  bays which stands for Fast Moving Trade Counters to those in the trade and Fuck Me That’s Cheap to everyone else in the world. These are high visibility areas of the shop where the management can reactively load up stuff which isn’t shifting and price it to go. The whole online retail world can be compared with FMTC from one point of view.