The Social Web And The Digital Panopticon – TechCrunch

I don’t much care for modern “yellow peril” stories. Such stories rely on the fact that China is a foreign country with a billion and a half people in it that still reflexively scares Americans.

It makes it very easy to write alarmist “trend pieces” where a few headlines get blown up into a story about how a whole country is filled with drivers intentionally killing pedestrians they hit to avoid lawsuits or whatnot.

Even so, I approve for the most part of the coverage of China’s Social Credit System, with the government paying tech companies to scour through everything they can find of a citizen’s financial and professional history, use secret-sauce algorithms to turn that history into a flat numerical score, and then make that score available to the general public for the purpose of everyone being able to judge everyone else.

Sure, the coverage is to a degree exaggerated. There are no plans, yet, to use SCS comprehensively crunch the numbers on how liked or disliked you are on social media, how loyal or disloyal you are to the government, how of your friends are the “right” or “wrong” people, etc. But the technology to do so clearly exists and companies are experimenting with “social network credit scores” in many places, including China.

The Chinese government isn’t using Big Data algorithms to automatically detect dissident thought before it happens a la Minority Report, as far as we know, but scouring through your entire financial, professional and legal history to tell all future employers whether you can be trusted with a job is surely scary enough.

China’s SCS is only a mild extrapolation of where our society’s lust for Big Data is already taking us. We’re scared at the thought of that kind of power being in the hands of a foreign authoritarian government, without noticing that that’s power already held by unaccountable private actors here in the “free” West.

Interestingly, China and the United States have this in common–they’re both in favor of relatively unrestricted data-hoovering of their citizens done to build Big Data model of those citizens to predict their behavior and control it with incentives. It’s just that China’s oligopoly of companies doing the data-mining are acting under the explicit direction of the state, whereas in the United States it’s more often the state acting under the implicit direction of the oligopoly.

Complaints about FICO scoring and the concept of a “credit rating” aren’t new. I’ve been following one of my favorite bloggers, Fred Clark, fighting the good fight against the Experian/Equifax/Transunion protection racket for years.

Up till now we’ve had regulations restricting what kind of information can go into a credit report and what kind of information can come out of it.

SCS proposes to ignore such petty consumer protections, vacuuming up everything from points on your driver’s license to negative performance reviews at your last job and spitting out a numerical score that anyone at all can look up — not just potential landlords and employers but potential romantic partners, roommates, neighbors or random strangers who want to attack your reputation.

SCS is drawing attention because it’s being pushed by the same people who erected the Great Firewall of China, people whom the Internet can mobilize against as enemies of freedom, enemies of the future, enemies of the West, etc.

But make no mistake about it. The oppressive world that SCS implies, the Orwellian nightmare of being constantly surveilled and constantly judged, is not an attack on the Internet as it currently exists. It’s the Internet’s logical endpoint. What data-broker sites like Spokeo clumsily try to pretend to do, China simply wants to do for real.

As with so many other complex metaphors from the past, Foucault’s idea of the “Panopticon” in Discipline and Punish has become depressingly easy to explain because the Internet has made it depressingly literal.

Foucault said you can build a prison without walls just by letting your prisoner know he’s always being watched. In real life in 2015, you can totally control someone’s behavior by training them to tweet or instagram every tiny thing they do and see how many likes it gets.

Foucault said you can have a jail without any jailers as long as all the prisoners know there might be someone watching at all times, even if they can’t see who it is. In real life in 2015, we can all be each other’s wardens as members of the amorphous mob that hands out likes and dislikes.

Indeed the only way to not feel totally helpless in the face of the invisible mob’s upvotes and downvotes is to aggressively participate in voting yourself–and so like the crabs in the bucket we all cooperate in crowdsourced self-imprisonment.

Does this sound overwrought? Melodramatic? Does it trigger an urge in you to say “It’s just online BS, just don’t read the comments”?

Well, consider that even though New York City can enact a law against employers pulling credit checks against you, no one can do anything to stop a hiring manager from Google-stalking you.

Consider hiring managers outright admitting that finding negative remarks about you on social media from people they don’t know can still sink your chances of a job offer. Consider the stories of people like Lindsey Stone losing their jobs after a tasteless photo went massively viral. Consider organized attempts to leak information from people’s private lives–nude photos from a romantic relationship, a past career in sex work–in order to poison their reputation in their future careers.

Look at the world of Yelp reviews, which makes drama over hits to one’s credit rating look tame. Look at the cutthroat warfare of retaliating to bad reviews with bad reviews we see small businesses engaging in. Look at how one possibly faked review can lead to an explosion of one-star reviews and negative press.

Look at how the fear people have of a damaged credit rating leads to profits for companies that sell access to your credit report–now look at the far greater fear companies have of Yelp’s power over them, leading to allegations of “Yelp mafia” tactics of filtering negative reviews in return for cash.

Look at the panic and uproar that erupted over the announcement of Peeple, the “Yelp for People” app, an attempt to bootstrap a version of SCS using purely user-generated content. At this point it almost doesn’t matter if the proposed clearinghouse for people’s “ratings” of other people they know comes out or not.

The idea is simple enough and the reasons it’s frightening are obvious enough that it was already thoroughly mocked in the “Meow Meow Beanz” episode of Community, and yet that didn’t stop Nicole McCullough and Julia Cordray from moving ahead with Peeple.

Even if Peeple’s founders bow to public pressure and scrap the project or try to radically rework it to be purely “positive,” nothing stops the next set of founders from creating a more “neutral” or outright malicious version of Peeple to feed our dark appetite for saying nasty things about each other behind each other’s backs.

It’s the same logic, after all, behind apps like Yik Yak or the subculture around comment sections and chans. No matter how many times we write think-pieces exhorting each other to pay no attention to the voice of the anonymous mob we still find ourselves jerked around by it.

Human nature is to be deeply invested in knowing what “people” in general think, in getting some reified abstract consensus of the population as a whole as a neat tidy number. We feel like we’re validated if we get to vote on what “we” as a whole think and we treat that consensus as more valid than any individual opinion, no matter how flawed that concept may be.

Our technology is happy to enable this obsession, aggregating and averaging and quantifying to our heart’s content, turning our collective opinion of restaurants and films and consumer products into tidy integers.

Cory Doctorow, tech prophet of our times, predicted all this back in 2003, with his novel Down and Out in the Magic Kingdom that introduced the concept of “Whuffie.” Doctorow imagined a post-scarcity world where everyone was guaranteed basic food and shelter.

Rather than ownership of material goods, economic life revolved entirely around a “reputation currency” called Whuffie, which, like favs on Twitter or likes on Facebook or stars on Yelp, represented nothing but what other people thought of you.

The Whuffie system promised to deliver us from the arbitrariness of who gets rich and who gets poor in the real-money economy and instead focus on what really mattered, which was your social standing among your peers.


Write a Reply or Comment:

Your email address will not be published.*


  • Mobile