The Social Web And The Digital Panopticon – TechCrunch
I donât much care for modern âyellow perilâ stories. Such stories rely on the fact that China is a foreign country with a billion and a half people in it that still reflexively scares Americans.
It makes it very easy to write alarmist âtrend piecesâ where a few headlines get blown up into a story about how a whole country is filled with drivers intentionally killing pedestrians they hit to avoid lawsuits or whatnot.
Even so, I approve for the most part of the coverage of Chinaâs Social Credit System, with the government paying tech companies to scour through everything they can find of a citizenâs financial and professional history, use secret-sauce algorithms to turn that history into a flat numerical score, and then make that score available to the general public for the purpose of everyone being able to judge everyone else.
Sure, the coverage is to a degree exaggerated. There are no plans, yet, to use SCS comprehensively crunch the numbers on how liked or disliked you are on social media, how loyal or disloyal you are to the government, how of your friends are the ârightâ or âwrongâ people, etc. But the technology to do so clearly exists and companies are experimenting with âsocial network credit scoresâ in many places, including China.
The Chinese government isnât using Big Data algorithms to automatically detect dissident thought before it happens a la Minority Report, as far as we know, but scouring through your entire financial, professional and legal history to tell all future employers whether you can be trusted with a job is surely scary enough.
Chinaâs SCS is only a mild extrapolation of where our societyâs lust for Big Data is already taking us. Weâre scared at the thought of that kind of power being in the hands of a foreign authoritarian government, without noticing that thatâs power already held by unaccountable private actors here in the âfreeâ West.
Interestingly, China and the United States have this in commonâtheyâre both in favor of relatively unrestricted data-hoovering of their citizens done to build Big Data model of those citizens to predict their behavior and control it with incentives. Itâs just that Chinaâs oligopoly of companies doing the data-mining are acting under the explicit direction of the state, whereas in the United States itâs more often the state acting under the implicit direction of the oligopoly.
Complaints about FICO scoring and the concept of a âcredit ratingâ arenât new. Iâve been following one of my favorite bloggers, Fred Clark, fighting the good fight against the Experian/Equifax/Transunion protection racket for years.
Up till now weâve had regulations restricting what kind of information can go into a credit report and what kind of information can come out of it.
SCS proposes to ignore such petty consumer protections, vacuuming up everything from points on your driverâs license to negative performance reviews at your last job and spitting out a numerical score that anyone at all can look up â not just potential landlords and employers but potential romantic partners, roommates, neighbors or random strangers who want to attack your reputation.
SCS is drawing attention because itâs being pushed by the same people who erected the Great Firewall of China, people whom the Internet can mobilize against as enemies of freedom, enemies of the future, enemies of the West, etc.
But make no mistake about it. The oppressive world that SCS implies, the Orwellian nightmare of being constantly surveilled and constantly judged, is not an attack on the Internet as it currently exists. Itâs the Internetâs logical endpoint. What data-broker sites like Spokeo clumsily try to pretend to do, China simply wants to do for real.
As with so many other complex metaphors from the past, Foucaultâs idea of the âPanopticonâ in Discipline and Punish has become depressingly easy to explain because the Internet has made it depressingly literal.
Foucault said you can build a prison without walls just by letting your prisoner know heâs always being watched. In real life in 2015, you can totally control someoneâs behavior by training them to tweet or instagram every tiny thing they do and see how many likes it gets.
Foucault said you can have a jail without any jailers as long as all the prisoners know there might be someone watching at all times, even if they canât see who it is. In real life in 2015, we can all be each otherâs wardens as members of the amorphous mob that hands out likes and dislikes.
Indeed the only way to not feel totally helpless in the face of the invisible mobâs upvotes and downvotes is to aggressively participate in voting yourselfâand so like the crabs in the bucket we all cooperate in crowdsourced self-imprisonment.
Does this sound overwrought? Melodramatic? Does it trigger an urge in you to say âItâs just online BS, just donât read the commentsâ?
Well, consider that even though New York City can enact a law against employers pulling credit checks against you, no one can do anything to stop a hiring manager from Google-stalking you.
Consider hiring managers outright admitting that finding negative remarks about you on social media from people they donât know can still sink your chances of a job offer. Consider the stories of people like Lindsey Stone losing their jobs after a tasteless photo went massively viral. Consider organized attempts to leak information from peopleâs private livesânude photos from a romantic relationship, a past career in sex workâin order to poison their reputation in their future careers.
Look at the world of Yelp reviews, which makes drama over hits to oneâs credit rating look tame. Look at the cutthroat warfare of retaliating to bad reviews with bad reviews we see small businesses engaging in. Look at how one possibly faked review can lead to an explosion of one-star reviews and negative press.
Look at how the fear people have of a damaged credit rating leads to profits for companies that sell access to your credit reportânow look at the far greater fear companies have of Yelpâs power over them, leading to allegations of âYelp mafiaâ tactics of filtering negative reviews in return for cash.
Look at the panic and uproar that erupted over the announcement of Peeple, the âYelp for Peopleâ app, an attempt to bootstrap a version of SCS using purely user-generated content. At this point it almost doesnât matter if the proposed clearinghouse for peopleâs âratingsâ of other people they know comes out or not.
The idea is simple enough and the reasons itâs frightening are obvious enough that it was already thoroughly mocked in the âMeow Meow Beanzâ episode of Community, and yet that didnât stop Nicole McCullough and Julia Cordray from moving ahead with Peeple.
Even if Peepleâs founders bow to public pressure and scrap the project or try to radically rework it to be purely âpositive,â nothing stops the next set of founders from creating a more âneutralâ or outright malicious version of Peeple to feed our dark appetite for saying nasty things about each other behind each otherâs backs.
Itâs the same logic, after all, behind apps like Yik Yak or the subculture around comment sections and chans. No matter how many times we write think-pieces exhorting each other to pay no attention to the voice of the anonymous mob we still find ourselves jerked around by it.
Human nature is to be deeply invested in knowing what âpeopleâ in general think, in getting some reified abstract consensus of the population as a whole as a neat tidy number. We feel like weâre validated if we get to vote on what âweâ as a whole think and we treat that consensus as more valid than any individual opinion, no matter how flawed that concept may be.
Our technology is happy to enable this obsession, aggregating and averaging and quantifying to our heartâs content, turning our collective opinion of restaurants and films and consumer products into tidy integers.
Cory Doctorow, tech prophet of our times, predicted all this back in 2003, with his novel Down and Out in the Magic Kingdom that introduced the concept of âWhuffie.â Doctorow imagined a post-scarcity world where everyone was guaranteed basic food and shelter.
Rather than ownership of material goods, economic life revolved entirely around a âreputation currencyâ called Whuffie, which, like favs on Twitter or likes on Facebook or stars on Yelp, represented nothing but what other people thought of you.
The Whuffie system promised to deliver us from the arbitrariness of who gets rich and who gets poor in the real-money economy and instead focus on what really mattered, which was your social standing among your peers.
It sounded appealing enough to some people that at TechCrunch 50 in 2009 the Whuffie Bank nonprofit was founded to try to make reputation currency into a reality.
Now, in 2015, the Whuffie Bank is dead and every attempt to implement something like WhuffieâKlout, Peeple, SCSâfeels less like a dream and more like a nightmare. Doctorow admits that he got the term âWhuffieâ from local slang at his high school for teenage âbrownie points.â
Anyone whoâs been in high school knows that the way the popularity game works is capricious and often vicious. That the process of scoring âpointsâ is not in any sense meritocratic and can easily be gamed by those unscrupulous and skilled enough to do so, and that if anything a âpopularity economyâ is one where the rich get richer and the poor get poorer far more reliably than even the real money economy.
One of my favorite responses to Doctorowâs Whuffie is Ben Rosenbaumâs 2010 short story âThe Guy Who Worked For Money,â positing that in the reputation economy of the future there will be some of us who long for the good old days of good old fashioned alienated labor and trading toil for coin because for some of us that might be less stressful and scary than spending all our time worrying what everyone thinks of us.
The âguy who works for moneyâ in Rosenbaumâs story does so by fleeing the ubiquitous reputation web in Europe for the un-networked hinterland of âChina and the âstans.â
In the real world of 2015, China is embracing the reputation-based web faster than Europe and âFacebook Zeroâ is working overtime to get the social web embedded everywhere else. Before long there may be nowhere left to flee.
Sarah Jeongâs The Internet of Garbage argues that despite all the appealing metaphors about size and space and bigness we use about the Internetâthe âelectronic frontierâ and so onâthe Internetâs real effect has been to make our world smaller.
Now, wherever you go and whatever you do, your reputation is what comes up on the first page of Google when someone looks up your name. Now we all live in the same small town, or high school, or prison, depending on your perspective.
Iâm high-profile enough that Iâve had the experience of people coordinating to broadcast smears about me and push them up my Google search results. I havenât gotten it nearly as badly as other people, some of whom have seen their entire careers implode over essentially nothing because they pissed off the wrong dedicated hate mob.
But even thatâs not really what concerns me. When quasi-famous people get slimed by other quasi-famous people thereâs at least an argument to be had, specific points to contest, evidence to grapple with.
What about when our existing paranoia over how one late payment affects our credit spills over into paranoia over how one errant tweet or photo or comment might give our Whuffie a hit? What about when you donât even have a list of Google search results to pore over, just an opaque number that jumps upward or downward as unpredictably as the stock market?
Every day we take another step toward that future. The whole world of social networking and the drive to monetize social networking by pulling useful data out of it is inexorably pulling us toward that future. And every step we take in the other directionâthe EUâs âright to be forgotten,â one city trying to limit the overuse of credit scores in decision-making, our transient moment of outrage over Peepleâseems laughably weak in comparison.
The EU is stepping up their dissent against the US-led ideal that more access to everyoneâs data will always make for a better world. Critics of the EUâs increasingly restrictive privacy policies worry that it will split the Internet into pieces, that it will throw a roadblock the wonderfully economically productive process of shrinking the world into a global village.
Perhaps we tech leaders in the United States need to think about the fact that in opposing such measures, weâre standing against our fellow First World democracies and standing with the authoritarian bogeyman of Chinaâs one-party state.
Itâs easy to get called a Luddite these days for advocating any proposal to limit the growth of the social web. But even if there is no stopping or even slowing this train weâre on, itâs at least worth thinking about the downsides of the destination. Even if we have to imagine it happening in Communist China before we get it.