by Paul Rowan Brian · April 12, 2019
Advanced technology isn’t just getting out of control, it’s being used as a means of control—big time. That’s the key message of Shoshana Zuboff’s new book where the Harvard professor emerita and business analyst argues that Big Tech is determined to fully commodify, control, and co-opt human experiences to provide raw behavioral data that sustains its massive profits and power.
Put simply, Zuboff contends that surveillance capitalism is the constant tracking, analysis, and attempted modification of human behavior for the profit of tech giants who trade in what she terms “behavioral futures markets,” places where knowing what people are very likely to do tomorrow or next year is of enormous value to those trying to sell a product or service.
Surveillance capitalism renders human behavior by tracking, measuring, and analyzing from your smartphone to your smart home—from browsing the internet to private messages or emails with a colleague. This relatively new, dominant force intrudes through cookies and privacy permissions that in most cases must be accepted for a service to work properly or at all, including, for example, many smart home security systems.
Zuboff distinguishes surveillance capitalism at the outset from information capitalism. Whereas information capitalism makes money from information you provide, surveillance capitalism disguises itself in intimidating terms of service agreements and actually nudges your behavior so that you do what it wants in various ways, perpetuating a feedback loop of predatory control and emotional espionage enabled through advanced machine learning and algorithmic programming.
“Global revenue for AI products and services is expected to increase 56- fold, from $644 million in 2016 to $36 billion in 2025,”she notes, so we should probably be talking about this quite a lot as a society.
A New Economic Order
According to Zuboff, surveillance capitalism is “a new economic order that claims human experience as free raw material” and “a parasitic economic logic in which the production of goods and services is subordinated to a new global architecture of behavioral modification.”
Zuboff argues that the main problem with surveillance capitalism is that it makes us mere objects whose every life experience and intimate detail must be analyzed and predicted for the benefit—and potential influence—of others. As Zuboff writes, “the essence of the exploitation here is the rendering of our lives as behavioral data for the sake of others’ improved control of us.”
Although Zuboff claims in the book that technology and its discoveries do not have to be malign if data were not inappropriately shared and used, she says that surveillance capitalists basically see the technology as justifying its use in manipulating docile populations.
“Despite all the futuristic sophistication of digital innovation, the message of the surveillance capitalist companies barely differs from the themes once glorified in the motto of the 1933 Chicago World’s Fair: ‘Science Finds—Industry Applies—Man Conforms,’” Zuboff writes.
Provocatively, Zuboff contends that just as industrial capitalism devastated the environment in the nineteenth and twentieth century, surveillance capitalism now directly threatens human nature and relation, twisting our inner self into a weak, squashed bug that can be listlessly herded where companies (or governments) want us to be.
Surveillance capitalism’s aim, according to Zuboff, is to force a new collectivist order on humanity founded on the certainty of AI systems and to steadily take away people’s rights, freedoms, and even conscious thought, by limiting the choice architecture around us and conceptually shepherding people into increasingly tightly controlled avenues of mentation, decision, and action.
Rather than searching on Google or posting on Facebook, people do not realize they are being searched and reacted to—not only for their interests, emotions, and beliefs, but also for their subtle but highly valuable outputs of “data exhaust” such as hesitation in clicking, ways of phrasing questions, time of searches, predictive emotional patterns, ways of reacting to content and much, much more—all information that lets advertisers target people on a highly pinpointed level and lets behaviorists psychometrically isolate an individual’s “type” and potential reactions to situations or products with incredibly high degrees of accuracy.
“Wired magazine’s founding editor, Kevin Kelly, once suggested that although it seems like Google is committed to developing its artificial intelligence capabilities to improve Search, it’s more likely that Google develops Search as a means of continuously training its evolving AI capabilities. This is the essence of the machine intelligence project. As the ultimate tapeworm, the machine’s intelligence depends upon how much data it eats,” Zuboff writes.
Zuboff states at the outset that she is not solely focused on one corporation but rather on the phenomenon of the “reality business” itself. However, the book largely centers on Google, which she considers to have founded surveillance capitalism in the early 2000s, and on Facebook, Amazon, Verizon, and other corporations’ follow-up attempts to become the next Google and capture more data about more people and sell it at an even higher price in a cycle of “muted, sanitized tyranny” that makes Smith’s and Hayek’s capitalism look like baby food. At least, Zuboff argues, Hayek and Smith extolled a system where workers needed to get at least enough spending money to feed a consumer economy for the products they were making.
Zuboff draws a distinction between totalitarianism and its desire to possess and remake humanity into a collective unit of the state ideology, and surveillance capitalism’s “instrumentarianism,” which she describes as a determination to achieve total certainty and predictive power over human affairs from a standpoint of commercial exploitation built on moral and ideological indifference. Whereas totalitarianism derives its power from “Hierarchical Administration of Terror,” instrumentarianism gets its juice from having “Ownership of the Means of Behavioral Modification.”
The surveillance capitalism system is “radically indifferent” to our fate or wellbeing. According to Zuboff, it is amoral and lacks an ideological position apart from its raw appetite for data and control, although this would seem to contradict her repeated statements that the surveillance capitalist system wishes to sell us products and services. (Dead people don’t typically buy products or services, nor do systems generally perpetuate profit incentives if they, for example, cease to be market-based).
The book is formed around the question a mill manager asked Zuboff years ago: “Are we all going to be working for a smart machine, or will we have smart people around the machine?”
Through discovery of behavioral surplus, or the behavior of users in terms of speed of clicking, patterns, emotional trends, and other intricately cross-correlated data, Google gained a huge edge, which Zuboff calls “surveillance assets.” These assets Google gained to analyze behavior, thought, and patterns of action, “are critical raw materials in the pursuit of surveillance revenues” which are then translated “into surveillance capital” as part of a “surveillance economy” that was propelled forward partly by “surveillance exceptionalism” in the post-9/11 world. If you get the feeling that Zuboff is rather fond of using the word “surveillance” in this book, you would not be wrong.
Real Life vs. The Internet
Those who worry about people glued to their smartphones or sitting at home all day are missing the big picture, according to Zuboff. The real future imagined by our tech overlords is one in which, essentially, real life is the internet.
“The aim here is a grand synthesis: the collation and fusion of every sort of sensor data from every channel and device to develop a ‘virtual sensor environment’ in which ‘crawlers will constantly traverse data. . . calculating state and estimating other parameters derived from the data’ collected from everywhere from office interiors to entire cities.”
Big Tech is much further down the road in terms of sophistication of data collection than many people may realize—not only in the odd ways it snoops on people’s private correspondence and daily decision-making but in how it utilizes that information for vastly profitable predictive advertising and in its all-encompassing “connected” vision of the future. What some sci-fi movies have shown, but many may still not grasp, is that the internet or home-helping robots aren’t the future we’re looking at.
As the former executive chairman of Google Eric Schmidt put it at Davos in 2015, “The internet will disappear. There will be so many IP addresses . . . so many devices, sensors, things that you are wearing, things that you are interacting with, that you won’t even sense it. It will be part of your presence all the time. Imagine you walk into a room and the room is dynamic.”
Although headlines misunderstood Schmidt’s statement, what he was saying was deeply profound, and built on the ideas of computer scientist Mark Weiser, who wrote in an influential 1991 paper that “the most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” He described a new way of thinking “that allows the computers themselves to vanish into the background. . . . Machines that fit the human environment instead of forcing humans to enter theirs will make using a computer as refreshing as taking a walk in the woods.”
In other words, as we can already see with “smart cities,” the powers that be envision a future of complete integration and interactivity, with information flowing at all times and everyone linked up to the information grid.
“Surveillance capitalists understood that their future wealth would depend upon new supply routes that extend to real life on the roads, among the trees, throughout the cities. Extension wants your bloodstream and your bed, your breakfast conversation, your commute, your run, your refrigerator, your parking space, your living room,” Zuboff writes.
Zuboff further compares surveillance capitalism to colonial abuse of power, writing that “these twenty-first-century invaders do not ask permission; they forge ahead, papering the scorched earth with faux-legitimation practices:
Instead of cynically conveyed monarchical edicts, they offer cynically conveyed terms-of-service agreements whose stipulations are just as obscured and incomprehensible. They build their fortifications, fiercely defending their claimed territories, while gathering strength for the next incursion. Eventually, they build their towns in intricate ecosystems of commerce, politics, and culture that declare the legitimacy and inevitability of all that they have accomplished.
And like the indigenous people on a beach in South America when the Spanish first arrived, we are not so much being overpowered by force as by the extent of what is simply unprecedented:
Ours is not simply a case of being ambushed and outgunned. We were caught off guard because there was no way that we could have imagined these acts of invasion and dispossession, any more than the first unsuspecting Taíno cacique could have foreseen the rivers of blood that would flow from his inaugural gesture of hospitality toward the hairy, grunting, sweating men, the adelantados who appeared out of thin air waving the banner of the Spanish monarchs and their pope as they trudged across the beach.
A Fine Line Between Commercial and Political Incentives
So let’s summarize: Should you read this book? Yes. Does this book have shortcomings? Yes.
One shortcoming of this book is its primary focus on the intersection between technology and commerce. Although Zuboff touches on China in a sub-chapter called “The China Syndrome,” the book largely eschews the subject.
China is in the process of implementing an incredibly sophisticated surveillance and reward-punishment control grid on its citizens and is definitely worthy of discussing in depth, particularly as its enforcement efforts stem from the Chinese state, rather than private commercial motives. As Zuboff notes, “the system tracks ‘good’ and ‘bad’ behavior across a variety of financial and social activities, automatically assigning punishments and rewards to decisively shape behavior toward ‘building sincerity’ in economic, social, and political life.”
China’s system is all the more relevant given Google’s decision to return to China with its censored Dragonfly service. Google also announced they will be opening an AI research center in China where the non-existent privacy laws give them free reign to experiment.
The revelations of Google’s Dragonfly move were leaked by an employee who said, “I’m against large companies and governments collaborating in the oppression of their people.” Zuboff writes that “In the Chinese context, the state will run the show and own it, not as a market project but as a political one, a machine solution that shapes a new society of automated behavior for guaranteed political and social outcomes: certainty without terror.”
It is hard to understand what Zuboff is referring to by “without terror,” considering that China is currently running concentration camps and re-education centers filled with communist brainwashing and torture for an estimated 1 million ethnic Uyghur Chinese Muslims on the basis of their ethnicity and faith. Granted, Surveillance Capitalism is focused on the United States and the start of private corporate information dominance, but Zuboff makes claims about the world as a whole and a move to an omniscient corporate control grid that would seem to necessitate bringing up the case of China, where it is not so much corporate power as government ideology and repression that rules. In fact, corporations get their taxes waived if they agree to be state-run.
It would be interesting to see Zuboff write a follow-up book more fully addressing the potential for authoritarian surveillance states to use advanced technology to control and oppress their population—including, potentially a future U.S. government. Let’s be honest: the Soviet Union would have paid everything they had for the chance to get hold of the kind of technology Zuboff is describing here, and Belarus, North Korea, and numerous other countries are already currently doing their best job to make Stalin and Co. look like surveillance amateurs.
Zuboff’s semi-dismissal of China’s fusion of surveillance with state power because it’s “not a democracy” is, frankly, short-sighted and a bit elitist, although she does acknowledge that “perhaps the most shocking element of the story is not the Chinese government’s agenda, but how similar it is to the path technology is taking elsewhere.”
Moreover, when Zuboff acknowledges that 9/11 greatly acceded the rise of surveillance capitalism and that “state and market institutions demonstrate a shared commitment to a relentless drive toward guaranteed outcomes,” why is the book as a whole focusing only on commercial incentives when governmental incentives of control, military dominance, and surveillance form a potentially greater rationale and motivation for the expansion of technological wizardry?
More Suggestions, Less Rhetoric
Standing up to surveillance capitalism is difficult, and Zuboff’s book does a good job of demonstrating the all-encompassing reach of our new not-so-beneficent overlords. The book is very light on any suggestions, however, and if you’re looking for guidelines to stopping cookie tracking or limiting how your smartphone knows about you, look elsewhere.
Despite admitting that typical restraints and regulation do little and will be bypassed by Big Tech, and presenting a nightmarish vision of a dystopian future, Zuboff offers only vague hints that real “democracy” or a collective demand to be granted “a future of our own making” can put up a fight against the encroaching everything-web. In terms of other specifics, she also reserves some accolades for the EU’s General Data Protection Regulation that came into effect in May of last year.
For all the praise this book has been receiving in mainstream media, it needs to be said that the author repeats herself often and the book does not fulfill its titular purpose of telling us much of anything about the fight for a human future against surveillance capitalism. Instead it talks ad nauseum about the problem itself, ironically increasing precisely the kind of “inevitablism” about the creeping advance of technology that Zuboff slams throughout the book.
Although it is understood that Zuboff wishes to emphasize certain issues, there are many cases where long-winded explanations and reflections simply repeat precisely what was already said—at length—five times before in earlier chapters. While this may not impede researchers, academics, or intellectual types, it is a potential roadblock to lay readers who want to know what’s the deal with runaway technology. Instead, they will keep getting enmeshed in Zuboff’s lengthy, repetitive treatises parsing philosophical distinctions and economic theories.
Another thing this book could do without is some of the overdone language and rhetoric.
“It is the one idea to have emerged from the long story of human oppression that insists upon a people’s inalienable right to rule themselves. Democracy may be under siege, but we cannot allow its many injuries to deflect us from allegiance to its promise,” Zuboff writes, apparently channeling a Corey Booker-style barnburner. If one were to invent a drinking game for every time Zuboff uses words like “respect,” “democracy,” and “values” you’d be drunker than Boris Yeltsin at the dacha on the weekend.
And, frankly, you’d have to be drunk to not be occasionally annoyed at Zuboff’s heavily overwrought prose and vague, Boomer-esque exhortations for “young people” to “stand up” to Big Tech in some way, such as donning surveillance-blocking bandanas and supporting a university project that inhibits facial recognition technology with face masks and stylized dreadlocks (no joke).
It’s perfectly clear that what Zuboff is saying is very serious. So why restate it 100 times and use the flowery language of a discreditable strain of neoliberalism that makes most people cringe inside? Why talk about the Berlin Wall coming down in an emotional passage of attempted historical parallelism instead of suggesting actual ways to counteract surveillance capitalism?
Another drawback is that by choosing to quote respectable and mainstream philosophers such as Hannah Arendt and Emile Durkheim, Zuboff also shies away from the controversial and uncomfortable discussions to be had about violent responses to technological dominance. Such violence, advocated by figures like domestic terrorist Ted Kaczysnki, not to mention the growing impetus on the fringes of the left and right toward eco-militancy and aggressively nihilistic responses to technological advancement, are not unimportant.
To be sure, Zuboff is not to be expected to provide a response to the fanciful idea that roving bands of green rebels will overthrow the massively powerful and interconnected technological structure of surveillance capitalism (“For the Green New Deal and AOC!” the renegades shout, hurling Molotov cocktails), but it does seem her book would be improved by addressing the despair and violence that some turn to when faced with the enormity of technological control, climate change, and post-industrial ennui.
Another uncomfortable and politically charged discussion Zuboff shies away from is the extent to which Big Tech is explicitly liberal and acts punitively toward social conservatives and religious people, especially Christians. This clashes with her thesis that Big Tech fundamentally doesn’t care what you think as long as it knows all and can sell your info to the highest bidder. Presumably, this is because Zuboff appears to have liberal sympathies — expressing support for abortion, for example, at one point in the book — and she otherwise champions a typically liberal defense of individualism and absolute freedom.
Discussing the anomie of modern life in our “Doomer” Age is at least part of tackling the puzzle of why we have allowed tech wizards to run our lives, provided Zuboff’s argument that control and manipulation are not inherent to technological advancement is correct. To be fair, she acknowledges that, “The consequences of this new logic of accumulation have already leaked and continue to leak beyond commercial practices into the fabric of our social relations, transforming our relationships to ourselves and to one another.”
The content is insightful and interesting, but is brought down by the overwrought, wordy, self-righteous presentation and the frequently italicized terms like this denoting a major new wordy concept that could actually be said much more simply and succinctly.
So let’s say it simply: Big Tech is out of control and it has too much control of us and our lives. We need concerted legislation and withdrawal of consent to start making a change and turn the tide, and we will leave it to another author—or perhaps Zuboff in the future—to offer us some of those specific policies and strategies for how to regain autonomy from the machines.
The Federalist · by Paul Rowan Brian