Shoshana Zuboff | Surveillance Capitalism in the Age of the Unprecedented

Shoshana Zuboff | Surveillance Capitalism in the Age of the Unprecedented


Today’s episode of Hidden Forces is made possible
by listeners like you. For more information about this week’s episode or for easy access
to related programming, visit our website at hiddenforces.io and subscribe to our free
email list. If you want access to overtime segments, episode transcripts and show rundowns
full of links and detailed information related to each and every episode, check out our premium
subscription available through the Hidden Forces website or through our Patreon page. Remember, if you listen to the show on your
Apple podcast app, you can give us a review. Each review helps more people find the show
and join our amazing community. With that, please enjoy this week’s episode. In 1609, while in search of a rumored, Northeast
passage to Asia, the English explorer and navigator, Henry Hudson landed on what is,
modern day New York City. Though written accounts exist of Hudson’s
encounters with local tribes. Native accounts have been handed down to us through oral tradition,
often transcribed much later by missionaries and settlers. In 1765, a Moravian missionary who lived for
many years among the Delaware and Mohican tribes recorded a native account of the first
meeting: “Some Indians who had been out fishing where
the sea widens spied, at a great distance, something remarkably large swimming or floating
on the water. It was agreed among those who were spectators that as this phenomenon moved
towards the land, it would be well to inform all the Indians on the inhabited islands.
Chiefs from scattered tribes who arrived in numbers concluded the strange appearance to
be a large canoe or house in which the great Supreme Being himself was coming to visit
them.” Such descriptions by native peoples of their
first encounters with White settlers were common throughout the Americas. The first
men to bring news to the great Aztec King Moctezuma, of Spanish ships described what
he saw from the gulf coast as “towers or small mountains floating on the waves of the sea.”
In other accounts, natives looked from the shore and thought the awesome, approaching
ships were giant white seabirds or floating islands. There are even theories that the Arawak, the
first tribe to encounter Columbus’ ships off the coast of Hispaniola could for a time see
only their ripples across the horizon, unable to picture what was for them unprecedented,
unimagined, alien. What all these people had in common was that
they were unable to name, let alone recognize a force of creative destruction so vast and
boundless that it would make their worlds unrecognizable, their homes uninhabitable,
their lives unlivable. They were unable to see the ships and their crew for what they
were. The vessels of conquerors, pillagers and looters of unsullied lands, unable to
name them, they welcomed them, agents of their own annihilation. This week on Hidden Forces Shoshana Zuboff,
Digital Natives, the unprecedented, and finding home in The Age of Surveillance Capitalism. So, Dr. Zuboff, I was just telling you before
we turned on the microphones how much I valued reading your book. What was your objective
in writing it, and why the name Surveillance Capitalism? I think the larger project here, and the reason
why I called this new market form, surveillance capitalism, is that it’s mechanisms are designed
to be hidden, that it’s a backstage operation. Its mechanisms are designed to keep us in
ignorance. The result of that is that even though it’s something unprecedented, it’s
very hard for us to detect it in our midst. So, my aim has been to unveil piece-by-piece
the actual mechanisms, what it is, how it works, why it strives to remain hidden in
secret, why our ignorance is its bliss, if you will. You also talk about this concept of radical
indifference in the context of what you call “Instrumentarianism,” which is to surveillance
capitalism what totalitarianism was to industrial capitalism. How does that fit in? Well, a lot of people have, you know, in the
sense of unease that we have with the direction that the digital has taken under surveillance
capitalism, people know that we’re being monitored and we’re being tracked. There’s a lot of
reference to Big Brother, and a lot of reference to concepts like digital totalitarianism. I thought it was important to really ask the
question, “What is the kind of power that surveillance capitalism produces?” Because
really, it’s not a totalitarian power. Totalitarianism is a form of power based on, first of all,
terror, murder. Totalitarianism aims to control individuals from the inside out. It wanted
you to believe what it wanted you to believe, and it wanted to control your thoughts, and
your feelings, and your emotions, and it wanted you to conform to everything that it held
as ideologically correct and necessary. It was willing to murder you on the way to that
conformity, and purging the human species for the sake of this kind of perfection inside
and out. The kind of power that’s created by surveillance
capitalism is really very, very different. What I found interesting is that when totalitarianism
bursed upon the world in the 1930s and 1940s, it was also an unprecedented form of power.
People couldn’t recognize it in their midst because it was so unprecedented, so bizarre.
It demands and claims we’re so unimaginable and intolerable that really, people couldn’t
get their arms around it or believe that it was even real until the evidence of the murder
and the terror and the Gulags and the concentration camps, and so forth. Well, the kind of power that’s created by
surveillance capitalism is equally unprecedented, but not with the same kind of violence. It’s
a much more subtle and abstract kind of violence, but because of its unprecedented nature, it
has also been very challenging to perceive, to name, to grasp, to understand, to predict.
So, this has been a big part of my work, to try and fill in those blanks. The nature of instrumentarian power, since
you bring up radical indifference, is that it’s a power that is used to really shape,
tune, herd the behavior of individuals and populations toward the kinds of commercial
outcomes that surveillance capitalists and their business customers seek. It uses the instrumentation of the ubiquitous
digital architecture that now surrounds us in our daily lives. It uses that instrumentation
as the means to a global capability for behavioral modification that can push us, and herd us,
and shunt us in the direction that it wants us to go for the sake of its commercial outcomes. The thing is here that it doesn’t care what
we do. It doesn’t care what we believe. It doesn’t care what we feel. It doesn’t care
if we are in great pain. It doesn’t care if we are experiencing great joy. All it cares
about is that whatever we are doing, whatever we are thinking, whatever forms of action
we are taking, that we are doing those things in ways that it can capture the behavioral
data from those activities, translate those behavioral data into predictions, sell those
predictions into new markets that trade exclusively in behavioral futures, in predictions of what
we will do now, soon, and later, sell those to its business customers who have a vested
interest in knowing our future behavior. So, this is a completely different direction
where our behavior becomes simply instrumentalized on the way to achieving commercial outcomes
for business customers interested in the future of our behavior. This is the essence of instrumentarian
power. I call this “radical indifference” because instrumentarian powers’ attitude toward
us, its point of view, if you will, is a point of view where it really doesn’t care about
us. It just cares that it has access to the data that leaches from our activities, that
it can use for its predictions. The result of that is what I call radical
indifference. There was a memo that was produced by a Facebook executive named Andy Bosworth,
that was released accidentally at some point just a little over a year or two ago, where
he was actually coaching his team on what I would call the operations of radical indifference. The way he put it was the economic imperative
for Facebook is growth. Growth comes from connection. I’m paraphrasing, of course, because
I’m not reading this memo right now. Essentially, he said, “Look, if people connect on Facebook,
and they use that connection to create a terrorist plot, and people get killed, we continue to
connect because connection is growth, and that’s our life. If people use connection
on Facebook to fall in love and get married and live happily ever after, we continue to
connect because connection is growth, and growth is our imperative.” So, here, he’s illustrating the point of view
of radical indifference. It may lead to terror and death. It may lead to love and happiness.
We can’t care about that. We don’t care about the content of behavior, the content of connection,
the content of what happens on our platform. All we care about is the fact of connection
that we get the data from all of that connection because that’s the data that drives economic
growth for Facebook as a premier surveillance capitalist. So, Bosworth was acting as a very responsible,
accountable, excellent surveillance capitalist, and beautifully illustrating the point of
view of surveillance capitalism, which is radical indifference. This was something I was wondering about when
I was reading your book. The aim as you state here is to obtain this data and to compute
it in order to make more accurate predictions. The more accurate the predictions, the more
profitable the enterprise. Aren’t the most accurate predictions, if you were to follow
this logic to its final destination, the most accurate predictions are those which you have
a hand in formulating? Yes. In other words, if you are to herd this hive
– you’ve described as a hive – if you were directing the behavior of the automata that
are in the system, then you can perfectly predict in the final iteration their behavior.
Isn’t that the final state of the system, a perfectly controlled, automated system? All right. So, I think it might help a little
bit if I gave a quick definition of surveillance capitalism, and just very quickly the architecture
of its economic imperatives, and then come back to this question because it’s the apotheosis
of the economic imperatives. Is that all right? Yeah, and I’ve been told you don’t forget,
which is actually great because I would forget. Yes. [Laughter] I don’t forget. So, I would define
surveillance capitalism this way for our listeners. There are many respects in which surveillance
capitalism diverges from the history of market capitalism, but in this one respect that’s
essential to its definition, it follows in the pattern of how capitalism has evolved
over the centuries. Historians have described this process as capitalism continuously claims
things that live outside of the market dynamic and it drags them into the market dynamic,
converts into commodities that can be sold and purchased. So, famously, industrial capitalism claimed
nature. Nature lives in its own time and space. The meadows, the waters, the oceans, the forests
live in its own time and space, but industrial capitalism claimed nature for the industrial
dynamic to have it reborn as real estate, as land, as water that could be sold and purchased
in a commodity fashion. At the time that that happened, that was a radical breakthrough
in the whole structure of life. Also, industrial capitalism took the idea
of work, which was something that people did in their homes, in their gardens, in their
fields, in their cottages, and it brought that into the market dynamic to be reborn
as labor, that could be sold and purchased in the marketplace. Surveillance capitalism does something similar,
but now with a dark twist. What it does is it claims private human experience as a source
of free raw material, raw material, which is translated into behavioral data. Those
data then combined with advanced computational capabilities, the dark satanic mills, if you
will, of surveillance capitalism, what we call machine learning, machine intelligence,
artificial intelligence, combined with these computational capabilities in order to produce
predictions of our future behavior. This began in the context of online targeted
advertising. The specific behavior that they wanted to predict was whether or not we would
click through particular ad. That’s a piece of human behavior, clicking, maybe trivial
in the larger scheme of things online, clicking, but their aim was to predict that bit of behavior.
Those online targeted ad markets, if you just zoom out a little bit, those were markets
where people were trading in these behavioral futures. “Who’s going to click on my ad? If
I put my ad where they tell me to, will I get the clickthrough rates?” So, they’re betting
on future human behavior. This became the essence, the mechanisms of
the economic logic that is surveillance capitalism. Begun at Google, migrated to Facebook, became
the default option in Silicon Valley, and is now migrating across many sectors of our
economy, many different industries. Well, you have a starting point that you identify
in the book, which is I think 2001 or thereabouts, what you described as a crisis in Silicon
Valley after the bursting of the bubble, and Google needed to find a business model that
could make their company profitable, and this was born out of Google, and then has spread
and become a logic, an economic logic as described it for the rest of the economy. That’s right. So, it now has a life of its
own, and it should not be confused with either one company like Google or two companies like
Google and Facebook. It’s in the insurance industry. It’s in the retail industry. It’s
in the healthcare industry. It’s now on the automotive industry. It’s a kind of virus
that is spreading through these different sectors and transforming each sector as it
moves through. Early on, the competitive dynamic in this
new logic is around the quality of prediction. So, what’s going to come out of that black
box of this computational machinery that tells us what people are going to do now, soon,
and later? Early on, the competition for predictions begin to delineate a new kind of economic
imperative. There are several of these economic imperatives. The first, of course, was scale, that you
needed a lot of behavioral data in order to make powerful predictions. So, you need a
volume of these behavior data flows. Again, as competition heated up, pretty soon, volume
was not enough. Now, it also became variation. So, you need both scale and scope. You need
a lot of behavioral data, but you need different kinds of behavior data. So, now, we’re going to get you offline. We’re
going to give you a mobile phone. Android. Give you an Android at the least possible
cost, so that there is no friction presented to the continuous flow of behavioral data.
So, that Android has to be understood as simply a supply chain interface for behavior data.
It was debated within Google. Some people said, “See, now we have a way of competing
with Apple and with the iPhone, and we can make money on our devices, and have margins
on our devices. We don’t have to make a lot of money on data.” Other people said, “No,
no, no, no, no, because we have discovered this powerful logic that is just driving unbelievable
volumes of revenue growth. So, what we want to do is lower every single barrier to the
flow of behavior data.” So, the voices that won in this debate were
the voices that said, “Hey, if we could give Android away for free, let’s do that,” because
all it is is a supply chain interface for behavior data that goes with you wherever
you are in the world, your voice, your camera, your contacts, all the stuff on your phone,
but also, your location, where you’re going, who you’re going with, what you’re doing,
what restaurant you’re entering, and so on, and so forth. Ultimately, your face, your voice, all these
other ways of getting depth insight into you from the more emotional, personality-related
characteristics of yourself. So, now, we have economies of scale, and economies of scope,
but prediction competition continues, more people coming on stream, more people searching
for these behavioral data, more people trying to get a dominant position in these new prediction
markets. So, the next horizon, as it were, becomes
something that really is new under the sun, economies of scale, economies of scope. This
next one I call economies of action. The idea here, as you so rightly anticipated, is that
the very best predictive data comes from actually intervening in the state of play, and finding
ways to use, as we said before, the digital surround, all the instrumentation in the digital
milieu, what people talk about ubiquitous computing to use all of these instrumentation,
to usually, gently nudge, coax, tune, herd subliminally shape and modify your behavior
in the direction that we want you to go to fulfill our commercial goals, and the commercial
goals of our customers in these behavioral futures markets. So, the very most powerful means of predicting
is to actually be shaping your behavior, and this is when the digital architecture that
now pervades our lives, our daily lives has been repurposed as a global means of behavioral
modification. This has resurrected an old paradigm of behavioral modification that really
our societies rejected in the 1960s, in the 1970s when the great leaders of behavioral
modification like the scientists, my former professor at Harvard, B. F. Skinner, considered
the father of radical behaviorism. Which is the incarnation of that, the economic
incarnation. This is the economic incarnation of that now
on the scale of whole populations. There’s a great of B. F. Skinner’s. I don’t
know if I got it from your book, but it resonated with me, “It is not a matter of bringing the
world into the laboratory, but of extending the practices of an experimental science to
the world at large.” Yes, I do cite that in the book. That is exactly
the reversal that we’ve been through here, where in the ’60s and ’70s, behavioral modification
was being applied to captive audiences in prisons, and hospitals, and schools. There
was a wide congressional senate inquiry. The American people were up in arms rejecting
these practices. An auspicious group of senators that formed
this committee included Edward Kennedy and Sam Ervin, and the very famous senators came
down on this, and finally concluded that behavioral modification was an affront to the ideal of
the autonomous individual, which was essential for democracy and an affront to democratic
society itself. They forbade any kind of federal funding to go to any kind of program that
used behavioral modification on its populations. Now, we fast forward a few decades later,
and here we have under the auspices of private capital, private surveillance capital the
entire digital infrastructure for which as we began our conversation about home, for
which most of us harbored so much hope for an empowered and democratic future. This entire
digital architecture now is being hijacked by a new form of capitalism, a mutation of
capitalism, a rogue capitalism that uses the digital now as a global means of behavioral
modification for the narrow commercial ends of the others not us, business customers,
not the people whose behavior and experience is at stake. Our problems are not being solved.
Our problems are simply being instrumentalized for the sake of other’s commercial gain. A lot of things have come to my mind as I’m
listening to you talk. One of them has to do with Skinner’s black box, and that these
algorithms are black box. That relates to something you’ve written in the book, which
is that these companies demand to know everything about us, and we can know nothing about them.
It’s interesting play on words. Also, there’s this imagery that keeps coming
to my mind. It isn’t exactly the spider’s web. Again, there’s this surreal quality to
what comes across from your writing or from the implications of your writing, maybe. There’s
a lot of imagination that’s left to the reader. So, maybe it is my own imagination, but there’s
this dark, nightmarish quality of a spider’s web, where almost like a deer drinking from
a river, and it gets tangled up in it. It feels like we’re getting tangled up in this
web. That comes up a lot when you talk. That’s
something that I think often about. Do you feel that way at all? Does it feel like we’re
getting entangled and meshed in this architecture that is being driven by this economic logic
of surveillance capitalism? It’s very interesting the way you bring this
up because as I’ve been going around talking to groups of people on this subject, I’m finding
the same kind of pattern over and over again, where people talk about a sense of anxiety,
and unease, a lot of control, a feeling of manipulation, concerns about freedom, concerns
about sovereignty, a sense that they need to resist, but not really having a clear grasp
of what to resist or how or whom. So, it’s this foggy, amorphous, cloudy thing
that is closing in around us that’s making us feel anxious and uneasy like things are
not okay. More and more, we feel like we have to hide from it, but how to hide? Where to
hide? The response that I’m getting with the word surveillance capitalism is that here,
finally, we have a way of naming this thing that we’re entangled in. Just in the idea of naming, being able to
see it clearly, being able to discern its mechanisms, being able to perceive the logic,
see how it operates, see how it functions, even though it doesn’t change it immediately,
it complete changes the experience of it because once we can name it and once we have language
about it, we start to be empowered, and we start to be able to see where it ends and
where we begin. We start to be able to imagine a space in
which there can be resistance, and where there can be combat, and we can come together and
talk and name and develop our bonds of collective action that can reinstate some form of combat
that we have been robbed off precisely because these mechanisms. So much capital has been
devoted to ensuring that these mechanisms are invisible. So much skill has been devoted
to making sure that the rhetoric in this space of surveillance capitalism is misleading. … and cloaked in inevitability. … and rendered as though it was an inevitable
consequence of digital technology. So, it’s laden with euphemism, with the rhetoric of
emancipation, with very artfully employed misdirection, telling us it’s one thing when
it’s really another thing, masses of engineering skill and capital devoted to making sure that
these mechanism operate in a way that consistently comprehensively bypasses individual awareness,
so that our ignorance is engineered by these vast accumulations of capital and skill that
are now trained on our experience. So, out of all of that, we have been robbed
of the right to combat. We have been robbed of our decision rights. We have been robbed
of a choice to participate or not in indeed every single piece of research. Going back
to the early 2000s shows that when people are informed of the nature of these backstage
operations, they want nothing to do with them. They do not want to be entangled. They want
to be protected, but more and more, a surveillance capital has come to own and operate almost
all of the spaces of the internet. There is no place to go. There is no place
to escape. The alternatives have been foreclosed. Therefore, our means of social participation,
just the basic kinds of things we need to do everyday, whether it’s getting our health
data from our physician or getting our grades from our children’s teachers or making arrangements
with our friends and family for dinner. There’s the basics of the social participation force
us to march through the same channels that are surveillance capitalism supply chains. So, social participation and surveillance
capitalism are increasingly conflated, and their alternatives are foreclosed. Therefore,
we feel resigned and we feel helpless, and racing around all of this as you’ve already
hinted at is a larger category error that has been imposed on us. That’s the idea that
this is the only way that digital can be. I think it’s a powerful idea, though. I think
that’s something that the more I meditated on that, it grew an importance. This point
that we’re made to believe that what we encounter today is technology. It is technology. It isn’t surveillance capitalism. It’s not economics. It’s technology. That’s the wolf in the sheep’s clothing. That’s
the puppeteer behind the puppet. The puppet is the technology. The puppeteer are the surveillance
capitalists. That’s right. The puppet is this whole milieu
of digital instrumentation. That doesn’t have to be governed by this type
of logic. That can be anything we want it to be. It
was. Demetri, it was. I agree. I mean, I remember what the 1990s
was like. That’s right. It was nothing like this. There was a window before surveillance capitalism
broke upon the public imagination when Google was finally IPO in 2004, and we got a chance
to see the incredible revenues that flowed from this economic logic in just a few short
years. Between 2000 and 2004, Google’s revenue line increased by 3,500 and 90%. That’s a remarkable number. That’s just astonish, and that was on the
back of this economic logic. Look, in the year 2000, a group of really smart designers
and engineers and data scientists at Georgia Tech published a really interesting big report
on what they called the Aware Home. That’s what today we think of as the Smart Home.
The Aware Home was going to do all the things that we want a Smart Home to do today. It
was going to give us data to make our homes more efficient, and data about our behavior
that could help us with our health, and our fitness, and so on and so forth. Anyway, when they got down to the schematics
for the Aware Home, the schematics were pretty simple. It was a closed loop. It was the devices
built in to the home itself. It was one node. The other node was the occupant of the home.
The occupant or the occupants, those are the only people who receive the data. They decided
if it would be shared or not, and they decided how to make sense of it. They decided what
to do with it. It was a simple, closed loop that was empowering. Similarly, in the telemedicine field, telemedicine
in the year 2000, 2001, 2002, same goals that we have today, but conceived as a simple,
closed loop, the physician, a server typically located in a hospital, and the patient at
home in her or his bed or bedroom or living room or wherever, a simple, closed loop where
it was the person who receive the data, the person who had the decision rights over those
data. Just a couple of year ago, 2017, a couple
of scholars at the University of London did an analysis of the Nest thermostat. As you
know, Nest is owned by Google. They concluded that for a moderately vigilant consumer to
install one Nest thermostat in the home requires that person to review a minimum of 1,000 privacy
policies and user contracts, and user agreements because the thermostat, of course, is collecting
all kinds of behavioral data. It also becomes a hub for other smart devices in the home
that are feeding it with behavioral data. It could be your mattress. It could be your
dishwasher. It could be your television set. It could be your stereo system. Then all these data are not only going to
Nest, but they’re going to Nest third-parties. Nest takes no accountability for what the
third-parties do. Each third-party shunts the data to third-parties, and they take no
accountability, and those third-parties and their third-parties and so on and so forth
in an infinite regress. Then, of course, it turns out that if you
don’t agree to these thousands of contracts, you slowly lose the functionality of the product.
Nest warns maybe the smoke alarm won’t work. Maybe you end up with frozen pipes. So, the
very reason that you bought the thermostat in the first place is obviated, is deleted
because the real point of the thermostat is as a supply chain interface for behavioral
data, not to serve your needs in your home for you, for your family, for your energy
cost, for your health, everything else that you had hoped for. So, this is where we now have smart products.
It means that we’re no longer really working on the product. We’re simply using the product
as a supply chain interface, personalized services. We’re no longer working on the service
to really help you and solve problems in your life. The service is a supply chain interface
for behavioral data, and that’s the direction that surveillance capitalism is taking our
economy. A phrase that has become popular recently
is that you’re not the customer, you’re the product, but you make the point that you’re
not really the product, you’re actually the resource to be extracted. What’s interesting
about this is that now, we’re getting to a point where not only are we the resource to
be extracted as you say, we’re the data that leaches from our sorrow, but we are now to
be increasingly guided. It’s a two-way mechanism. It’s monitoring and actuation. That’s what
the data scientists call it. The first phase is monitoring. We can know everything through
the digital surround. We can know everything, but then the data scientists describe a later
phase called actuation, where we can actually use the same devices that let us know everything.
We use that knowledge to now actuate real action, real behavior in the real world of
people and things. It’s an interesting, interesting is not the
right word. I’m trying not to be alarmist because your book is not alarmist. Like I
said, it is an intimate read, and it actually has a very … I don’t even want to use the
word positive message because that also sounds fake. It is positive. It feels genuinely positive,
but many of us have been in situation where there’ve been people in our lives who are
somewhat manipulative. They can manipulate us a little bit. There’s an experience of being around someone
like that or in a relationship with someone like that, where things don’t feel right.
Something feels off. There is this unease. There is this anxiety. I find this to be very
relatable to where we are today. I feel that this is a shift where we went from this period
where the entire attention was focused towards harvesting data. So, now, we’re increasingly
that data is being put to use in actuation. You’ve, of course, used one great example,
well, a few. Actually, two I really liked. Three, maybe two with Facebook, one with the
voting, and the other more pernicious with the happiness and the sadness. The emotional contagion. The emotional contagion, and then the other,
more disturbing in the context of instrumentarianism, the case of Pokémon Go. So, maybe you can
also tell our listeners who aren’t familiar with these case studies what they are because,
again, they’re significant for where we’re going in this discussion. Yeah. You’re absolutely right. Well, Facebook
conducted two what they call massive scale contagion experiments. One was to see if they
could affect real world voting behavior. In this case, it wasn’t telling you to vote for,
but just getting you to go vote. So, it was like a pre-Cambridge Analytica run through
on this kind of thing. Their second massive scale contagion experiment was around seeing
if they could actually change the emotional valence of a person, make you happier or make
you sadder. The whole idea here was could we use subliminal
cues in the online environment to affect changes in your real life in the real world in one
case, getting more people to go vote than would have voted without the subliminal cues,
and the other case, making some people happier and making other people sadder. When they went to write up these experiments
in the scholarly journals, reputable, very reputable scholarly journals that publish
these studies, the researchers both from Facebook and from Academia boasted that not only one
had they discovered that you could unloose an emotional contagion through the online
media that would change real life in the real world, but two, that you could do that by
continuously and comprehensively bypassing individual awareness, that people never could
see it coming. They never knew it was happening. That’s the thing that’s so scary, and that’s
what I feel like we feel in a subconscious way when we discuss this unease and this anxiety. Yes, that’s right. That’s absolutely right. This feeling of being manipulated without
our knowing. That’s right. It’s right that we feel unease
because it is happening. I mean, people who are on Facebook, the news feed is like a version
of this all the time because the news feed is a complex algorithm. That news feed is
constantly being manipulated in order to engender the maximum engagement from you on the site. So, this is all an exercise in this kind of
manipulation, but some of it like these massive scale experiments begin to show us a sense
of its ambitions not only toward just collecting our data online, but toward society itself,
toward how surveillance capitalism moves toward the real world, toward organizing real populations
as we move through our real lives in our cities, in our countries, in our societies to tune
and herd and coax us in the real world toward its guaranteed outcomes. That’s where Pokémon Go comes in. Pokémon
Go as you know was an augmented reality game developed and incubated at Google, spun off
at the last minute for its introduction to the world, but Pokémon Go really was a population
level experiment in how we use now this digital architecture to herd populations through the
real places and our real cities toward the guaranteed outcomes that our business customers
want. In the case of Pokémon Go, which was nominally
owned by a company called Niantic Labs, which also came out of Google run by John Hanke
who was behind Street View. Before that, he was behind Google Earth. Before that, he invented
the satellite system that was invested in by the CIA that then was bought by Google
that became Google Earth. That’s an important piece also just to throw
it in there is the surveillance state component, and how important 9/11 was and the terrorist
attacks were to enabling this not just in the regulatory front, but also for the culture
to get to this place where people say, “I don’t have nothing to hide. Why do I care?” Right. That’s a key piece of this. The interest
of the surveillance state were critical to incubating and nurturing, and these fledgling
companies with their really cool, amazing, new surveillance capabilities that bypassed
awareness and so forth, and making sure that there was very little, if any, law that impeded
their growth and their development, so that these capabilities would be there when the
state needed dip in because these capabilities were allowed to grow and flourish under the
auspices of private capital free from constitutional constraints, free from other kinds of democratic
law that, ultimately, even our intelligence agencies are accountable to. So, in the case of Pokémon Go, what we saw
was that, at first, it seemed like, “How is Pokémon Go making money?” Well, it’s making
money because people buy doodads for their Pokémon creatures, and they buy banners,
and tokens to get them up to the next level of the game. Digital banners. These various accessories that enhance the
game. Ultimately, what became clear was that Pokémon Go, Niantic Labs, had convened its
own behavioral futures markets. So, it had establishments in the city, the pizza joints,
the restaurants, the bars, the McDonald’s franchises, the other kinds of services, places
where they may fix your car, where you might want to go shopping. All these place were
paying Niantic Labs for guaranteed footfall. So, now, again, you see the exact same structure
as what we talked about before in the online targeted advertising. Online targeted advertising
were paying for a piece of future behavior, which is a clickthrough. Now, we’re in the
real world, on the real streets, in the city, in the parks, in and out of homes, in and
out of cars. We’re paying for a piece of future behavior. This time, it’s not clickthrough,
it’s footfall. It’s you’re real body in a real place, spending real money in my restaurant,
in my shop, whatever it might be. So, now, we have come full circle now to how
we learn to herd populations through the real places of our real lives toward others’ guaranteed
commercial outcomes, all of it outside of our awareness while we think that we’re just
playing Pokémon Go. This, of course, is a rehearsal. It’s a dry run for the model of
the smart city, which Google calls the Google City. It’s what citizens in Toronto are contesting
this very day as the public officials of Toronto have seeded to Google a part of the waterfront
to develop and Eric Schmidt when he heard about this great coo that they had landed
said, “Oh! Now, it’s our turn.” Our turn means that if it’s our turn, it’s
private capitalists turn, it’s surveillance capitalists turn to run the city instead of
what? Instead of democracy’s turn, instead of the citizens’ turn. It means that democracy’s
turn is up and now, it’s surveillance capitalism’s turn to run the city, to replace politics
with computation, to replace citizens with statistics, to replace the city with the idea
of a population. This is where we begin to see the economic
imperatives driving an economic logic inexorably offline into the real world, beyond scale,
beyond scope, into our action, into our behavior, into our society itself. That’s a terrifying image. I know, again,
we’re not trying to draw comparisons, but there is at least on an emotional level, it
generates feelings and images related to a totalitarian state. I understand that it’s
not, but emotionally, it feels overwhelming in that way. Well, it feels totalitarian because it feels
like a place of no escape. Yeah, an overwhelming power and control over
our lives. It feels like a place of absolutism because
computational truth replaces politics. Human kind is not meant to live in absolutism. Uncertainty
is a feature of the human condition, and that’s why we invented language, contracts, law,
society as a way of combining together and collaborating together in order to minimize
the downside of uncertainty, which is inevitable in the future. As we do that, building our trust, building
our bonds, learning how to work together, learning how to solve problems. This is the
social fabric. This is how social trust is created. Then if we imagine a world of computational
certainty, we no longer have to do that. We no longer have to work together. We no longer
have to debate and contest. We no longer have to collaborate. We no longer have to problem
solve. We no longer have to put our shoulders together in the face of uncertainty and make
our way to the future together. We just rely on the absolutism of the computational answer.
That becomes a new form of profoundly antidemocratic absolutism. Also, you touched on this earlier. We’re just
talking about the distinction. So, totalitarianism uses force in the fear of violence that’s
generated from the threat of violence to compel people to moderate their behavior. This is
something where the behavior is moderated by those who have the power instead of it
being from the outside and it’s from the inside out. Well, you’re right. The whole essence of totalitarianism
is that terror is used to shape behavior, but that’s why I think this distinction is
so important instrumentarian power. It’s the instrumentation that is used at an arm’s length
to shape behavior and you don’t know it. You’re not aware of it. You’re not afraid of it.
You’re not terrified. No one is trying to cut your head off or send you to the Gulag
or the concentration camp. So, as I’ve reiterated a few times now, the
surveillance capitalists are very proud of the fact that these systems are designed to
bypass awareness, so you never know it’s happening. You never know it’s coming. Therefore, you
are robbed of the right of combat. Resistance is annihilated at the root because we don’t
know that it’s happening to us. So, it’s a slow burn. It’s a slow burn, and
you go out and walk on the streets outside the studio. Everybody is walking on the sidewalk
looking at their phone. It’s already entrained. So, you hinted at this before and you’ve talked
about it at length in your book, but societies of human beings are complex systems. That’s
because freewill is a feature of the system. The economic systems that we have today, you
mentioned Hayek in the book, for example. Hayek deals with the information problem.
This is what prices are for. It’s because we can’t perfectly know as human beings what
the price of something should be, and so the market manifests that price in a sense. These economic systems that we’ve developed,
they don’t seem that they could work in this society, in this type of a system. I also
wonder, and I think you make the point in your book, I wonder also about democracy as
a system. In other words, democracy requires freewill also. You point some statistics on
your book about this loss of faith in democracy in Western countries. There’s also more and
more talk about universal basic income. You make the point about General Motors, at the
height of the great depression, employed more people than Google and Facebook at the height
of their market capitalizations. I think what’s clear from reading your book
is that there is no way to actually live under such a system and have these systems of government
and economics that we have. You devoted a good deal of time in the book, I don’t know
that we’ll get time to talk about it here, about solutions, ways of basically resisting
because you talk about it in terms of resistance. You also talk about outrage and finding our
outrage and naming and shaming. I think there are a lot of positive lessons
there. I don’t know if you want to comment on any of that before we go because I know
you have to get going, Prof. Zuboff. Well, you’ve raised so many important points,
and I think the big picture here is that I mentioned at the very beginning when we’re
talking about the definition of surveillance capitalism that there are many ways in which
surveillance capitalism seems to diverge quite dramatically from the history of capitalism,
the one you mentioned. The whole idea of why do we have to have free
markets, why do we have to have free enterprise, why all the emphasis on freedom, going all
the way back to Adam Smith, and this is the theme that Hayek, Friedrich Hayek, the architect
of neoliberalism took up. The whole idea was that we can never know what is going on in
the market. It’s so complex that it’s deeply mysterious, ineffable. Therefore, every actor
has to have maximum freedom to do the best they can to make the best judgments they can
because no one can see the totality. Well, now, in this new regime that we’ve been
discussing, we have a situation where the totality is visible. Yet, surveillance capitalists
are still demanding absolute freedom. So, here’s the great contradiction. What I say
in the book is that they know too much to qualify for freedom, that the whole justification
for freedom, for the freedom of the market was that we cannot know. Well, this is also, you touched on this with
the divisions of learning in society, right? This is what you mean by the division of learning
in society. Yes. So, now, the fact that they know too
much to qualify for freedom, we’ve entered the 21st century, which is a century in which
the key principles of social order are really about the division of learning, not much less
so now about the division of labor, right? It’s less about, “What do I do?” than “What
do I know?” The key questions for the division of learning in society, “Who knows? Who decides
who knows? Who decides who decides?” The basic philosophical questions, “Who will
guard the guardians?” That’s right, knowledge, authority and power,
the essential questions. So, now, we’re entering the 21st century with these huge institutional
creations of surveillance capitalism that have amassed these asymmetries of knowledge,
these concentrations of knowledge that really are unprecedented in human history. They know
everything about us. We know almost nothing about them. Their knowledge- That’s a remarkable statement. … is about us, but it’s not for us. So,
these are very peculiar, unprecedented asymmetries. The knowledge that they have accumulated is
not knowledge that they will and in many cases, even can share because this is now embedded
in these machine systems that create the predictions that drive the markets. So, this is what and
Durkheim talked about the philosophers, sociologist Durkheim, who first wrote about the division
of labor in society as the axial principle of the 20th century. He wrote about these
things can go haywire, and you can have an abnormal division of labor, where elite power
takes control of the economic machinery, and things are very unequal in the way they’re
divided, the way these opportunities to participate and contribute are divided. Feudal systems. Feudal systems, elite systems. Well, now,
we’re finding ourselves early stages of creating the 21st century society, division of learning
in society already pathologically shaped by these deep asymmetries of knowledge and the
deep asymmetries of power that accrues from that knowledge. Power. Right. So, what this means is that in addition
to economic inequality, we’re introducing a whole new access of inequality into 21st
century society, which is the inequality of “Who knows? Who decides who knows? Who decides
who decides who knows?” The answer to these questions are now lodged in the terrain of
the surveillance capitalists. What I say is that is not okay. That is an intolerable situation.
It’s a situation that no 21st century citizen should face. It is now up to us to name this, to grasp
it, to call upon, insist upon the resources of our democratic institutions, which have
largely slept while surveillance capitalism has flourished, and to insist that democracy
retake the high ground, that it’s democracy that decides the division of learning in society,
that it’s democracy that decides the future, that it’s democracy that oversees the trajectory
of the digital, that it’s democracy that invites new competitors to take hold of the digital
in ways that actually can fulfill its promise of empowerment and emancipation and democratization. Our democracies have to be reinvigorated,
resuscitated in order to understand what has happened, intervene, interrupt, outlaw this
new economic logic in its very specific mechanisms, so that the way is opened again where the
digital is freed from its enslavement to the surveillance capitalism, if you will, in this
parasite that has taken hold and hijacked it. It’s interesting. For listeners, in the overtime,
I’m going to go through some of these solutions and some of the things that Prof. Zuboff outlines
in her book because I also think they are market opportunities. You have a great quote.
I think it’s from actually an interview you gave. I tweeted it out. I really loved it.
It’s having to do with market opportunities because I completely agree and I’ve been making
this point myself. I think there’s a humongous market opportunity here because of this profound
market failure in not addressing the progress- This is a market failure, a market failure
of world historic proportions because nobody wants this. It’s an existential threat to the business
models of these enormous platforms. Now, Apple is an interesting case, of course, because
they’ve made a point to differentiate themselves from Google, and I think they’re trying to
take this competitive advantage, which is to say that, “We can offer you a platform
and give you back your privacy,” and by doing that, it’s taking a competitive shot at Google. So, we’ll see. We’ll see. Obviously, Apple has taken that stand. Tim
Cook has taken that stand, but Apple is still full of many contradictions, many things that
would have to be resolved, but were it to be an Apple or a company, alliance of companies,
alliance of business leaders who confound a new ecosystem that really plants a flag
in the ground changes the trajectory toward the digital future. This alliance, these new
operators, these new competitors have the opportunity to engage every single person
on planet earth as their customer because nobody wants to be entangled in surveillance
capitalism. Well, I think that’s where the optimism comes
from. I’m sure you’ve seen it because you’ve been touring now on your book, and you see
how many people respond to this. I think that’s what’s so optimistic about this. The demand
is there, and I think people are waking up because that anxiety is becoming impossible
for some of us to ignore. Absolutely right. I have every faith that
we triumph here because, and the key thing goes back to our distinction between technology
and capitalism, surveillance capitalism, specifically. Our societies have experienced successful
experience in fighting and tethering the raw excesses of capitalism to the requirements
of our populations, of our people, and of our democracies. We ended the gilded age.
We did it again during the depression. We did it again in the postwar era. We brought
law, regulation, and opened the way for new forms of competition that allowed capitalism
to reach an equilibrium with democracy. We called it market democracy. I see every possibility today as people all
over the world are being activated, and the naming is a key part of that. As people are
activated, politicians are being activated. I’ve just spent a week in Brussels, 10 days
in London. I’ve seen it in Europe. Politicians are being activated. Political leaders are
talking about surveillance capitalism. We see folks who have nominally been called users,
the name given to them by surveillance capitalists now beginning to understand we have political
interests, and we have economic interests. We have social and psychological interests. We want something else. The demand is there.
That is going to make its way into the political system, and it’s going to make its way into
the market system. Absolutely. So, I have every belief that we know how to
create this change, how to take on this kind of combat and do it successful. We’ve done
it before. We’ll do it again. This is not the end of the story. This is the beginning
of the story. I feel 100% the exact same way. I think it’s
been validated by many of the conversations that I have. Dr. Zuboff, I want to thank you
so much for coming on the show. Thank you, Demetri. That was my episode with Shoshana Zuboff.
I want to thank Dr. Zuboff for being on my program. Today’s episode of Hidden Forces
was recorded at Creative Media Design Studios in New York City. For more information about
this week’s episode or if you want easy access to related programming, visit our website
at hiddenforces.io and subscribe to our free email list. If you want access to overtime
segments, episode transcripts and show run downs full of links and detailed information
related to each and every episode, check out our premium subscription available through
the Hidden Forces website or through our Patreon page. Today’s episode was produced by me, and edited
by Stylianos Nicolaou. For more episodes, you can check out our websites at hiddenforces.io.
Join the conversation at Facebook, Twitter, and Instagram at @HiddenForcesPod or send
me an email at [email protected] As always, thanks for listening. We’ll see you next week.

9 Replies to “Shoshana Zuboff | Surveillance Capitalism in the Age of the Unprecedented”

  1. It's the apotheosis of the economic imperitive, says the wise sage in response to the interviewer's misguided concepts, the meaning of which I can deduce from the context as both accurate and smoove.
    However, I think the author's assumptions that 'the economic value of the "predictive data" is in marketing' obscures the interests by A.I. fanatics.

  2. Holy macaroni and cheese, the author had to suffer through classes by "her former professor" Boofy Skinner.

  3. Great episode. Have you looked into the STEM education push that has unfolded over the last decade or more? An aspect to this picture that has been overlooked, it seems to me crafted to serve the military industrial complex.

Leave a Reply

Your email address will not be published. Required fields are marked *