My first encounter with cloud computing was in the early 2010s. It took the form of a problem—a crisis, even—that needed solving. By crisis I don’t mean a serious issue such as climate change, but rather the kind of business problem that’s treated as an existential threat, prompting late nights and “all hands on deck” emails from management.
In this case, it was a problem caused by a system that couldn’t keep up with demand. I worked for a company that I’ll call Libros, which ran a book-ordering website. The website was powered by servers located on-premises—that is, within our data center. A data center is a kind of specialized warehouse used to house computers known as servers. These computers are called servers because, unlike your personal computer, they are used to serve many people simultaneously. When you use a web application such as a banking website, there are servers in data centers behind the scenes. In a data center, you’ll also find the data storage, networking, and other equipment that together form computational systems.
Early in my career, I spent many hours in places like this. They’re kept very cold—think of the bone-deep chill of a February day—to prevent the equipment, which generates heat (lots of heat), from burning out. They’re also easy places to get injured. Although it was years ago, I can still vividly remember cutting my fingers (sometimes very badly) on the rails used to hold servers in place. No matter how many servers you successfully installed, there’d always be a few which were difficult to place, requiring careful work with hard metal and plastic in small spaces, a perfect recipe for cuts and pinches. It was an occupational hazard for many of us in those days, and we considered it a badge of honor.
Our data center worked, but the servers it hosted could become slow or unresponsive if there was too much customer demand. (“Too much” is a relative term: the amount of work a server can do is based on factors like the amount of memory and processing capacity it has—just like your laptop but on a bigger scale.) That was the source of the crisis at Libros: every year, as the popularity of our website grew, the system became more strained. Customers complained about slow response times, lost orders, and other annoyances. Upset customers meant lost revenue, which made our executives angry, who in turn made our managers demand solutions.
The trouble was that solving the problem required installing new servers. And new servers required money the company was unwilling to spend—a few million, not a trivial investment. So there we were, my colleagues and I, facing pressure to fix a problem but not provided with the tools to do it. There was another challenge: even if we were granted the money for the new servers, additional demand might easily outrun the added capacity, placing us back at square one. That is, by the time my colleagues and I finished installing the new servers—shivering and slicing our hands open in the data center—so many more users might be trying to access our website that the new infrastructure wouldn’t make much difference. Think of building a highway: if it fills up with cars right away, the traffic is just as bad as before.
What to do? The way we solved this problem was by turning to cloud computing, a new offering at the time—so new, in fact, that most techies and managers were unaware of it and, outside of early enthusiasts, those who were familiar with it harbored deep suspicions. (“No cloud” policies were not uncommon at the time.) We had no way of knowing that the technology we used to address our specific challenges, as helpful as it was from a technical point of view, would grow over the next several years to become extraordinarily large—indeed, to become the dominant form of computing, with immense consequences for the world’s political economy and for how the power of computation would be used and abused. What few people realize today is that, in those early days, the move to the cloud mostly wasn’t a top-down decision. It was typically brought into shops quietly, surreptitiously or against management objections, by the engineers themselves, simply because it made their jobs a little easier. We let the behemoth in through the backdoor.
Fee Fi Fo Fum
As our team at Libros sweated, one of my colleagues, always on the lookout for new tech industry trends, tentatively suggested a relatively new service from Amazon called Elastic Beanstalk. In those days, before Amazon’s rise into ubiquity, our first question was: Amazon? The bookseller? Naturally, the second question was: What the hell is an elastic beanstalk? We knew what the words “elastic” and “beanstalk” meant separately, but together it seemed like a nonsensical word sandwich.
“Hear me out,” my colleague said. “This is a web-based service we can use to host our customer portal—it can automatically expand and contract with growing or shrinking demand, solving our availability and response problems.”
Let’s pause here to consider the impact this news had on me and my coworkers. A major problem might just disappear, poof, because of a computational service we could use like water or electricity. In the same way we don’t run our own power or water treatment plants—rather, consuming these resources as a utility—Amazon was offering a utility model of computing. It had first begun doing so in 2006, through a new division called Amazon Web Services (AWS). Rather than running our website from our own data center, we could let AWS do it from theirs. And AWS would be able to increase capacity in response to rising customer demand much faster than we could.
Sure, there’d be real software development and networking effort required to get things started. But the promise of using something that was a commodity, rather than bespoke, was enticing. This was the appeal behind the new paradigm of cloud computing that was then emerging: we could rent access to data center resources like processor power, storage, and databases as needed.
We listened to that colleague and directed our plans, like a conference of nerd generals, to putting Amazon’s Elastic Beanstalk to use. There was, however, a significant hurdle: management did not trust the cloud.
Behind the Boss’s Back
Today, when the cloud is dominant, it’s difficult to understand just how hostile many managers were to the technology. There were a couple main issues. The first was that Amazon, the company that first offered cloud infrastructure services, was known more for book delivery than anything having to do with enterprise computing. The second, larger obstacle was the fear of a loss of control. Capitalist enterprises crave control and predictability—the ability to forecast spending, profits, and the behavior of workers. With on-premises computing, even an executive who didn’t know the difference between a SCSI disk drive and a microwave dinner (only a slight exaggeration) could walk into a company data center and survey the physical reality of all those tens of millions spent—the chill in the air, the blinking lights, the techies busy teching the tech. It was all there. Giving that up was a very hard sell for management in the beginning.
So my colleagues and I had to be stealthy. We diagrammed how to build an elastic web service—that is, a service that can expand and contract with customer demand—using Amazon’s cloud. We worked with the networking team to devise a strategy for redirecting user requests from the stressed, on-premises system to the cloud service. And we created a fallback plan, just in case our gamble didn’t pay off.
The late-night tests when demand was usually low were promising; the redirect and fallback methods behaved to plan. But the only way to really know whether the new service would work was to put it online during crush time. We (metaphorically) pulled a lever and prayed to the gods that it would all come together.
The customer rush began and the system responded as we’d hoped, automatically adding more capacity as demand grew and removing capacity as demand shrank. Management was pleased. For the first time in years, there were zero customer complaints and many smoothly executed sales. Eventually, after some time had passed, we let the cloud cat out of the bag. Surprise! We solved the problem by going behind everyone’s back and using new technology. At first, management was dubious. But it’s difficult to argue with success—and this particular success took pressure off middle managers, who for years had faced uncomfortable conversations with the C-suite because of crunch season complaints. The old saying that “it’s better to ask for forgiveness than permission” applied, because everyone came out looking good.
Of course, cloud computing also meant the professional lives of people like me would be utterly transformed. It’s difficult to overstate the effect on those of us who had built our careers working with what in the industry is known as bare metal—those of us who had been able to touch, and engineer into complex systems, the machines that run the software we use. For some, the rise of the cloud was seen as a threat—an unwelcome change that might eliminate their job, a not unreasonable concern, particularly for workers whose jobs were focused on tasks such as server maintenance and systems and database administration. But for many others, myself included, it was seen as a relief. When I thought of the lost weekends, the ruined dates, the late nights caused by server failures, I didn’t miss the old way of working at all. Because of elasticity, we no longer had to intervene when systems were stressed. Because of hosted platform databases (instead of databases running on our own servers) we could focus on using the database, rather than constantly worrying about whether a system would work as needed.
What I didn’t realize at the time was that alongside this new capability came the accumulation of a new form of techno-political power. I didn’t realize that the increasing concentration of data center capacity by a few US-based tech firms (primarily Amazon, Microsoft, and Google) would help intensify algorithmic harms and inaugurate a new era of monopoly. While trying to solve an immediate problem, my colleagues and I inadvertently helped this behemoth grow, welcoming the services it provided, failing to foresee the future we were making possible.
Points of Failure
What kind of world has the cloud helped create? Cloud data centers, which have grown so large that they now operate at what’s called hyperscale, enable cloud providers like Amazon and their customers to run the types of computationally demanding systems that only a very few companies could have hoped to assemble in the past. These systems have greatly enlarged the command and control capabilities of firms large and small. For example, without the cloud, Amazon would not be able to monitor their drivers or automate aspects of management in their warehouses. Moreover, the recent growth in machine learning startups—some offering dubious and harmful services such as facial recognition to determine hiring decisions—can be directly linked to the growth of the cloud. Hyperscale-level cloud computing makes it possible for these VC-funded companies to build complex software without needing to invest in expensive data center equipment and real estate.
Using the Marxist theory of base and superstructure, we can view the cloud’s data centers as an element of the mode of production—the combination of tools, machinery, technical expertise, and labor that propels capitalist activity. Computation is an industrial process. Data centers are created by bringing together computers (servers) and associated equipment that are manufactured using the extraction of minerals, the forging of metals, and shaping of hydrocarbons into plastics. The tech industry presents this hard physicality—which makes software useful—as a cloud, an amorphous, unlimited digital resource adding pleasure and efficiency to our lives. But there’s another way to view the cloud era: as the concentration of computational power and, therefore, real power with political consequences, into fewer and fewer hands.
This concentration means that only a small handful of companies—and more critically, their executives and investors—possess the ability to determine how computing capacity should be deployed. This not only grants them a vast amount of leverage, it also creates a new mode of failure. Consider the AWS outage on December 7, 2021. This event impacted many companies, including global content providers Disney and Netflix, connected devices such as Ring cameras, and even Amazon’s internal processes that utilize their computational infrastructure (“eating their own dogfood,” as we say). Before the cloud era, each organization might have made large investments in maintaining their own data centers to host the computers, storage, and networking equipment required to offer online services. But now, the access to hyperscale capability and the possibility (if not always reality) of reducing the cost of computation have pushed most companies into the cloud, leading to consolidations that have introduced new kinds of fragility.
At the dawn of the cloud era, none of this was apparent to me. Like my colleagues, I was focused on technical metrics such as speed and elasticity. What few of us realized is that we were facilitating the birth of a new form of power that rose, like Godzilla from the sea, to loom over everything. The cloud is rapidly becoming the computational infrastructure of global capitalism and the basis of immense digital empires that wield significant political power. But it was workers like me, hands on keyboards, toiling behind the scenes, who laid the foundation.