An algorithm, not a person, can now determine whether we get bumped from an oversold plane, wait two hours on hold, pay a surcharge that others don’t, or suffer a wide array of other consumer harms large and small. Businesses make these decisions based on an endless stream of data points about each one of us, using our behavioral quirks and vulnerabilities to make increasingly accurate predictions about our future behavior. Scholars in law and other fields have shown persuasively how such tools can reproduce and even worsen existing patterns of discrimination. They are correct to sound the alarm. But the existing literature devotes much less attention to the worrying prospect that our list of groups in need of protection— those persistently on the losing end of marketplace decisions— may now be woefully incomplete. This Article seeks to broaden the scope of our concern, explaining how these same algorithmic tools can create new, and in some ways more invisible, patterns of discrimination and disadvantage. Even more worrying, members of the new harmed groups may not realize that such groups exist as “groups” at all, or that they are members, hobbling their ability to join forces and fight back. Yet from the business point of view, these groups are very real, be they people with low self-esteem, introverts, or late risers, among innumerable other categories known and unknown. Drawing from scholarship on privacy law, consumer protection law, civil procedure, and social theory, this Article argues that the invisibility of these new harmed groups has troubling implications for distributive justice and democratic deliberation. The argument proceeds in two parts. First, I take the position that as long as such algorithms function without any meaningful chance of accountability, they impose a democratic harm on the entire polity. This is because the norms we use to assess the fairness of a given distributive mechanism aren’t independent of empirical reality. Rather, we update and apply our norms in response to our shared understanding of what’s happening on the ground—or at least, as I argue, that is how the deliberative process ought to work. The invisibility of these algorithmic tools short-circuits the mechanisms we would otherwise use to ensure oversight of emerging technologies and their uses in the marketplace. I further explain why giving members of the new harmed groups the opportunity to communicate among themselves would be essential for any such deliberative process. Second, I argue (perhaps counterintuitively) that litigation could play a crucial role in kickstarting this sort of democratic deliberation. Before more “classically” democratic modes of engagement can fulfill their potential, we may need the coercive power of the courts to help uncover what precisely is happening. Our legal system provides us with helpful procedural tools to shed light on these collective harms, but only if we use and refine them strategically. In particular, we must ensure our system of aggregate litigation remains able to surface and redress new patterns of harm where they occur. Only by giving the new harmed groups the chance to mobilize against the market logics that created them, empowering them to pursue collective action and articulate both moral and legal claims, can we stop market actors from dividing us all into winners and losers on whatever basis they choose.
