Self-driving cars with Nassim JafariNaimi: New Research in ST&HV
10 August, 2017
In this series of Backchannels posts, we’ll be highlighting new research in the 4S journals, ST&HV and ESTS. Here, Backchannels interviews Nassim JafariNaimi, author of the recent ST&HV paper, "Our Bodies in the Trolley’s Path, or Why Self-driving Cars Must *Not* Be Programmed to Kill." Dr. JafariNaimi is an Assistant Professor at the School of Literature, Media, and Communication at Georgia Tech. Her research interest is in the ethical and political dimensions of design and technology especially as related to democracy and justice.
Backchannels: What brought you to this research topic?
NJ: I came to writing about self-driving cars through my longstanding interest in values and design. For several years I have been exploring questions about multiple facets of the intersection of ethics and design, including what values are and do in the context of (design) actions, leading me to the thesis that values are hypotheses or theories of action. This idea has been the central thesis of my work ever since my days as a graduate student and I developed it further with my collaborators Ian Hargraves (Mayo Clinic) and Lisa Nathan (University of British Columbia) in a recent article.
Our article had just come out when I came across the short piece titled: “Why Self-Driving Cars Must Be Programmed to Kill,” based on an article that was later published in Science Magazine. The title was both provocative and disturbing to me. On the surface, the argument appeared straightforward: Many lives are lost every year due to car accidents. Many of those accidents can be avoided since they are due to human error. Machines make fewer errors although they are not perfect. Still, we are better off with machines in control than people, as this will reduce the number of accidents. Yet, as I explain extensively in the paper, a series of problematic assumptions are at work behind the seemingly straightforward argument. Moreover, in its discussion of algorithmic morality, the argument bypassed many years of philosophical debate about what values are and how they guide action, as well as issues such as algorithmic bias or vulnerability to hacking. Emotionally, I also found the prospect of living in a city where a machine or algorithm can decide to kill you dreadful and dystopian. Would you want to live in such a city?
I tried to write a quick comment in response to the many Facebook/Twitter shares of the article by numerous academic friends of mine. But that was not easy, as so many layers of assumptions needed to be unpacked. I then aimed for a short response that could be published as a blog or opinion piece, but this soon turned into a full article! In this way, the topic emerged as a spontaneous response to a public debate.
Backchannels: What challenges did you overcome to complete the study?
NJ: For better or worse, self-driving cars have captured the public imagination as well as lots of public and industry investment and academic research funds. Any argument that challenges the viability or desirability of self-driving cars should recognize this enthusiasm and carefully lay out the pitfalls. My approach was to write in a way that balances this technical imaginary with ethical imagery that is equally engaging and vivid. I sought to make it easy for researchers in many disciplines—including those predominantly focused on the technical side with little knowledge of the ethical/philosophical discourse—to engage with the subtle ethical issues at stake.
The Trolley Problem and its many variations gave me an opportunity to do just that. I could play with the rather rigid snapshots that are often provided by trolley problem scenarios and turn them into short stories that help readers begin to grasp the humanity of the situations and gravity of being put in a situation of choosing one life over another. This is also why I structured the paper around three questions—each marked by a vivid image (e.g., what if grandma is pushing a stroller?)—to engage with the three core premises of arguments for self-driving cars and the validity and/or necessity of algorithmic morality. It is also exciting, I think, because the argument contributes to the larger discourse in feminist ethics and philosophy especially the uses and limits of the trolley thought-experiments. For example, I talk about the objectifying and inhuman reference to the “fat man” and the role this reference might play in utilitarian positions that argue it is just to kill him in order to save lives. We might understand the situation very differently if we learned his name, his lifestory, and other details of the scenario that would illustrate the complexity and nuances of the situation more like life itself.
Backchannels: What lesson would you like people to take away from the paper you’ve just published?
NJ: Much is unknown about self-driving cars and a lot is at stake. It is my hope that we move away from overly simplistic and optimistic views of this new technology toward more nuanced debates that take into account how this new technology might shape the urban landscape and quality of our day-to-day lives. We might consider how arguments such as the one put forward by the MIT Technology Review piece risk normalizing accident deaths and make the idea of sacrificing some to save others seem acceptable or even inevitable. I invite readers to ask: can we use this turning point to make the urban infrastructure and experiences of mobility more convivial or would we give up our public spaces and possibilities for walking, biking, playing, and gathering to autonomous vehicles? What if we redirect the research talent and public and private funds devoted to deciding the “best kill decisions” to creating mobility infrastructures that eliminate the necessity of such decisions? I want researchers and designers to use the paper as a starting point to rethink and reframe the problem and imagine alternative possibilities.
Backchannels: What is next on your research agenda?
NJ: I am always working on several parallel projects that engage ethical and political aspects of technology from both theoretical and design perspectives. I am currently writing a theoretical piece that looks critically at participatory design and participatory media, especially in how they make use of stories of marginalized groups. At my design studio, my research group and I are also working on some exciting research projects: one on advancing public understanding of science, a feminist art/visualization initiative engaging biological data, as well as an ethnographic study of the smart cities initiative in Atlanta. Related to the latter, I am co-convening two panels titled, “Smart yet (in)Sensible? Feminist Critical Perspectives on ‘Smart Cities (I and II),” with Kathi Kitner (Intel) and Beth Coleman (University of Waterloo) at 4S this year. I am very excited about this panel because it is a venue to rethink the predominantly technological discourses on smart cities and to advance a productive critique of ongoing initiatives. I invite readers of this blog at 4S to join the panel— and please come up and introduce yourself!