At the end of this chapter you will be able to:
1. Explain the difference between the objectives of normative and descriptive ethics and what can be accomplished with each.
2. Summarize the key tensions present in ethical discussions including limited resources, competing kinds of good, and different ideas about what is good.
3. Contrast the differences between using case studies and science fiction for studying ethics and technology.
4. Discuss the difference between personal and professional ethics and list reasons why studying ethics is important for technologists.
Should autonomous weapons be legal? Will I be cared for by a robot in my old age? Does it matter if I submit buggy code for a beta release? When I ran the red light, was there a camera recording me, and was it able to read my license plate? If it couldn’t get a clear image, will someone else get my ticket? Will other people be able to find out that I got a ticket?
Computing is a tremendously powerful field. Our daily lives, from travel to communication to medical care, have been shaped by computers and related technologies in ways that were barely imaginable twenty years ago. Computing technologies have already transformed the world, and they will continue to do so (Greenfield 2017).
These changes have not been wholly for the better. Social media has enabled people with common concerns and interests to find one another, which has benefited birdwatchers and indie music fans but also white supremacists. Putting financial records and transactions online has made banking more efficient and oftentimes more secure, but it has also left individuals and major institutions more vulnerable to hidden theft (Timberg et al. 2017). In addition, the growing technology gap between rich and poor countries could leave many developing economies at an even greater disadvantage than before (Baller 2016).
This mixture of good and bad is not unique to computing technology specifically, or even to technology in general. Anything that changes the world will have negative effects as well as positive ones. But when is a particular trade-off worth it? Which kinds of costs are acceptable, and how do we distinguish acceptable costs from unacceptable ones? Will this new product cross some sort of line established by religion, society, or the common sensibility of (some) people? Is it possible to design and execute a new project in a way that causes less harm?
These are hard questions, and they don’t have “right” answers the way that many technical problems do. Technical problems, in ideal, stylized settings, offer clear benchmarks for success (e.g., efficiently produce this output from that input). But if you’re trying to answer questions like the ones above (e.g., Should I build this system in this particular way? Should I build it at all?), the benchmarks are less clear. Those questions—ethical questions—demand that we evaluate the merits of the benchmarks themselves. They require us to consider why a project, or a particular way of building it, is helpful or beneficial; whom it helps; and what specific kinds of benefits it produces, or to articulate why it is the right thing to do. It also requires that we imagine how our projects affect the world, what kinds of harm they might cause, and to whom.
Technology has enormous potential to make human lives safer, freer, healthier, more interconnected, more full of beauty, and less plagued by poverty. It also has the power to dramatically increase inequality, to enable people with malicious intentions to harm others, and to create the conditions for people to make poor decisions. Which will your designs and implementations do? Asking the right questions, and answering them thoughtfully, is crucial to ensuring that your designs, systems, and work help make the world better.
The future in which we live, and the lives of those who come after us, will depend in part on your actions and choices as a technology development professional. That’s why it’s important that you understand what is potentially at stake in your work, whom it will impact, and how. You will be much better equipped to do these things if you take some time now—before these questions arise in your own working life—to clarify your own goals and values, to understand the questions you can ask, the kinds of choices you can make, and reflect on what it means to live up to those goals and values.
This book won’t give you answers for how to be a good person or how to build good things. It will give you tools for thinking about problems as they arise, so that you can answer your own questions or talk productively to others in the field to figure out answers together.
This book introduces some of the persistent ethical problems and issues in the design, development, and deployment of technologies, especially those related to computing. However, our coverage of these issues is necessarily incomplete. Instead, we use these problems and issues as a jumping-off point to engage with the basic questions that one should ask in any situation where values are at stake. There are two reasons for this approach. The first is that technology changes all the time. Our daily lives are shaped by apps and platforms that did not exist ten or even two years ago or that existed in a very different form, with different capabilities and pitfalls. So rather than talking about specific issues related to specific technologies, which come and go and change rapidly, it is more useful to deal with the structural questions they raise.
The second reason for this focus on basic questions is that even though the ethics of technology and computing may present themselves in different forms, they are not necessarily different from other kinds of ethical questions. The problems examined in this book take their shape from recent technological innovations, but many of them are old problems, which resurface again and again in different areas of life. Accordingly, this book draws on philosophy, sociology, political science, literary studies, psychology, and the history of science. These cross-disciplinary insights only scratch the surface of relevant knowledge held by experts in these fields, but even these brief glances can give you a sense of how those experts could help you in your own future work as either consultants or collaborators. Our discussion in this chapter, as well as throughout much of the textbook, focuses primarily on computing technologies, both hardware and software. That said, many of the topics, problems, and questions apply more broadly to a range of specific issues concerning technology in the twenty-first century.
The point of this book is not to tell you what to think, nor is it to give you a formula for determining the correct ethical choice in situations you encounter. Instead, we hope to equip you with “a means and a process for achieving [your] own moral judgments” (Callahan 1980, 71; emphasis added). That is, we aim to train you to recognize ethical problems when you encounter them and to give you several sets of tools that you can use for evaluating and making decisions about what to do. As with any kind of problem-solving, the most important step is often a matter of figuring out what to ask.
Throughout the book, you will encounter boxes like this one standing apart from the main text. We use these sidebars to highlight common misconceptions, elaborate on important but tangential points, offer alternative ways to understand issues, and recommend further reading. Many of the sidebars are meant to offer jumping off points for discussion or additional inquiry.
Conversations about ethics would be easy (or at least easier) if we all agreed either on what the world is like or what it should be like. But in practice, we don’t agree—and in general, we aren’t very good about talking about the reasons why we don’t agree. Our ideas about good and bad are often rooted so deeply in our personal experience that we find it hard to imagine how a decent person could not share them. That’s why, in practice, people often use the words “ethical” and “unethical” to mean “I agree with X” or “I don’t like Y.”
The language of ethics carries tremendous power, even when it is used clumsily or without reflection. If you’ve spent much time watching people argue about ethics on the internet, you have probably felt tempted at some point to give up on the language of ethics entirely. Although the authors of this textbook sympathize with that response, remember that there are also many good-faith attempts to discuss ethics (both on- and offline) and this discussion can be productive. Understanding the underlying logic of ethical assertions will help you better understand these discussions, even when they are sloppy or happening in bad faith. It will also equip you to better communicate with classmates or colleagues when working through ethically charged subjects.
Ethical statements typically come in two varieties: normative and descriptive. In a normative statement one provides an assessment of how things should be rather than how they are: for example, “he was wrong to do that” or “being kind is more important than being the most successful.” Normative statements are rooted in value judgments about what is good and what is bad, or what is permissible and what is forbidden, or they are assessments of the relative value of different things. Ethics is often (mis)understood to be only normative: that is, to be aimed at establishing norms of thought, values, or conduct. This assumption is especially prevalent in many professional ethics courses, which are often used as a means to steer students’ future behavior toward a set of professionally agreed-upon values, such as professionalism or honesty (Colby and Sullivan 2008; Martin and Weltz 1999). We discuss the professionally agreed-upon values for computer science, particularly some of the professional codes of ethics, in chapter 6.
But ethics can also be used as a tool for description, furnishing you, the decision maker, with a critical framework that enables you to understand what is happening in a given situation and what is at stake in any action you might take. Learning about the descriptive functions of ethics is as important as learning the professional norms of the field. Computer science is a field in which everyday practice and problem-solving takes place in a context that could barely be imagined a decade before. We cannot predict the ethical quandaries you will face. But with an education in ethical description, you will be equipped to engage in careful and substantive ethical reasoning when new and challenging problems confront you.
While description is an essential part of ethics, it is vital to recognize that making a judgment always necessarily involves norms. Although it is the normative element that often makes ethical judgments recognizable, the process that is used to arrive at ethical judgments is often described in a way that erases the norms themselves. When this erasure has happened, ethics appear to be simply a matter of knowing the relevant facts, and it can seem as if normative assessments follow automatically from an accounting of those facts. But a person who reasons in this way is not being objective, even though they may feel like they are. They are simply overlooking the foundations of their own observations and logic.
Even apart from the question of underlying norms, “simple factual descriptions” are not as simple as they first appear. There are many different ways to describe a single situation, and those differences often reflect normative judgments, intentional or otherwise, on the part of the person doing the describing. Some of those differences are a matter of which details you choose to include and which details you overlook or decide aren’t important. That process of choosing details involves a normative judgment about what is relevant or useful for understanding a particular situation. But it’s also important to realize that two different people describing exactly the same details might perceive those details in different ways. Those differences both reflect the normative judgments of the person who is describing and implicitly communicate normative judgments to others.
As an example of how normative judgments shape even “simple factual descriptions,” imagine a newspaper article about a person who has been arrested near a crime scene. Imagine that this person could be accurately described either as a “suspect,” a “loiterer,” a “passerby,” a “boy,” or a “juvenile.” The newspaper article could use any of those words and meet the criteria for simple factual description, and yet each word would communicate different normative judgments about what kind of person this is, what he might have done, and what he deserves. The newspaper’s decision in this situation is unavoidable; any choice they make is normative and is possibly more persuasively so because the chosen word appears to be only descriptive. Every act of description involves similar choices, even if the describer is not aware that she is making them.
Ethics can be used as a tool for making normative judgements or for reflecting on the methods and criteria we use to reach those judgments. But, as stated above, ethics can also be a tool for description. The difference between descriptive and normative may seem clear in theory, but in practice, the distinction gets fuzzier the closer you look. Which facts are the most important in a given situation? How do we describe them? How do we fill in the gaps between the facts we have? These are all questions about how to describe a given situation, but any one of them could affect the ethical judgment you or others make in the end.
Every person has their own ideas about what is ethical and what is not, including you. Those ideas are shaped by the culture around us, but they are also specific to each of us. This book will help you to recognize the normative judgments that are part of the cultures around you and to understand their implications. It may also challenge you to reevaluate some of your beliefs about the world, in light of those values—or to reconsider your values, in light of your beliefs about the world.
It can be intimidating for those in the engineering and technology development disciplines to grapple seriously with ethical questions because, as these technologies ingrain themselves in almost every aspect of our lives, the questions seem too big and the stakes too high. As a reaction, you might find yourself tempted to focus solely on the technical dimensions of your work precisely because the social and ethical dimensions seem too big or too removed or even too grim for you to take on yourself. Many of us become scientists and engineers because we want to build things and to solve interesting problems, and the nontechnical questions raised by your work may feel insurmountable in a way that makes you want to focus on only the building part. However, you likely wouldn’t release an untested program or drive across a bridge that had not been inspected, and building a system or a program without reflecting on its possible uses or impact is just as irresponsible. Many of the major avenues in computing, from autonomous robots to messaging apps, have an incredibly broad range of implications and applications, and many of those implications are still poorly understood. Even the developments that seem straightforwardly good, such as the digitization of medical records or easy access to live-updating navigation programs, emerge as more ethically ambiguous, or even harmful, when we look closer at who is using these technologies, how they are using them, and for what.
The good news is that in many cases you don’t have to do this kind of thinking alone. In most settings, the best practices endorsed by professional societies and design professionals place emphasis on foregrounding multiple types of reflection in multidisciplinary teams, often referred to as participatory design (Lee et al. 2019; Vines et al. 2013). Development is a social enterprise, and we must confer with experts in other fields, best practice manuals, and focus groups of potential users to help think through design choices and to anticipate how a given design will be adopted and integrated when deployed in the world. Unfortunately, not all work environments are willing to dedicate time and resources to this kind of interdisciplinary consultation—and even when they are, you, the computing professional, still need to have a sense of what to pay attention to, and what to read about or ask about, and when.
Given the diversity of perspectives, values, and cultures in the world, dealing with ethical issues will never be easy. But there are ways to get better at identifying ethical problems and methods for determining more ethical courses of action. The ethical issues that arise in computing fields often look brand-new because they deal with phenomena that haven’t existed before, such as intelligent robots or the unprecedented volume of available information. But, in fact, human beings have been wrestling with similar versions of these same questions for the whole of recorded history. When the ancient Greeks argued over the ideal values to instill in their children and which methods of education that would be most effective, they were grappling with many of the same issues that roboticists and programmers grapple with today when they contemplate the value and risks of artificial intelligence (AI). When educators and scholars in fifteenth-century Europe debated how the printing press might democratize (or destroy) established bodies of knowledge, they were struggling with many of the same issues that, in contemporary times, plague social media platforms like Facebook and Twitter (Greenfield 2017).
The absence of a foolproof formula for resolving ethical quandaries in all the situations you face doesn’t mean that there are no codes or guidelines for ethical action. There are many different useful approaches to tackling ethical challenges. All of these are built on some assumptions about the world. What this means is that different ethical frameworks rely on different ways of conceptualizing the world and thus of conceptualizing the problem; part of the challenge is figuring out which approach best suits the problem you are confronting. This book will help you recognize ethical problems when you encounter them and give you concepts and tools that will help you analyze situations and make informed and responsible decisions about what to do.
Over the years, ethical theorists have come up with different systems of thought to help think through difficult ethical problems. Studying these ethical frameworks will give you different ways of asking questions about tough ethical challenges, as well as different ways of answering them. But as we discuss in chapter 2, strengthening our capacity to identify, analyze, and respond to ethical issues requires much more than applying concepts to specific situations. By learning about different kinds of ethical frameworks, you will be able to find the approaches that make the most sense to you and to use concrete language to explain your position. You will also be better able to understand the views of those who disagree with you and learn from their perspectives, even if you don’t agree with them.
There are some types of problems that come up repeatedly in ethics. Although every situation will be different, recognizing the kind of problem you are facing can be very helpful. It will allow you to focus on what is most important about the situation, and it will also enable you to think about how you or others have handled similar problems in the past.
It is often the case that there is no single best answer. In most cases, every possible choice has drawbacks of some kind. It is rare to be able to arrive at a clear-cut solution to a problem that is universally satisfying. But if you have a clear grasp of the nature of the problem you are considering, you will find that you are better able to conceive of and identify a way forward, with a better grasp of what you are sacrificing and why.
The first problem that we see often is the reality of limited resources. What should be done when the demand for something—such as jobs, oil, medicine, or teachers in the classroom—outstrips how much of it is available? When somebody donates a kidney, do you give it to the patient who is most seriously ill and will otherwise die in a few weeks? Or do you instead transplant the organ into someone who is still mostly healthy and who will likely recover more completely and live longer afterward? Note that this is not an abstract concept for those developing computing technologies—the largest paired organ exchanges in the world are being built and maintained by computer scientists (Purtill 2018).
Problems created by limited resources often seem as if they could be solved by balancing the scales, usually with technology: artificial hearts to replace the failing ones, more fuel-efficient cars so that oil is less in demand, or high-yield crops that can feed more people. In some cases, solutions like this can really help, but more often than not they do not solve the problem completely. Typically, there are other costs associated with the solution; for example, the crops deplete the soil quickly or additional costs of driving are made even worse because people drive more. But even more important for the study of ethics is that this kind of problem-solution structure assumes that everybody can agree on what the most important problem is, and that is rarely the case.
In making an ethical decision, there is often a need to consider competing kinds of goods: multiple goals, or good things (not necessarily objects), that come into conflict. At a basic level, the choice to attend one college instead of another is an example of competing goods. It might be a simple matter of which school you like better or which has a program for your major; on the other hand, it could be a question of what your family expects of you or what they can afford. Or maybe you have the chance to work in one of two research labs, each of which pursues a different kind of exciting projects, but you cannot work in both because each job would require you to commit all of your student work hours.
Even when people agree about how to think through a particular ethical question, they may still disagree because their aims are different: they have different ideas about what is good. Sometimes the clashes are obvious: abortion-rights advocates want it to be easy to terminate a pregnancy safely, and abortion opponents want to prevent this exact thing because these two groups have very different ideas about the morality of abortion and what is at stake. Other times, though, people who seem to share the same goals disagree significantly about how to achieve them: is nuclear power a good option for the environment or a terrible one? Two environmental advocates with the same set of facts and the same overall goal can come to different conclusions on nuclear power, because they disagree about how best to assess environmental well-being or have different ideas about what policies or practices could be changed.
In the case of the donated kidney given above, a doctor advocating for the sickest patient might say, “the whole point of transplant surgery is to save people’s lives past the point when other kinds of medicine can help them; we should help the person most desperately in need.” Another doctor might say, “the sicker patient will never be able to lead a normal life again, but the less-sick patient could live a long and healthy life after the surgery, so it is a better use of this scarce resource.” These two doctors don’t just disagree about which patient is more deserving; they disagree about how to decide what counts as most deserving. These kinds of fundamental disagreements are often invisible even to the people who are having the argument because they both describe their ethical vision in the same way: in this case, they both want to improve people’s lives with transplant surgery. But they interpret that goal to mean different things.
What is good? What do we owe to others? How do we balance our obligations to others against our obligations to ourselves? And when we make judgments like these, where do our criteria come from? Studying ethics isn’t about arriving at decisive answers to any of these questions. But studying ethics does make it easier to confront them, and it allows you to think more clearly about what is at stake in specific situations in which perennial issues arise and to explain your reasoning in an effective way.
There is a common misconception that good or ethical people ignore their own needs and sacrifice themselves for others. This is not true! Being ethical does not mean that you have to be selfless all the time or completely give up on your own success or pleasure. None of the ethical outlooks discussed in this textbook require the agent to forget about her own well-being. There are some ethical traditions, such as some forms of Christianity, that demand selflessness of this kind, but those ideas of good and right are specific to that value system rather than intrinsic to ethics (or even to any given ethical framework) in general.
Self-interest is not a single unified thing; it is a lot of kinds of things all tangled up together, and some of them are necessarily in conflict. There are also different ways that a thing can be good for you. Even when you’re thinking only about your own interests and desires, you often have to choose between two good things (such as going on a fun vacation versus staying home to work on an important personal project). We often imagine ethical conflicts in binary terms: “Should I do the thing that’s good for me and bad for you, or the thing that’s good for you and bad for me? ” But the options, and their stakes, are almost never so binary; there are often many possible choices. But more to the point, there are lots of ways to evaluate how something could be good or bad for you, or how it could be good or bad for the others who will be affected by your decision.
Some kinds of goods really are zero sum. For example, when you audition for a role in a play or a spot in a band, you are in direct competition with others who are auditioning for that same role. Their success in this particular area means your loss in that area, and vice versa.
But many, even most, other kinds of goods are not a matter of direct competition. It is entirely possible to take pleasure in the experience of helping, giving, or sharing scarce resources such as money, time, or cake and that pleasure can outweigh those resources that have been given away. For example, many people who volunteer their time talk about how much happier they are now that they are spending part of their free time improving the lives of others. It is also possible to create better situations so no one must sacrifice directly: for example, adding more members to your project team so that everyone can contribute and share in the success does not require any notable sacrifice.
Being ethical does not mean that you have to give up on your goals for your own life or give away the things you value. It does mean considering the well-being of others as well as your own. And in some cases, it means choosing to value mutually beneficial things as much as (or more than) benefits that come at the expense of others.
Sidebar: Ethics and Morality—What’s the Difference?
It is not uncommon for the terms “ethics” and “morality” to be used interchangeably. However, the two terms may have different connotations for different people. In fact, moral philosophers sometimes disagree about whether ethics is a subset of morality or if morality is a subset of ethics (Piercey 2001). Before offering a definition of the two terms, it might be helpful to consider whether you think there is a difference between the questions “is it moral? ” and “is it ethical”? If you decide the two terms are getting at something different, then consider: what is it, exactly, that marks the difference between these two terms?
Throughout this textbook we use the term “morals” to refer to a person’s standards of behavior or beliefs concerning what is and is not acceptable for them to do. We use the term “ethics” in a similar way, but we additionally use the term to refer to the thoughtful reflection on and application of our standards of behavior and beliefs about what is acceptable to do and not do. The distinction between the two terms is intentionally subtle for two important reasons. First, as noted above, we draw from a range of different scholarly disciplines, and the two terms are not defined in the same way across all of these disciplines. Second, many of the references we use do not make a point to distinguish the two terms, and so forcing a distinction would potentially be misleading.
You may be wondering, why do I need to make time for ethics of computing technologies specifically? Many ethics classes are located in arts or humanities departments and not in departments such as informatics, computing, or engineering (Davis 2009). There is increasing consensus among experts that focusing on specific problems related to those practitioners who design and develop technology and its use can give you the tools to succeed in your professional life. Moving these problems closer to the applied work you will do helps you learn how to recognize and deal with these issues as you become a professional (Narayanan and Vallor 2014; Rogaway 2015).
Many of the major issues involving the ethics of computing technologies can be understood as a subspecies of one of the major types of problems discussed in section 1.3.1: limited resources, competing kinds of good, and different ideas about what is good. In this sense, the ethical issues that arise from even the most cutting-edge technologies are rarely new problems. New technologies, from computers to irrigation to writing, frequently change the conditions in which ethical problems manifest, and these changes sometimes mean that one kind of problem is replaced by another, but it is often the case that the same basic patterns persist.
Yet it is also true that modern computing technology (including information processing and communications systems) has brought about some specific shifts in human social configurations on both a local and a global level. These changes in human society need to inform the way we ask and answer ethical questions.
In the 1980s, as information and communication technologies grew expansively, social critics, scientists, and engineers began to see the need for a specific focus on computer ethics to address the societal and individual ethical issues raised by computing technologies. James Moor (1985), one of the first of such critics, argued that there are features of computer technology that have changed, or soon would change, a broad range of human activities and transform long-standing social institutions. The problem is that we lack a conceptual framework for addressing these changes and in turn are faced with policy vacuums, in which regulations, laws, values, and norms have not been established.
One of the features of computing technologies that Moor argued makes them unique is what he called the “invisibility factor.” Think for instance of the numerous computer chips and programs running silently in your car or on your smartphone. Moor targeted three issues that stem from the invisibility factor: (1) the malicious invisible abuse of the lower-level operations of a computer to invisibly exploit the user who is, for example, transferring money; (2) the invisible programming values present in any decision-making algorithm that may ensure that different people are invisibly advantaged or disadvantaged by the use of that algorithm; and (3) the largely unchallenged assumption that computers perform invisible complex calculations that no human is able to inspect and understand completely. We pick up many of these themes of invisibility throughout the book.
Technology and society scholar Deborah Johnson (1998) has argued that three specific changes have been brought about by the broader societal reconfigurations stemming from computer technologies:
These broad social changes are still very much in progress, and what happens next has not yet been decided. Much of it will depend on what you, your peers, and your coworkers build in the near future. It will also depend on how you choose to build it. If you are empathetic, creative, committed to your own values, and willing and able to think carefully about what and how you build, you can play a role in creating a better future.
There is no denying the fact that modern computing technologies have enhanced human life in various ways. In addition to helping human beings achieve certain goals, it can be argued that the mark of a great design is that it allows human beings to imagine new goals that would have been unthinkable without that particular piece of technology. But what kinds of goals are worth having? What makes a human life good or complete?
This question might seem more appropriate for a department of philosophy (or literature, or social science) than for a computer science or engineering department. It’s not a question that a computer science student usually has to answer in order to complete a programming assignment or that an engineer must confront to fulfill a particular client order. But as we will discuss in this textbook, technology increasingly creates the conditions in which human beings live their lives; imagine how different our present world would be without social media! The choices and actions of technology creators can have a meaningful impact on the wider world—and those choices and actions are based to some degree in how they answer the basic philosophical question of “what is worth having or doing? ” Because of the potential impact of their work, computing professionals have an obligation to think about that question and to learn about how others answer it. Human beings have a tendency to treat our own personal, individual answers to this question as ground truth, when in fact there are many ways to answer it. Different people may have widely varying ideas of what makes human life good and valuable.
Practitioners, professional societies, teachers, and leaders in the fields that deal with the design and development of computing technologies have a responsibility to drive the discussion about the impacts of their own work (Boyer 1997). The research and development community around big data, AI, and machine learning writ large has started to address this in a number of interesting ways, including the International Joint Conference on Artificial Intelligence (IJCAI) letter on autonomous weapons research (Future of Life Institute 2015) and the follow-on letter signed by CEOs of tech companies around the world (Conn 2017); technology companies releasing statements about their values in relationship to IT and AI (Pichai 2018); the Association for Computing Machinery (ACM) statement on Algorithmic Accountability (ACM US Public Policy Council 2017); the development of the Institute of Electrical and Electronics Engineers (IEEE) standard for algorithmic bias considerations (IEEE Standards Association 2021); and new conferences and research groups focused on fairness, accountability and transparency (ACM Conference on Fairness, Accountability, and Transparency 2021) as well as conferences focusing on the impact of AI on society (AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, n.d.). Many technical conferences encourage or require statements about the ethics of a project with paper submissions about the project. The utility of such debates is not that they result in standardized practices but rather that individual practitioners become more thoughtful and better informed about their work and its long-term effects. As in other areas of thought, this diversity of viewpoints is a strength when it can be harnessed to achieve a productive exchange of ideas and perspectives. The goal of this textbook is to foster your ability to fruitfully participate in such debates by informing you about the range of ethical, descriptive, and evaluative tools available to you.
As a professional who works with the design, implementation, and deployment of computing technologies, you will be able to do far more good in the world if you take the time to understand what other people value and why, instead of trying to make them conform to your own assumptions about what they should need or want. Although programs can specify value in a single variable, human values are complex. Although human desires and values can be influenced, they are deeply embedded, and humans often make decisions on the basis of deeply held values without analyzing those values or even being aware of their role in the decision.
There is also more than one way to talk about how human beings value things. Some argue that all human desires boil down to basic biological drives for food, sex, and physical safety. Others argue that more conceptual values—things like love, justice, or personal recognition—are distinct from the physical drives (Haidt 2012). But however they are measured or talked about, values and desires are an ineradicable part of human life. They play a fundamental role in how human beings think and act. Your own values undoubtedly play a role in your life and career goals, and in your “dream projects” as well.
Although modern computing, information, and communication technologies can definitely help to make human life better, they do not by themselves help us understand what makes human life better. People who work on the design, development, and deployment of computing technologies, much like everyone else in the world, typically default to the problem-solving approaches that they use most frequently and that have been helpful to them in the past. However, there are profound differences between humans and computers, hence treating questions about human good as an engineering problem is not always sufficient. That’s why this book, and this course, will often demand that you think in ways that may feel unfamiliar or even uncomfortable. The critical skills required for responsible ethical thought are not the same as those required for good work in the design and development of computing technologies. But if computing is going to achieve the goal of making human lives easier and better, then computing professionals need to be good at both.
Why use science fiction to study ethics? It is often the case that introductory textbooks on professional ethics use case studies. Case studies are specific scenarios that capture common or representative challenges in ethics. Sometimes they are based on specific events that actually took place, and sometimes they are fictional. One reason that case studies are useful in the classroom is that they give everyone in the class a shared set of information to work with. Another reason is that a good case study captures some of the messiness and complexity of ethical quandaries when they occur in the midst of other experiences, as they do in real life. A good case study helps capture the reasons why a given problem is difficult to recognize or understand and why it is hard to figure out a good response. In this book we use stories to stand in for case studies, if you want to find additional case studies to supplement the stories, the National Academy of Engineering maintains a large list of ethics related material for professionals including numerous case studies (Online Ethics Center for Engineering and Science, n.d.).
A good science fiction story can function in the same way as a case study. They are also more fun! But that’s not all. There are also reasons to think that science fiction can be more useful than case studies for honing your skills in ethical analysis. For one thing, stories are embedded in larger story-worlds, in which many other things are also happening. In a good story, you are not merely presented with a problem that requires solving. Instead, you get to see a particular individual (or set of individuals) with their own goals and interests and unrelated problems, who ultimately has to confront the ethical quandary. In this way, stories can better capture how challenging it can be to pay attention to ethical issues in the first place. They also can effectively capture the reality that ethical quandaries tend to crop up unexpectedly in the middle of a project, when you’re thinking about something else and have other plans.
Another advantage of stories is their characters. Instead of the featureless figures of a case study, a story’s characters have personalities of their own that shape their perceptions and choices. By watching a story’s characters make decisions (and, in many cases, mistakes), we can gain perspective on how we might react in a similar situation, and why. Stories can also help us understand how a particular person might have interests, concerns, or vulnerabilities that we might not share and that are affected by the story’s situation in ways that we otherwise might not have noticed. This combination of qualities also helps an engaged reader develop the moral imagination that is a key component of successful ethics education (Callahan 1980).
Finally, stories can be a useful way to get some distance from a charged issue. For example, many people are likely to have strongly held opinions about many technology-and-society issues such as warfare, elections, and appropriate forms of public speech. Because of that baggage, it can be hard to have a productive conversation about the underlying ethical issues or even to get to those issues in the first place. But if those same tensions and dynamics are relocated to an unfamiliar place and time, where nobody in that space has been affected, it can be easier to understand and appreciate a range of perspectives on the events that take place and the conditions that have created them. This productive estrangement is perhaps best described by a quote from philosopher Martha Nussbaum:
[Fiction] reading frequently places us in a position that is both like and unlike the position we occupy in life; like, in that we are emotionally involved with the characters, active with them, and aware of our incompleteness; unlike, in that we are free of the sources of distortion that frequently impede our real-life deliberations. (Nussbaum 1990, 48)
Science fiction in particular is an easy fit for an ethics of computing technologies course. For at least the past 75 years, engineers and futurists have recognized that science fiction is, in key respects, better able than “realistic” fiction to reflect the near future (or possible futures) in which computing professionals work. Additionally, many science fiction writers are deeply immersed in the literature about cutting-edge technological progress. Some of the most famous and influential ones are, themselves, working scientists and engineers. As Alec Nevala-Lee writes of Astounding, one of several popular science fiction magazines that launched in the early twentieth century,
[The] magazine counted Albert Einstein and the scientists of Bell Labs among its subscribers, and it made an indelible impression on such fans as the young Carl Sagan, who stumbled across it in a candy store.… [These] writers were creating nothing less than a shared vision of the future, which inevitably informs how we approach the present. Science fiction’s track record for prediction is decidedly mixed, but at its finest, it as a proving ground for entire fields—such as artificial intelligence, which frequently invokes [science fiction author Isaac Asimov’s] Three Laws of Robotics – that wouldn’t exist for decades. (Nevala-Lee 2017, 11–12)
The science fiction stories collected in this book represent a range of possible futures. Some of the technologies in the stories are far beyond anything that exists right now; others could be developed and deployed in the next decade or the next month. The point of reading these stories is not to worry about the technical questions surrounding their design and implementation: some of them are not necessarily all that technically plausible, although they often resemble other present and future technologies that are plausible or might actually exist. Rather, the point is to immerse ourselves in different sociotechnical circumstances and to consider how society and the individuals in it adapt to that technology and are influenced by it—often in ways they don’t expect.
My work was to make him look larger than life, heroic, a proper vessel for the grand project of atoning and reconciliation that he represented for us all.
Throughout the text we have inserted story points such as this one. We elaborate more on how to integrate both the science fiction stories and the textbook in the “Introduction to the Story Bank” in part 2 of this book. Like a good set of case studies, the stories that we have selected for this book raise many questions and serve as examples, guides, and jumping-off points for discussion, description, and deeper investigation. These are some, but of course not all, places where the themes of the book intersect with the particular science fiction stories that we have included. Each story is prefaced with a story frame that highlights important topics, themes, or events of the story that one should read critically.
The future society of “Apologia” has perpetrated great wrongs in its past—racialized violence, colonization, and genocide—and the guilt weighs heavily. But because this future society possesses technology far more advanced than our own, they are able to conceive and execute an innovative solution to this guilt: they send a poet back through time to the moments of greatest horror, to apologize to the “natives” of those moments for the harm done. It’s not hard to figure out that this solution is intended to benefit the denizens of the poet’s own time rather than the people to whom he is apologizing: the story’s narrator, the editor who splices together the public video feed of these apologies, says as much. By forefronting this reality and by taking us deeply into the social and cultural dynamics that lead to the Apology project and those that emerge from it, “Apologia” invites us to consider how and why such “solutions” come to pass—even though apologies after the fact cannot undo harm.
As is the case for several stories in this book, fanciful or improbable technologies offer us a fresh perspective on how humans use, inhabit, and/or are otherwise shaped by the technologies around them. On the basis of what we know of the world today and how technology is currently integrated into our lives, what would people be likely to do if new technological avenues became open to them? Stories like “Apologia” challenge us to move beyond shallow optimism and to imagine those possible technological futures with clarity, insight, and a more timely attention to potential harms.
Throughout most of this book, we discuss situations in which the answers to ethical questions are not clear-cut, and even the presence and nature of an ethical question is not always obvious. Often, it is not in the power of one person to make decisions about the appropriate design, implementation, and use of computer technology. Making progress on these questions is a long-term, society-wide effort, and deferring our judgment on them is often a better idea than deciding where we stand beforehand.
But as a practicing professional, you may face situations in which you do not have the luxury of deferring your judgment. In some cases, there are applicable principles that have been agreed upon by much of the profession or by a governing authority, even though you may personally disagree. In addition to the ethical norms of society at large, it is important to be familiar with the ethical norms of your field.
One of the roles of professional societies in any field is to articulate a code of ethics for practitioners in that field as well as to articulate the collective wisdom of the people who work in that field (Johnson 1998). Perhaps the most famous example of such a code is the Hippocratic Oath, which requires medical professionals to “use treatment to help the sick according to my ability and judgment, but never with a view to injury and wrong-doing” (American Medical Association, n.d.). These codes can provide guidance when the most accepted way to proceed needs to be stated unequivocally, or sometimes because the way forward is unclear.
Within computing-related fields, there are two major professional societies: the ACM (ACM Code of Ethics and Professional Conduct, n.d.) and the IEEE (Institute of Electrical and Electronics Engineers, n.d.). Each of these societies has a professional code of ethics and extensive online resources for professional ethics, as does the National Academy of Engineering (Online Ethics Center), which acts as an umbrella for most engineering disciplines.
Like all ethical codes, those of the IEEE and the ACM aim to strike a balance, offering consensus on some fundamental principles while still allowing for a range of views. Both sides of this equation are very important for any code of ethics. Without some basic principles, such as a commitment to honesty, a code of ethics isn’t saying anything at all. But a code that has no flexibility and tries to define the correct point of view on every subject won’t be useful or meaningful to the majority of people who supposedly use it. In chapter 6 we discuss each of these codes of ethics, how they came about, how one can employ them in one’s professional life, and the limitations inherent in any code of ethics. However, this book strives to give you more ways of approaching problems than professional guidelines alone.
Our goal in this textbook is to give you tools for thinking about dilemmas you may face as a computing professional. Ethical theory is included because it is one kind of tool. But rather than introduce you to only a handful of well-known ethical theories, we invite you to think with different ethical frameworks—frameworks that include important features of some ethical theories. These frameworks provide different resources for thinking about a variety of situations as they arise. By learning to describe problems using the principles and concepts provided by each framework, you will learn how to identify and articulate what is at stake and to which stakeholders in a broad range of ethical dilemmas. The various approaches to ethics presented in this book offer different ways to see the world, different vocabularies of significance, different axes along which to make distinctions between good and bad or right and wrong, and different conceptions of what it means to live a good life.
Moral philosophers sometimes invent new concepts, but some of their most important contributions have been rigorous accounts of the principles that people have already been using. In this book, we center discussions on several styles of ethical reasoning as philosophers have identified them: deontology, virtue ethics, communitarianism, utilitarianism, and some of the modern responses. By making these principles explicit and exploring them in depth, we can apply them to think about ethical dilemmas more methodically. Recognizing these different lines of thinking can shed light on disagreements—when two people talk past each other or reach different conclusions on what to do, it could result from ignorance or failure of reasoning, but perhaps the disagreement arises due to approaching the issue from fundamentally different premises.
This book introduces you to information and concepts that may be unfamiliar and gives you practice in using them, with the hope and expectation that you will continue to use them in your professional life. But continuing to make use of ethical concepts and insights requires a different kind of effort from the material for most other computing and technology courses. In fact, an ethics class is as practically useful as a programming class, and its usefulness is often complementary to the skills and tools you learn in other classes.
This course is also asking you to do something additional and more difficult in order to continue to apply what you will learn. If someone chooses to create or program a system to solve a problem and knows how to do it, there is little reason not to solve the problem in the most direct and efficient way possible. But ethical understanding requires an additional layer of commitment. It’s one thing to reach a judgment about the best course of action while sitting in a classroom. It’s another thing entirely to make that kind of choice in your own life and stick to it, instead of adopting an easier or more self-serving course. It’s easy to pay attention to ethical questions in an ethics course because that’s the whole point of being there. But when you’re at your job and worried about pleasing your boss or your client or keeping costs down, it can be easy to forget even to think about ethics at all and to fail to recognize a developing ethical problem that’s right in front of you.
In order to do the difficult thing of holding onto what you learn here, it may help to think of the course as a middle step, neither the beginning nor the ending of your development as an ethical thinker. It may be the first time that you sit down and think about ethics in a dedicated and systematic way, but it’s almost certainly not the first time you’ve thought about ethics. If you have ever worried about whether you are really a good person or wondered whether a given choice is making the world better, then you have already begun.
This book is here to help you continue your development as an ethical thinker and to give you some tools to continue growing once your encounter with it is over. It is a concentrated window of time in which to learn about your own mind and about the views and values of your current and future colleagues in the field. Succeeding at ethics doesn’t mean getting 100% on your ethics quiz; it means closing out your experience here with a clearer sense of what you value and an improved ability to articulate why you have these values and what it means to live up to them.
This book has two parts. Part 1 is a textbook in the traditional sense, and part 2 is an anthology of short science fiction stories that might be part of your ethics course, depending on how your instructor is teaching it. These stories have been selected because each of them raises vital issues in computer science or technology ethics in complex and illuminating ways. Each story is framed by a brief introduction that will help orient you to its core issues and a set of study questions you can use to provoke or guide your thinking about the story after you have read it. Throughout the textbook, you will find several story points, such as the one at the end of section 1.5, that direct you to particular stories in the anthology that explore the issues touched on in that part of the textbook.
1. What does it mean to say something is ethical? Give an example of a normative ethical assessment that you agree with and one that you do not. Give an example of a descriptive ethical statement.
2. Pick a piece of technology that you use every day. Make a list of the goals or values that you think the system is designed to achieve. For example: Does it connect people? Does it make something easier? Does it make something harder? Does it make something less expensive? Or does it make something impossible? What are the benefits? What are some of the potential drawbacks if the particular piece of technology achieves its goal?
3. Consider some ethical dilemmas that arise due to limited resources. Give an example of a resource that has become less limited due to advances in technology or computing, and give an example of a resource that has become more limited.
4. Think about some of the science fiction that you enjoy—pick one piece you are already familiar with and identify an ethical theme that appears in the work. Do all the characters in the story respond as you would? Pick one character that you feel embodies your “ideal” and one that does not. What can you learn from watching your ideal character? What can you learn from characters whose responses do not match the ones you would make?