Singling Out the Agent
by Bruce Waller
The usual requirement for being deserving in this sense is to be morally responsible. Moral responsibility is a particular sort of responsibility, that which justifies receiving one’s just deserts: praise and punishment are obligatory – deserved – because of the act alone, independently of their consequences. The qualifications for being morally responsible ordinarily include such things as acting sanely, consciously, on the basis of one’s own reasons and motives, and without being coerced or duped. All these are compatible with our being fully subject to natural causal laws, so are what philosophers call compatibilist requirements for moral responsibility. However, depending on who you ask, being morally responsible also requires that the agent have libertarian, contra-causal free will: that one’s actions aren’t fully explicable in terms of deterministic cause and effect but are the agent’s own doing as an ultimate originator, a first cause. But whatever the requirements, being morally responsible entails that one deserves praise and blame for good and bad behavior.
In his important new book, Against Moral Responsibility, Bruce Waller claims that although people generally meet the compatibilist requirements for being morally responsible, we nevertheless aren’t morally responsible; no one ever deserves praise or punishment. This of course is a shocking claim, contrary to commonsense, and to defend it Waller must take on the philosophical establishment, both the dominant compatibilists, who claim that being morally responsible is compatible with determinism, and the far fewer libertarians, who hold out hope for a human causal exceptionalism. Waller sets out to destroy, “root and branch,” what he calls the moral responsibility system, which he sees as being fundamentally incompatible with science-based naturalism, morally indefensible, and deeply destructive by leading us to ignore or discount the causes of human behavior and the often harmful outcomes of holding people morally responsible.
Waller’s case against the libertarians is straightforward and takes up relatively little space in his book: there’s no good naturalistic account of how human agents could be first causes, or self-caused, in the way libertarian philosophers (and perhaps many ordinary folk) think is necessary for being morally responsible. There’s no evidence for or logical coherence to contra-causal freedom; on a naturalistic view of ourselves, human agents can’t be the ultimate originators of their character and actions in the miraculous, god-like way that, Waller suggests, originally justified the idea of moral responsibility and just deserts.
Working within a science-based naturalism that accepts that human agents are likely fully caused phenomena with genetic and environmental antecedents, Waller concentrates his fire on the compatibilist apologists for moral responsibility, that is, the majority of philosophers concerned with human action. This makes his book very useful for those interested in the current debates about free will and moral responsibility; it addresses the central arguments of many of the major players, including Harry Frankfurt, Daniel Dennett, Al Mele, George Sher, and John Martin Fischer. None of their arguments, Waller believes, can justify the claim that wrongdoers should be punished for deontological reasons having nothing to do with the consequences of such punishment, the central demand of moral responsibility. And careful considerations of consequences will often suggest that punishment isn’t the best course of action in response to failure or wrongdoing. (I will focus mainly on blame and punishment in this review, but the points Waller makes apply to praise and rewards.)
Waller argues that to justify the notion of moral responsibility, compatibilists have to provide reasons that outweigh what he sees as the basic unfairness of (non-consequentially, deontologically, retributively) punishing agents who haven’t ultimately chosen their character and motives: “When we reflect deeply and carefully, we recognize that the behavior and characters of those we want to blame and punish were shaped by forces beyond their control, and it seems fundamentally unfair to punish people for their bad luck in being misshapen” (p. 94). The naturalistic causal story shows that none of us is self-made in the way that would make such punishment morally fair, and no other naturalistic considerations, Waller says, can make it fair. Moral responsibility is thus morally indefensible within a naturalistic framework. But of course many philosophers think otherwise. They adduce reasons that, they think, make agents deserving of punishment, that is, make it fair and just to punish someone simply because he has behaved badly. Waller points out that, paradoxically, the very breadth and diversity of these reasons suggest that the case for moral responsibility is weak.
What drives the compatibilist quest to justify moral responsibility, Waller suggests, is our strong, biologically-endowed retaliatory disposition to harm those who have harmed us or those we love, what he calls the strikeback response. As Waller puts it, “we feel those desires [to strike back] so deeply that we are certain that they must be justified” (p. 13, original emphasis). The strikeback response obviously had, and still has, naturally evolved functions, for instance to deter aggressors and maintain social cohesion (p. 135), but equally obviously it can be counterproductive when retributive emotions lead us to inflict suffering and death unnecessary for producing any social good. We can overreach in our zeal for punishment. By laying bare the psychological drivers of moral responsibility, Waller suggests that its justifications are simply rationalizations for acting on our retaliatory instincts. Seeing this, we’ll be less likely to act on them when they are counterproductive (which they often are, he argues), and be open to more enlightened and effective responses to wrongdoing.
The way Waller sees it, compatibilists are basically begging the question against moral responsibility; they take for granted the longstanding, culturally embedded presumption that of course agents are morally responsible and deserving of punishment. Their justifications therefore operate within the moral responsibility system, picking out characteristics of agents that supposedly make them morally responsible in contrast to agents that aren’t. As a result, skepticism about the system itself is in short supply. But if the presumption of moral responsibility is not taken for granted, Waller argues that it’s difficult to justify on any deeper grounds.
Despite the diversity of compatibilist arguments for moral responsibility (to which I will return shortly), there is nevertheless a basic theme or strategy uniting them that Waller tellingly exposes: they seek to highlight the agent as the primary causal player when explaining wrongdoing, while diverting attention from the agent’s antecedents and situation. This strategy involves focusing on particular characteristics of the agent (e.g., being rational, reasons-responsive, sane, and uncoerced), while downplaying her causal history and the influence of the situation and systems of which she is inevitably a part. This selective emphasis – singling out the agent – works to replicate, as far as naturalistically possible, the libertarian picture of persons as causally independent of their circumstances. Highlighting the agent, while obscuring (intentionally or not) her antecedents and situation, creates fertile psychological ground for generating attributions of agent-focused blame and the desire for punishment.
This is the inverse of the mitigation strategy routinely employed by defense attorneys: to highlight an offender’s history, especially any history of abuse (since that can generate sympathy), as an explanation of her character and actions. Assignments of causal responsibility, the first step in placing blame, quite properly track all contributing causes brought to our attention, including the causes of the person herself. Placing the offender in a wider causal context, whether abusive or not, makes it psychologically more difficult to place blame on her alone, thus damping the strikeback response.
Waller’s approach throughout the book is similar to the defense attorney’s: using a myriad of everyday examples, he counters compatibilist agent-centrism by drawing our attention to the actual causes of how persons become morally virtuous or deformed. This helps to motivate the judgment that they are not morally responsible (blameworthy) for who they have become, even though they remain moral agents, justifiably subject to social norms and moral evaluation. The wide view of the agent-in-context is what science-based naturalism affords us, which is why it’s more difficult under naturalism to place blame: singling out the agent requires a positive disregard of context, something antithetical to science.
So how do compatibilists single out the agent? In many ways, only a few of which I have space to consider here (see note 6 for more). One is to focus on the role of internal self-reflective processes in shaping our own character. This role is perfectly real, but Waller points out that having the capacity for (proximate) self-formation is not something we bear responsibility for, since we are bequeathed it (or not, or in whatever amount) as a function of factors we didn’t choose. Likewise, we didn’t choose the factors which set the direction in which we exercise self-formation, for instance toward developing a good or bad character.
Compatibilists spend little time with these facts, which of course makes self-formation and the resulting person the most salient causes of behavior; this in turn incites attributions of agent-focused moral responsibility. Although compatibilists have to admit that our self-forming capacities aren’t ultimately self-made, it isn’t exactly a selling point for their side, so they tend to play it down. Waller quotes Daniel Dennett in his book Elbow Room: “Instead of investigating, endlessly, in an attempt to discover whether or not a particular trait is of someone’s making—instead of trying to assay exactly to what degree a particular self is self-made—we simply hold people responsible for their conduct (within limits we take are not to examine too closely)” (Elbow Room, p. 164). Waller comments: “So don’t investigate how one’s character was shaped, and take care not to examine too closely how responsibility works. Dennett is right. If we want to keep the system of moral responsibility functioning, it is essential that we not look closely” (p. 111). To not look closely is to single out the agent.
Similarly, Waller quotes (p. 103) compatibilist Harry Frankfurt in his paper “Three concepts of free action”: “To the extent that a person identifies himself with the springs of his actions, he takes responsibility for those actions and acquires moral responsibility for them; moreover, the question of how the actions and his identifications with their springs is caused is irrelevant to the questions of whether he performs the actions freely or is morally responsible for performing them.” Waller accepts and indeed champions the importance of taking responsibility, as well as the self-reflective endorsement of one’s own character and actions, an aspect of what he believes is a very desirable personal authenticity. But he denies the irrelevance, for moral responsibility, of the causal history of the capacities and characteristics that allow us to take responsibility and be self-shapers:
Dennett and Frankfurt would have us forgo such tracing. Why? Because, by putting the agent in a causal context, tracing reduces the relative salience of a person’s characteristics and capacities, making it harder to pin blame solely on the agent, which is exactly what attributions of moral responsibility do. That an agent might even strongly identify with and endorse his bad traits – slavishness, selfishness, dishonesty, cruelty – is simply the result of a particular causal history, not grounds for placing blame on him for having become that way.
All this might strike you as mere excuse-making, letting people off the hook, but Waller is careful to point out that deconstructing the moral responsibility system isn’t a matter of what he calls “excuse-extentionism.” Compatibilists (and the rest of us) rightly distinguish between those who act sanely and of their own volition and those we excuse for various reasons: they didn’t have the requisite control or rationality, were coerced into behaving against their will, or acted in ignorance. This distinction is perfectly real, but in denying moral responsibility Waller isn’t claiming we’re all excused in these senses. After all, most of us most of the time act sanely and at the behest of our reasons, and although from a scientific standpoint we are just the most proximate cause of our actions, we remain full-fledged agents. The key, however, is to always consider persons in the context that shaped them:
Since individuals, their actions, and right and wrong don’t disappear on Waller’s account, this means that moral judgments are still applicable, as is holding each other responsible in ways that shape and maintain good behavior. But this doesn’t require us to use the moral responsibility system, which picks out agents alone as specially blameworthy or praiseworthy. Indeed, in rejecting the system, Waller points out we’re more likely to admit fault and take responsibility since we understand our short-comings as not ultimately our own doing, but the working out of a larger network of causes and circumstances. We need not hide from the severe, implacable judgment that we could have done otherwise, but simply failed to do so, since by putting the individual in context, that judgment is seen to be naturalistically untenable. We can therefore more easily own up to the real but non-ultimate role of our faults and virtues in explaining our behavior.
The defender of moral responsibility might respond: but don’t we have to hold people morally responsible as a practical matter? No, Waller replies, since to do so is to appeal to the possibly beneficial consequences of punishment, which is precisely what defenders of moral responsibility can’t do. The claim of moral responsibility is that we needn’t justify our response to an individual’s failure or wrongdoing by its behavior-guiding effect. Defenders of moral responsibility are committed to the non-consequentialist, deontological good of inflicting harm and suffering, so they can’t cite the future benefits of such suffering, e.g., deterrence, moral improvement, to justify it. The moral responsibility system is aimed at giving people what it’s supposed they deserve, not maximizing human well-being.
Indeed, Waller cites many instances in which moral responsibility practices are counterproductive from a humanitarian and practical standpoint, notably in how they stifle personal development (the luckily talented and motivated folks deserve the lion’s share of the rewards, the unlucky don’t, and so languish), encourage punitive excess in criminal justice (offenders deserve harsh treatment, not rehabilitative opportunities), and perpetuate social and economic inequalities (the rich and poor deserve their respective lots in life, so there’s nothing unfair about social and economic inequality). But since defenders of moral responsibility aren’t primarily interested in outcomes (at least when defending it), it’s no surprise they might not dwell on the downsides of allegiance to the moral responsibility system. Waller suggests that, if we abandon moral responsibility and take into account the entire systemic and situational story behind behavior, we’ll adopt more humane and effective interpersonal attitudes and approaches to education, criminal justice, and social policy. These will more likely enable human flourishing as progressive, humanistic naturalists conceive it, which involves minimizing the patent unfairness that luck deals out in life. Compatibilist defenders of moral responsibility are, perhaps inadvertently, working against this possibility.
Compatibilists believe it’s fair to punish (or reward) people for what Waller sees as essentially their good or bad luck in ending up good or bad, talented or mediocre. Some compatibilists like Dennett and George Sher have argued that luck evens out over the long haul, such that people genuinely, deeply deserve the fruits, sweet or bitter, of being just proximately self-made selves. But Waller points out this is false: those that (luckily) start out with genetic and/or environmental advantages in aptitude, motivation, education and opportunities often use them to further consolidate and amplify those very advantages, while the not so lucky are often hamstrung by their initial deficits and don’t ever catch up. The moral responsibility system therefore can operate as an inequality amplifier: it says that those who end up with the long or short end of the stick, morally or behaviorally or otherwise, deserve what’s coming to them, whether or not it helps them, hurts them, gives them unneeded benefits, further marginalizes them, or helps the culture to reduce or amplify future inequalities. Although sometimes necessary to guide behavior for the good, rewards and punishments, when divorced from consequentialist considerations, all too often serve to give the lucky further, often excessive advantages (think American executive compensation), and deprive the unlucky of even minimal opportunities to better themselves (think American jails and prisons). This seems morally indefensible.
What’s also indefensible, from a practical standpoint, is how the compatibilist focus on the person leads us to ignore what actually makes people good or bad. Moral responsibility effectively keeps us in the dark, for instance about the anguished question, after a murder, of how someone could do such a thing. The demand for just deserts almost always prevents an in depth analysis of the individual’s causal history, and what’s never admitted or allowed into testimony is the naturalistically obvious fact that killers are fully caused. On a wider stage, the same preoccupation with moral responsibility prevents adequate consideration of the social and economic determinants of the behavior of classes of persons (e.g., the underclass, the privileged few), which if addressed would go far toward reducing the probability of bad behavior in the first place. Waller on the Bernie Madoff scandal:
Apologists for moral responsibility give psychological support to the system by drawing our attention away from the full causal story and wider situation. They’d rather you didn’t focus on, or even admit, the fact that you, but for the luck of the genetic and environmental draw, would have been poor, criminal, or even, God forbid, a science-denying conservative. Waller cites philosopher Michael McKenna (p. 147): “The thought that we ought not blame does not seem compelling in these cases [such as Madoff] since it is hard to take seriously that we might have been like that—for we are now not anything like that.” But when we contemplate the fact that what are like now owes everything, ultimately, to circumstances not of our own choosing, then it’s impossible not to take seriously the fact that we might have ended up like Madoff, but for the conditions of our birth and upbringing. Contemplating this, we can’t neatly single out Madoff, or anyone, from their circumstances in the way which drives the psychology of desert that McKenna seeks to justify.
Can our innate predilection to reward and punish independently of considerations of consequences be justified on deeper, non-psychological, ethical grounds? To succeed in this would be to elevate the predilection into a moral principle, the normative good of moral responsibility. But Waller shows that by highlighting the agent’s rational, control and self-forming capacities, while obscuring their origins outside the agent, compatibilists merely incite the very propensity to blame which those capacities supposedly justify. They haven’t yet revealed, at least to my satisfaction, or Waller’s, a deeper moral principle that the propensity expresses.
The intuition of moral responsibility is given comfort by singling out agents, but threatened when we put them in context. If you think it’s fair and indeed obligatory to blame, punish, marginalize or otherwise make individuals suffer for their misconduct or shortcomings, it could be because you’ve not given due consideration to luck, to causation, to the history, situation and systems that fully explain (but not eliminate) the agent. What is it that makes it fair to punish the unlucky for their bad luck, whether or not it produces good outcomes? What is it that makes punishment obligatory? Answer that question convincingly and you’ll have won the day for retributivists, forcing naturalist progressives to concede that desert-entailing moral responsibility has a legitimate place in worldview naturalism. But as it stands, the question is very much open. In his thorough and persuasive critique of compatibilism, Waller shows why.
TWC - April, 2012
 Neil Levy defines moral responsibility similarly at the start of his interview at Philosophy Bites: “Well, as I use moral responsibility, it’s the property that makes agents appropriate targets of blame and praise and mabye even punishments and perhaps benefits. An agent is morally responsible if they performed an action and they deserve some kind of treatment on that basis, not on the basis of consequentialist considerations, not because it’s good for society or good for anybody else, but because they deserve it.” His host Nigel Warburton calls this “a fairly standard view of moral responsibility.” Tamler Sommers defines it this way – as essentially involving non-consequentialist desert – at the start of Relative Justice.
 Re the non-consequentialist essence of moral responsibility, John Martin Fischer says at the blog Flickers of Feedom, “The retributivist’s idea [is that] the (proportionate) suffering of wrong-doers is intrinsically valuable.” That is, such suffering is good independent of any good consequence it might have. And later: “By the way, I just saw the American remake of The Girl with the Dragon Tattoo... I really enjoy the moment when the girl's abuser gets what he deserves. I would even say: it is intrinsically good that he suffers in a way commensurate to the way he made the girl suffer. I think it is hard to argue for this kind of claim, but I do think retributivism taps into something important and deep about human beings.”
 About this, see Don’t forget about me: avoiding demoralization by determinism.
 In another attempt to deflect our attention from causation, compatibilists like Dennett say that the only sense of “could have done otherwise” that should concern us is counterfactual: that if I had wanted to do otherwise, I could have by virtue of having the capacity to do so. Dennett would rather you not contemplate the fact that in actual situations as they play out you could not have done otherwise, given that under naturalism you don’t have the capacity to transcend the cause and effect relations that constitute you and the situation in which you’re embedded. See http://www.naturalism.org/revolution.htm#chdo.
 This is the case, strangely enough, even though compatibilist requirements for moral responsibility such as reasons-responsiveness have to do with the capacity to be guided by the prospect of rewards and sanctions, a transparently consequentialist rationale for being held responsible, see http://www.naturalism.org/morse.htm#abandon.
 Some compatibilists have argued that satisfying retributive desires can be justified consequentially since their satisfaction affords benefits such as psychological relief, perhaps even pleasure. This raises the question of whether such desires should be satisfied, all things considered. Do we want to live in a culture which puts their satisfaction on an equal footing with possibly conflicting humanistic goals, such as producing a less punitive, less vengeful society which minimizes suffering? On this question, see Repressing revenge.
 Compatibilists will dispute this charge, saying that moral desert is essential to our well-being. For instance, giving people only what they deserve (no more, no less) is the necessary basis for proportionality in punishment, an essential check against consequentialist excess, for instance imposing draconian penalties for minor offenses. And only desert, they say, can supply the basis for treating each other as autonomous agents, as ends in ourselves, not mere objects of manipulation. But the existence of desert has to first be established before it can play such roles. Since it doesn’t seem to exist, consequentialists must offer alternative bases for proportionality and respecting human rights, and they can. Proportionality derives from what’s necessary for deterrence combined with the humanistic principle of minimizing harm, and our rights to autonomy are, for progressive naturalists, simply among the central values we seek to protect and maximize when considering consequences, see http://www.naturalism.org/morse.htm#autonomy.
 Billy Holiday’s classic ballad “God bless the child” is the perfect expression of how luck doesn’t even out. We could call this nature’s cruelest law, that of amplified inequality: “Them that's got shall get, them that's not shall lose…The strong gets more, the weak one’s fade…”. And who does God bless, according to Billy? Those that’s got (their own), as if they, not the weak, needed his blessing!
 A plug for Chris Mooney’s The Republican Brain: The Science of Why They Deny Science – and Reality.
 Waller (p. 9) quotes Dennett from a book chapter: “We ought to admit, up front, that one of our strongest unspoken motivations for upholding something close to the traditional concept of free will is our desire to see the world’s villains ‘get what they deserve.’ And surely they do deserve our condemnation, our criticism, and—when we have a sound system of laws in place— punishment. A world without punishment is not a world any of us would want to live in.” In a recent seminar on free will, Dennett admitted that the last sentence was an overstatement. This is progress, and with luck other retributivists will adjust their views, helping to rectify the scandal of compatibilism.
[back to Reviews]