Abstract:
Many views of moral agency include, implicitly or explicitly, a consciousness requirement—namely, the claim that phenomenal consciousness is a necessary condition of moral agency. This paper casts doubt on the consciousness requirement. I argue that consciousness is not necessary for instantiating four key capacities necessary for moral agency: action, moral concept possession, responsiveness to moral reasons, and moral understanding. I defend my picture of nonconscious moral agency as a plausible account of an entity that can act for moral reasons and can be morally responsible. Lastly, I discuss broader implications of my argument, especially on the possibility of artificial moral agency.

with Helena Ward & Rhys Southan

Abstract:
Those who worry about a superintelligent AI destroying humanity often appeal to the instrumental convergence thesis—the claim that even if we don’t know what a superintelligence’s ultimate goals will be, we can expect it to pursue various instrumental goals which are useful for achieving most ends. In this paper, we argue that one of these proposed goals is mistaken. We argue that instrumental goal preservation—the claim that a rational agent will tend to preserve its goals because that makes it better at achieving its goals—is false on the basis of the timing problem: an agent which abandons or otherwise changes its goal does not thereby fail to take a required means for achieving a goal it has. Our argument draws on the distinction between means-rationality (adopting suitable means to achieve an end) and ends-rationality (choosing one’s ends based on reasons). Because proponents of the instrumental convergence thesis are concerned with means-rationality, we argue, they cannot avoid the timing problem. After defending our argument against several objections, we conclude by considering the implications our argument has for the rest of the instrumental convergence thesis and for AI safety more generally.

with Paul Henne

Abstract:
While philosophers generally accept some version of the principle “ought” implies “can,” recent work in experimental philosophy and cognitive science provides evidence against a presupposition or a conceptual entailment from “ought” to “can.” Here, we review some of this evidence, its effect on particular formulations of the principle, and future directions for cognitive scientists and philosophers.

with Paul Henne, Vlad Chituc, Felipe De Brigard, & Walter Sinnott-Armstrong

Abstract:
Many philosophers claim that ‘ought’ implies ‘can’. In light of recent empirical evidence, however, some skeptics conclude that philosophers should stop assuming the principle unconditionally. Streumer, however, does not simply assume the principle’s truth; he provides arguments for it. In this article, we argue that his arguments fail to support the claim that ‘ought’ implies ‘can’.