source: arxiv artificial intelligence: bounded morality: defining the space of moral computation
level: research
moral cognition is often modeled as following fixed ethical theories like deontology or consequentialism, treated as static rules. this paper proposes bounded morality, a formal framework that analyzes the computational demands of moral problems for finite agents. it extends herbert simon's concept of bounded rationality by defining moral situations along two dimensions: moral breadth, the range of entities considered morally relevant, and moral depth, the inferential integration needed to evaluate their interactions.
limited computational resources force a tradeoff between moral breadth and depth, creating a feasible space of moral computation. within this space, ethical theories are not competing accounts of moral truth but locally efficient strategies adapted to different demand regimes. the framework introduces formal notions of moral regret and moral progress under resource constraints, showing how agents can optimize their moral reasoning given their limitations.
the approach shifts the focus from which ethical theory is correct to how agents can make the best moral decisions with finite cognitive resources. by quantifying the tradeoffs, it provides a way to compare moral strategies and understand the conditions under which different ethical approaches are computationally viable. this has implications for designing ai systems that must make moral decisions in complex, real-world environments.
why it matters: this framework helps ai researchers design morally competent systems by explicitly modeling the computational tradeoffs in ethical reasoning, moving beyond rigid rule-based approaches.
source: arxiv artificial intelligence: bounded morality: defining the space of moral computation