source: arxiv artificial intelligence: internal pluralism and the limits of pairwise comparisons
level: research
local pairwise comparisons are often used to learn how people want automated decision rules to behave, for example in participatory design or ai alignment. these methods assume that local comparisons provide enough evidence about a person's preferences and that people can always answer them decisively. this paper challenges those assumptions by introducing internal pluralism, the idea that individuals evaluate decision rules using multiple authoritative priorities that may conflict.
the authors present a formal model of pluralistic preferences over decision rules. this model reveals two distinct failures of forced local pairwise comparison data. first, priorities such as proportionality, egalitarianism, and equal treatment are inherently global: what they imply in one case can depend on what happens elsewhere, so local comparisons may not capture the full picture. second, when multiple priorities are present, a person may not have a decisive preference between two options locally, even if they have a clear global preference.
the findings suggest that relying solely on local pairwise comparisons can lead to decision rules that do not reflect people's true pluralistic preferences. this has implications for designing fair and aligned ai systems, as it highlights the need for methods that can handle global priorities and indecisiveness. the work encourages exploring alternative elicitation approaches that respect the complexity of human values.
why it matters: understanding the limits of pairwise comparisons helps build ai systems that better align with human values, especially when fairness and multiple priorities are involved.
source: arxiv artificial intelligence: internal pluralism and the limits of pairwise comparisons