source: arxiv statistics ml: a hierarchy of policy learning problems
level: research
policy learning from observational data often aims to find rules that minimize regret compared to the best possible policy. but when data is limited, low regret may be out of reach. recent work has shifted to a simpler goal: learning a policy that statistically significantly outperforms a baseline. this paper argues that even that goal can be too demanding in some settings, and there is value in studying a wider range of policy learning problems.
the authors propose a mathematical framework that relates different policy learning objectives. they formalize three problems: detecting if any policy can improve over a baseline, identifying a policy that improves, and minimizing regret. these problems form a hierarchy based on the amount of data needed. the framework clarifies the relationships and trade-offs between these tasks, showing that some are strictly harder than others.
the hierarchy helps practitioners choose the right objective given their data constraints. for example, with very little data, one might only be able to test whether improvement is possible, not find the improving policy. the paper provides theoretical results on the sample complexity of each problem, offering guidance on when each goal is feasible. this structure can inform the design of future policy learning algorithms and evaluations.
why it matters: it helps data scientists pick realistic policy learning goals based on available data, avoiding wasted effort on impossible tasks.
source: arxiv statistics ml: a hierarchy of policy learning problems