source: arxiv machine learning: neural bayesian sequential routing
level: research
human decision-making is sequential and uncertainty-aware, but standard neural networks use static, dense computation with little insight into evidence gathering or when to stop. neural bayesian sequential routing (nbsr) addresses this by modeling inference as active evidence accumulation over a hierarchical directed acyclic graph (dag). within a dirichlet–categorical conjugate framework, neural experts query a persistent global knowledge oracle to extract positive evidence vectors. these vectors act as pseudo-counts, updating a dirichlet belief state through exact conjugate addition.
a gumbel-softmax straight-through estimator enables hard, path-dependent routing while maintaining surrogate gradients for end-to-end training. this allows the network to make discrete routing decisions based on accumulated evidence, mimicking sequential human reasoning. the dirichlet belief state provides natural measures of uncertainty through precision and entropy, which can be used to decide when to stop computation or to quantify confidence in predictions.
the framework integrates uncertainty quantification directly into the routing process, offering a principled way to balance computation and accuracy. by actively accumulating evidence and updating beliefs, nbsr can dynamically allocate resources, potentially reducing unnecessary computation while maintaining or improving performance. this approach is particularly relevant for tasks requiring adaptive inference, such as question answering or decision-making systems where computational budgets vary.
why it matters: this method could make neural networks more efficient and interpretable by letting them decide when to stop computing based on uncertainty, which is useful for resource-constrained ai applications.
source: arxiv machine learning: neural bayesian sequential routing