Bayesian Intelligence from the Outside
A theory of Bayesian intelligence for agents such as language models is developed and the difficulty of aggregating coarse reports from intelligent agents is shown: unless the agent reports a belief about the complete state of the world, the optimal aggregation can assign arbitrary weights to states that have not been...