Oolong: Evaluating Long Context Reasoning and Aggregation Capabilities
The data and evaluation harness for Oolong is released, a benchmark of long-context reasoning tasks that require analyzing individual chunks of text on an atomic level, and then aggregating these analyses to answer distributional questions.