Compute Overlap Stall (COS): Predicting Performance of Power Management for Shared Memory Codes When Throttling Processors, Memory, and Thread Concurrency

Abstract

Maximizing performance under power constraints is a priority for highly parallel scientific applications. Modern systems offer throttling control over operating modes, including CPU clock speed (DVFS), memory speed (DMT), and concurrency level (DCT). Throttling speed reduces energy consumption at the cost of possible performance loss. Accurate execution time prediction mechanisms are useful for choosing system configurations that yield workload efficiency. The Compute Overlap Stall (COS) model predicts the execution time of homogeneous, bulk-synchronized parallel applications across simultaneous changes of these operating modes. The key insight of the model is that pure compute time, pure stall time, and compute-memory overlap are discretely affected by these three operating modes. We validate and update the COS model with an emergent architecture and reduce the size of the training set with negligible loss in prediction accuracy. We employ the optimized COS model on two architectures for 14 application benchmarks. We observe Good (<inline-formula><tex-math notation="LaTeX">$< 5\%$</tex-math><alternatives><mml:math><mml:mrow><mml:mo><</mml:mo><mml:mn>5</mml:mn><mml:mo>%</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="mccoy-ieq1-3712610.gif"/></alternatives></inline-formula> error) or Fair (<inline-formula><tex-math notation="LaTeX">$< 10\%$</tex-math><alternatives><mml:math><mml:mrow><mml:mo><</mml:mo><mml:mn>10</mml:mn><mml:mo>%</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="mccoy-ieq2-3712610.gif"/></alternatives></inline-formula> error) prediction accuracy for 11 out of 14 applications.