Skip to content
Open access

Too Many DevEx Metrics, Too Little Guidance

Aug 2026 · Queue · Vol 24 · 0 citations · 5 references

TL;DR

DevEx Metrics Compass is a public, open-source web app built on a structured analysis of that landscape across more than 50 engineering organizations, and what the dataset reveals about how DevEx is measured today and where the gaps lie is shared.

Abstract

As AI-augmented development becomes standard practice, engineering leaders face mounting pressure to demonstrate its impact. Yet most organizations are measuring AI adoption and output while the effect on developer experience (DevEx) remains largely unknown. The right metrics can close that gap, surfacing the everyday friction developers face and monitoring the real impact of initiatives such as AI rollouts. But with more than 120 metrics in circulation across frameworks, companies, and scientific literature, identifying the right, meaningful, actionable ones for a specific context has become a challenge in itself. This article introduces DevEx Metrics Compass, a public, open-source web app built on a structured analysis of that landscape across more than 50 engineering organizations. Alongside the tool, we share what the dataset reveals about how DevEx is measured today and where the gaps lie. Whether you are new to DevEx measurement and looking for a first meaningful set of metrics, or a seasoned practitioner assessing whether an existing collection has the right breadth and depth, Compass helps teams navigate the fragmented landscape and identify measures that actually matter for their context and goals.

Read PDF

Similar papers

What does AI mean for Open Source?

The recent meteoric rise of LLMs (Large Language Models) and associated tools was largely unexpected and surprising to most. The rapid ascent of this technology has caught many software developers unawares, leaving them suddenly somewhat ignorant, and arguably under-skilled. LLMs, whilst still advancing, have recently...

Adam Retter · 0 citations
Open access Jul 2026

Actionable insights from developer behavior: a practical approach to software defect prediction

These findings validate cognitive theory for explainable, actionable, and interpretable safety-critical defect prediction, laying empirical groundwork to evaluate analogous issues in LLM-generated code through the behavioral study of AI.

Carlos Andrés Ramírez Cataño, Makoto Itoh · 0 citations
Review Jul 2026

How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Study

This paper is the first to study how SE processes are changing in the development of SE agents and what challenges developers face, and describes a seven-stage workflow and five process shifts, including a move toward evaluation-driven development.

Yunbo Lyu, David Williams, Jieke Shi et al. · 0 citations
Review Aug 2026

Characterizing the Quality Profile of AI-Generated C++ in Production

AI-generated C++ code has a distinct quality profile, showing higher rates of interface and coupling burdens, copy and allocation overheads, and a reliance on explicit loops over optimized standard APIs, which translates into tangible downstream costs, including increased review effort and a 5-8% increase in compute re...

Michael Tran, Fred Lewis, Kun Yang et al. · 1 citation
Review Sep 2025

A Grey Literature Review of AI-Native Applications

This study is the first to propose a dual-layered engineering blueprint for AI-native applications, revealing that AI-native applications are distinguished by two core pillars: the central role of AI as the system's intelligence paradigm and their inherently probabilistic, non-deterministic nature.

Lingli Cao, Shanshan Li, Ying Fan et al. · 1 citation
Open access Jul 2026

VulnScanner Pro: A Rule-Based Automated Web Application Vulnerability Assessment and Risk-Scoring System

VulnScanner Pro is a self-hosted assessment tool that tries to sit in the gap between the two extremes of manual penetration testing and automated security assessment, and recommends four recommendations for fixing the six missing HTTP security headers.

Athili Laxmi Lavanya, Mycharla Madhavkumar, D. Gangadhar · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.