Overview

The article argues that traditional Application Security Key Performance Indicators (KPIs) often undermine program success and product delivery. As noted, “You can’t manage what you can’t measure,” yet choosing the wrong metrics can sabotage security initiatives.

The “Whack-a-Mole” Metrics

These counterproductive indicators focus solely on vulnerability identification and remediation:

  1. Vulnerability Counts - Total vulnerabilities discovered, treating finding more issues as program success
  2. Remediation Rate - Speed of fixing vulnerabilities, emphasizing rapid clearing of findings
  3. Vulnerability Types Tested - Variety of detectable vulnerability classes, attempting to expand testing scope gradually
  4. Number of Products Onboarded - Quantity of applications monitored by the program

The Problem: These metrics reinforce misconceptions that application security delays product development rather than enabling it.

The “Sherlock Holmes” Metrics

These attempt correlation-based evaluation:

  1. Percentage of Code Under Test - Coverage of codebase by security tests
  2. Exemption Rates - Frequency of vulnerability exemption requests
  3. Pass-Through Rates - New vulnerabilities escaping detection and reaching production

The Limitation: Despite appearing to shift focus beyond vulnerabilities, these metrics remain dependent on underlying vulnerability statistics.

The “Security Investment” Metrics

These provide genuine program value assessment:

  1. Total Cost of Production (TCP) - Overall product management and maintenance expenses
  2. Downtime Cost - Expenses related to application unavailability
  3. Cost to Address New Vulnerabilities - Expenses for fixing zero-day vulnerabilities

Core Insight

A more secure application doesn’t require increased development costs—it reduces maintenance expenses. Software vulnerabilities typically result from gaps in the development process rather than insufficient resources.