The Dangers of Key Performance Indicators (KPIs)

Does KPI stand for "Key Performance Indicator" or "Kinda Perverse Incentive?" • Leadership Land reader check-in #5

In late 2024, we published Why Inconsistency is Worse than Snorting Cocaine on the merits of consistency and how it builds habits. We then put theory into practice by launching the Leadership Land Consistency Challenge. Here’s how we did:

Orange dots are articles published during the Consistency Challenge.

• 11 planned articles, 13 published. The two unplanned articles (Thanksgiving and reverse psychology) have aged like milk. Within two days of Thanksgiving, we had to recant and rewrite a third of the article. In the reverse psychology article, we fell for the same trap we warned against in 3-D and 4-D thinking – we created a cogent narrative to retroactively explain a Black Swan event. Sure, the narrative makes sense…but is it true?

• Duration: 99 days planned, 106 elapsed. The Challenge was originally supposed to run from 11/22/24 until 2/28/25. We extended the end date by one week to accommodate the second unplanned article.

• 39,962 words written | 377 words per day | 3,074 words per article. For reference, once a novella’s word count balloons to 40,000-50,000, people start calling it a novel. Over the past three months, we wrote the equivalent of a long novella.

Here’s another graphical representation of our writing, this time measuring word count:

Orange spikes are articles published during the Consistency Challenge.

If we were professional writers in a bureaucratic organization, we could proudly proclaim that we “exceeded expectations” or “surpassed key performance indicators” or “demonstrated a proven track record of consistent content production while upholding high quality standards.” You know, the kind of chest-pounding buzzword-y verbiage that would:

• Get us promoted

• Squeeze a pay raise out of the boss during the next performance evaluation

• Polish our résumés in case the boss refuses.

We could, but we won’t. Resting on our laurels leads to complacency, and complacency increases our vulnerability (blindness) to Black Swan events – see Preventing Career Failure (Explained with 2nd-Grade Math). We’re also fighting the “self-serving bias” – that is, the temptation to attribute success to internal factors (e.g. talent, skills, testosterrific ambition), while attributing failure to external factors (e.g. bad luck, bad boss, bad economic conditions).

Furthermore, taking a victory lap would distract us from asking the really important questions: did our readers actually sign up to get a serialized novella dumped into their inboxes every week? Did we actually deliver what our readers wanted? Did we “win” a Pyrrhic victory?

---

All That Glitters is Not Gold; All That Is Gold Does Not Glitter

You were probably taught in B-school or on-the-job training that “KPI” stands for key performance indicator. Since “leadership success” is a fuzzy concept and few people agree on what it is, we must use KPIs to measure success by proxy. Some KPIs are more suitable (measurable) than others:

• A business’s profits, the conversion rate for an advertising campaign, or the number of widgets shipped to Nonexistonia last month: these are crisp, measurable, and spreadsheet-able KPIs.

• Employee engagement, enemy morale, and disasters prevented: these are squishy metrics that are unsuitable as KPIs because they cannot be accurately measured¹.

Note that suitable KPIs aren’t necessarily important, and unsuitable KPIs aren’t necessarily unimportant. Here’s yet another 2 × 2 matrix to illustrate the distinction:

“You’re telling me that checking the box for one teambuilding exercise per quarter didn’t improve trust?”

Confusing suitability with importance often causes a KPI to send you straight into the Desert of Good Intentions. For example:

1. You can’t measure the disasters you averted

Sometimes, a leader’s greatest contribution is to protect their organization from calamity. “Number of disasters prevented” will never be a suitable metric for success (you can’t measure anti-knowledge), even though it’s extremely important. A fragile organization that hyper-fixates on KPIs (i.e. visible, measurable success) usually discovers this later, in the worst possible way.

A famous example comes from around 217 BC, during the Second Punic War. The Roman commander Fabius chose to avoid direct confrontation with his adversary, the brilliant Carthaginian commander Hannibal. The guerilla tactics used by Fabius were effective but unpopular, earning him the nickname Cunctator (“delayer” or “procrastinator”). The Romans grew tired of Hannibal’s marauding army hanging around in their backyard, so they eventually demanded a decisive victory (the ancient Latin equivalent of a KPI). They removed Fabius from power and appointed another commander, who promptly led the Roman army into one of the worst debacles in Roman military history at the Battle of Cannae.

2. “Key Performance Indicators” vs “Kinda Perverse Incentives”

There’s a trade-off in focusing on a key performance indicator – it implies that other performance indicators are less important. If the key performance indicator conflicts with a “non-key” performance indicator, the less-important metric becomes collateral damage.

Likewise, fuzzy metrics tend to become the collateral damage of a KPI that’s more suitable (measurable) but not necessarily more important. For example:

• Widget quality. We met the KPI of shipping a bazillion widgets to Nonexistonia last month, but we had to find another supplier notorious for their quality control issues.

• Goodwill. We met the KPI of hitting a 10% conversion rate on the latest advertising campaign, but we had to advertise on disreputable platforms that might sully the brand’s image by mere association.

• Long-term vitality. We met this quarter’s KPI of a gajillion dollars in profit, but we had to cut R&D expenses, fire a bunch of specialists (thus losing their institutional memory), and perform some financial engineering on the books.

Leaders can (and should) set goals for their organization. However, we must be extra cautious not to allow a key performance indicator to turn into a different kind of KPI: a Kinda Perverse Incentive. This is the evil doppelgänger of the key performance indicator. The Kinda Perverse Incentive produces measurable results that impinge on less-measurable-but-still-important factors, like customer satisfaction, brand equity, organizational culture.

Boards and executives who set Kinda Perverse Incentives will be unpleasantly surprised when the organization succeeds in all the wrong ways. When the collateral damage becomes worse than the benefit of meeting a goal, the evil doppelgänger KPI has lured them into the Desert of Good Intentions.

3. Mistaking the map for the territory

Even if you avoid setting Kinda Perverse Incentives for your subordinates, you can still fool yourself by fixating on legitimate key performance indicators. When you spend your days focused on weekly/monthly/quarterly goals, it’s easy to mistake the map for the territory. That is, to believe “success = meeting KPIs.”

Let’s say you’re in the Middle Management Foothills and you prepare a weekly operational report for the Senior Vice President on Executive Mountain. At the end of the year, you can check “prepared 52 weekly reports” off your KPI list and declare victory, right? But what if no one actually reads your reports? What if the SVP who asked for your weekly reports has retired, and the new SVP has been ignoring your reports for the past nine months? Was it worth your time, and worth distracting all your staff, to prepare the report?

Goodhart’s Law

In its simplest form, the law states:

When a measure becomes a target, it ceases to be a good measure.

There are other adages that convey the same spirit (e.g. Campbell’s Law), but we prefer Goodhart’s Law because its name is a powerful mnemonic device. Goodhart’s Law afflicts people with good hearts – those who seek organizational success but end up chasing their KPIs into the Desert of Good Intentions.

---

Leadership Land Consistency Challenge: Our Collateral Damage

All that stuff about KPIs sounds good in theory. In practice, however, distinguishing between key performance indicators and Kinda Perverse Incentives is much more difficult. Let’s examine the collateral damage of the KPIs that we set for ourselves at the beginning of the Leadership Land Consistency Challenge.

Word count: key performance indicator or Kinda Perverse Incentive?

When we launched the Consistency Challenge, we committed to a single KPI: write one article every week for ten weeks. This was a suitable KPI and important for the goal of establishing better writing habits.

But what counts as an article? Is there a minimum word count? If we repeated “AI can’t tell you a secret” 50 times in a row, would that count as an article? What if we asked an AI to generate between 1,000 to 1,500 words on how it can’t tell us secrets?

We never committed to word count as a KPI, but we did unconsciously decide to produce more hand-written (not AI-generated) content. That seemed to align with the goal of building consistent writing habits…but it led to looooong walls of texts dropped into readers’ inboxes every week.

This is a personal flaw. To recycle a joke from Risk and Uncertainty in Leadership: 1-Dimensional Thinking, we’ve been putting the “suck” in “succinct” since we started writing.

For us, a minimum word count is a Kinda Perverse Incentive – an invitation to write too much! A better KPI for us would be a maximum word count that forces us to express ourselves more succinctly.

Quantity over quality

We picked an immensely difficult topic – risk, uncertainty, and epistemology applied to leadership – to cover for the Consistency Challenge. We feel like we compromised on quality to meet the “10 articles in 10 weeks” KPI. Looking back, some of the glaring inconsistencies/deficiencies include:

• Advocating for secret-seeking in How to Discover Secrets in Leadership Land (particularly in the middle of Part 2), then writing later that positive Black Swan events (like the discovery of secrets) are unpredictable. That naturally raises the question “How are you supposed to find Secrets and Inconceivabilia (the ‘unknown unknowns’) if you don’t know what you’re looking for?”

• We rushed the end of Risk and Uncertainty in Leadership: 3-Dimensional Thinking. The final section was extremely short because we ran out of steam a few hours before the deadline.

• The “butterfly effect” from Risk and Uncertainty in Leadership: 4-Dimensional Thinking was originally meant to illustrate the difficulty of predicting far into the future, but we hijacked the idea to show that it’s easier to think forward rather than backward. We don’t think our re-interpretation is wrong, or that it’s mutually exclusive with the original idea; we’re concerned it might throw off readers who are familiar with idea in its original form.

• In How to Discover Secrets in Leadership Land, Part 2, we wrote “read more history, biographies, and old books.” Then later in the article series, we made a big deal about how we can’t accurately post-dict the past. This section in the final article resolves the contradiction, but it would’ve been better if the two sections referenced each other about the devil in the details.

We feel like we Goodhart’ed ourselves a bit with the weekly time limit, but maybe we’re nitpicking.

Caffeine Collateral Damage

One unintentional side effect of the Consistency Challenge is that one of us (the primary author) has become addicted to caffeine…again. When I wrote about consistency and habit-building, a caffeine habit was not what I had in mind!

Like any addiction, caffeine dependency is a fragile state of existence. If I miss a dose (sometimes happens when I’m traveling), I’ll suffer from withdrawal headaches and mental murkiness for a day or two. Now that the Consistency Challenge is over, it’s time for me to go through caffeine rehab…again.

---

Were the Trade-offs Worth It?

If every decision had a clean and easy answer, leadership would be a cakewalk. Low-hanging fruit and no-brainers are rare treats. For everything else, some level of collateral damage is unavoidable.

The important question is “was meeting the KPI worth the collateral damage?”

The only way to measure the success of the Leadership Land Consistency Challenge is by listening to reader feedback. How did we do? How can we better serve your needs? Did our meeting the KPI actually benefit you?

Let’s start with a poll about quantity:

---

Next, a question about quality:

Multiple choice answers don’t afford much room for self-expression, so please share any additional thoughts in the comments below ↓. If you’re reading the newsletter, you can reach us by replying directly to the email. One reader has already told us that the weekly serialized novella was excessive – please share your opinion, too!

---

For the time being, we’ll return to an erratic publishing schedule. It might take a few weeks to get back to caffeine sobriety, and I might be duller than usual until I detox my brain of the addiction…

1. You can try to measure them with surveys and spies, but your measurements will be distorted by sampling bias, the Taboo Tunnels, or Liar’s Lair.