The perils of overestimating second-order effects

Overemphasising second-order effects can conceal motivated reasoning. This is bad.

Zeynep's law (which strikes me as closer to a philosophical razor than a law, but I digress) claims that in the absence of robust evidence, one can assume that counterintuitive findings are false, and second-order effects are dwarfed by first-order effects.

I want to zoom in on the part of this law razor that has to do with first-order effects dwarfing second-order effects. Put slightly more plainly, this means that the primary effect (first-order effect) of some action attempting to achieve a goal is often far more important for the purposes of achieving that goal than any knock-on effects (second-order effects) occurring afterwards.

The weak-form version of this razor — which I will focus this piece around — centres around the psychological phenomenon of overestimating second-order effects. This is a common blunder that leads people astray when evaluating cause-and-effect. It’s what happens when one supplants their common sense hat with their “let’s complicate things, social science style” hat. First-order effects are boring and obvious — surely we’ll get a richer explanation if we dig deeper, or so it goes.

Peter Wildeford @peterwildeford

Everyone should have a little @StefanFSchubert who sits on their shoulder and says things like "actually the obvious interpretation is correct"

Caveat: the purpose of this post is not to say that second-order effects don't exist, or that they can't be important. For example, some of the most interesting results in the social sciences arise when second-order effects are larger than we’d otherwise assume — like how rent control can actually decrease affordability in the long-run. But in the absence of evidence, it’s usually a good idea to default to the simplest explanation as a starting point when evaluating the causal impact of some action.

Let’s see with a few examples.

Sunscreen — does it reduce our odds of getting skin cancer?

For example, does wearing sunscreen reduce the likelihood of contracting skin cancer?

The intuitive answer is yes – after all, sunscreen is designed to protect us from the sun's harmful rays.

But perhaps sunscreen enables us to be complacent about other sun-safety measures. Maybe it makes us more likely to spend all day at the beach, soaking up the sun’s rays long after the sunscreen has worn off. If so, sunscreen could actually cause more skin cancer than the default!

Cash transfers — do they decrease poverty?

What about cash transfers to people living in extreme poverty — does it raise their incomes?

On one hand, duh. Cash transfers raise incomes by definition.

But maybe, just maybe, cash transfers make people lazy and handout-dependent, so they’re less likely to work hard and earn more money. Thus, cash transfers could actually increase poverty in the medium-to-long term! Oh no!

Grizzly bears — does avoiding them reduce our likelihood of getting mauled?

Alright, this one feels self explanatory.

First-order effects rule everything around me

Are second-order effects likely to be stronger than the intuitive first-order effects of applying sunscreen, receiving cash-transfers, and avoiding grizzly bears? Without any evidence, the answer should probably be no by default.

In some cases, claiming that second-order effects are far more important than first-order effects requires complicated explanations. These claims often rely on convoluted stories involving dubious chains of cause-and-effect — ones that can’t necessarily be ruled out a priori, but nonetheless rely on a web of assumptions that might not hold up in reality.

Worse, sometimes people use second-order effects as a way of demonstrating careful thinking, when they’re really just using motivated reasoning to justify one hypothesis over another. Every time I hear someone offer a shaky-at-best explanation for something that requires complicated second-order reasoning to justify, I wonder to myself what is actually motivating their intuition on this particular topic.

Person: “I wonder if sugary drinks are actually good for your health. Perhaps they improve your mood, which makes you more likely to do healthy things throughout your day, such as socialising with friends.”

Me, thinking to myself: hmmm, I wonder what this person’s actual underlying reason is for thinking sugary drinks are good. Perhaps they just really enjoy sugary drinks and would find them hard to give up.

Don’t use second-order reasoning to justify intuitive reasoning

Indeed, perhaps there are hard-to-articulate reasons or intuitions behind preferring one hypothesis over another. But rather than backing up these intuitions by vaguely gesturing toward convoluted second-order effects, it would help to explicitly clarify the actual core reasoning leading to these intuitions.

Eliezer Yudkowsky calls this core reason the “true rejection” — the real reason we disagree with something, which is not always the reason we offer as justification for disagreement. Suppose someone challenges your claim that unaligned AI is an existential risk, claiming you don’t know what you’re talking about because you don’t have a PhD in computer science. In this case, you should ask if they would buy your claim if you did in fact have a PhD. If they would still reject it, you have to dig deeper to find what their true rejection is.

Second-order reasoning can conceal true rejections. I sometimes hear people suggest, without offering any evidence, that abstaining from eating meat actually results in more animals getting killed, because crop production causes some wild animals to die. Therefore, if one cares about animals, one should eat meat.

This particular claim is easily refutable. But hearing it causes me to wonder, what is the actual crux behind this person’s aversion to going plant-based that is being concealed with second-order reasoning? Whatever this crux is, it’s far more interesting to dive deep into instead of the implausible second-order effects concealing it.

So, the next time you see someone engaging in what you think is second-order-fuelled-motivated-reasoning (catchier name pending), just ask them the following:

"Is there any evidence, other than a multi-layered hypothetical scenario, that these second-order effects are stronger than the obvious first-order effect?"

If they can't answer that with a clear and concise yes, you should default to assuming that the first-order effect is more important until new evidence arises.

I’d like to extend a special thank you to Stefan Schubert for his helpful feedback on my first draft of this post.