A long time ago I had a spreadsheet that was near and dear to me.
It was my commission calculator.
I’d plug in how much I thought I could sell, and it would spit out the size of my future commission checks. My favorite time-killer was plugging in bigger and bigger numbers and watching my fantasy paychecks grow.
I didn’t know it at the time, but focusing on those dollars and cents like that wasn’t good for me…
It was making me less likely to be involved with others.
It was making me less likely to depend on others.
It was making me less likely to accept demands from others.
So says researcher Kathleen Voss. She claims my financial daydreaming was doing these things behind my back - subconsciously. And Daniel Kahneman, Nobel Prize Winning psychologist, has made Voss’s work famous by featuring it in Chapter 4 of his book Thinking Fast and Slow.
But here’s my problem. Even though I’m a huge Kahneman fan, I think he’s wrong on this one.
I think Voss’s work is bullshit.
It’s not how I remember things at all. I know she claims this all happens subconsciously - so I shouldn’t remember it. But hear me out.
I'm convinced that my financial spreadsheet-gazing had the exact opposite effect on me. I used it…
to drive myself to have more contact with customers, partners, and employees.
to remind myself to listen and share more with our team.
to remind myself how dependent my success was on others.
Bottom line, that greedy spreadsheet was a positive part of my routine during the most connected period of my career. And while I believe my great connection was primarily driven by cultural FIT, the spreadsheet sure didn’t hurt anything. It was a pleasant distraction from the stress of a startup, and a regular reminder of what we could achieve if we all got, and stayed, on the same page.
I always tell you that if research doesn’t pass your smell test you should dig in. So I followed my own advice. And it didn’t take long to find that Voss’s work had a shaky foundation.
Here, in a February 2, 2017 blog post from a site called replicationindex.com, is the most damning information I found.
The post points out that folks have been unable to replicate Voss’s work. Which, you’ll recall from these posts (#57 - Fake News In Your Presentation, #57a - Fake News Supplement), is a huge deal. If your work can’t be replicated it’s more opinion than fact.
And, I found something even more interesting. If you read the comments beneath the blog post you’ll find one dated February 14th from Daniel Kahneman himself. He says...
“I accept the basic conclusions of this blog…What the blog gets right is that I placed too much faith in underpowered studies…This was simply an error: I knew all I needed to know to moderate my enthusiasm for the surprising and elegant findings that I cited, but I did not think it through.”
So, my hero, the number one bullshit-caller in all of psychology fell for bullshit. Even he was lured in by a sexy sounding study that hadn't been replicated. So what does that suggest for mere mortals like you and I?
Think twice before littering folks inboxes with research-based claims that haven’t been confirmed.
Hesitate before handing your management team the latest whiz bang research-based business book.
It literally took me three minutes, once I decided to look under the covers, to find that the Voss study was weak. And I get that you’re busy and it's not always worth the effort.
Agreed. But don't get lazy. There are times, when you're relying on some random study to backstop an important decision or argument, where you must dig in.
This is why I try to point out when I haven’t gone deep on something. And why I suggest that if you see something in my writing that you like, that you confirm it yourself. Don’t be lazy like I almost was with the Voss work.
One last random point. Hat tip to Daniel Kahneman for admitting his error. I doubt most Nobel Prize winners are spending their time admitting errors to random bloggers. His lack of ego in this case is inspiring.
(This site is all about building a Map that will help me do work and life better. So at the end of each post I check in to see if any changes / insights come to mind.)
This is the ultimate Wrong Story post. It’s about a Wrong Story - that our brains subconsciously lead us down a wrong path when we think about money - that itself turned out to be a Wrong Story. So it’s Wrong Story squared.
A SUMMER SERIES
Daniel Kahneman’s book - Thinking Fast and Slow - is my favorite Wrong Stories book. He does a great job of explaining how we come to wrong or biased conclusions. And that’s why I’m doing this series of posts covering the topics that Kahneman writes about. Here’s a list of the prior posts in the series.
I hope you enjoy them.