Bias As Superpower: Using Your Cognitive Bias To Become Smarter

Anyone who studies psychology and looks to improve their thinking has come across cognitive biases: our tendencies to think in ways that deviate from rationality or good judgment.  It’s worthwhile to peruse them and understand them, but it would be a mistake to claim that we have to avoid them completely in order to be a rational person.

As an example of what I mean, I want to explore the difference between learning and confirmation bias, and whether there is in fact any difference.  When you look at the processes involved in each, as I’m about to outline, it’s clear that there is some strong similarity between the two.  Looked at this way, I think there are some interesting takeaways for anyone interested in constant learning and self-improvement.

What is Learning?

There are a lot of different models of the learning process, but let’s take a simple one.  In this version, teaching and learning involve a process of advancing one’s state of knowledge through the mastery of small topics (or lessons) that get successively harder or more complex.

Take math skills as an example.  Students generally progress from counting and simple arithmetic to functions and algebra, then basic geometry, before finally moving to more complex formulas, trigonometry and calculus.

If you were a tutor responsible for ushering a student along that path, your teaching process would look something like this:

  1. Assess the student’s current state of knowledge
  2. Give lessons and assignments that are just within grasp of their current state of knowledge
  3. Repeat

A good tutor will devise every lesson as in a sweet spot of not too close yet not too far from the student’s state of knowledge (i.e. proceed from 8th grade algebra to 9th grade geometry, not straight to college level calculus).  Too close and the student is not learning fast enough, too far and they will struggle to keep up.

Screen Shot 2017-02-05 at 9.32.58 AM.png

In this way, each lesson builds on prior knowledge in as efficient way as possible.  Each lesson should confirm and incrementally advance the student’s state of knowledge.  I argue that this doesn’t read much different from the standard description of confirmation bias.

Confirmation Bias

Let’s examine a few of the common definitions of confirmation bias:

  • “A confirmation bias is a type of cognitive bias that involves favoring information that confirms previously existing beliefs or biases.” [link]
  • “[Confirmation bias] is the tendency to search for, interpret, favor, and recall information in a way that confirms one’s preexisting beliefs or hypotheses, while giving disproportionately less consideration to alternative possibilities.” [link]

A classic example would be someone who believes in astrology – that the positions of stars and planets have predictive bearing on earthly events.  If this person is exhibiting confirmation bias, they will interpret any correlation between an astrological event and a borne-out prediction as evidence that astrology works, while avoiding possible conclusions to the contrary.

Screen Shot 2017-02-04 at 10.06.04 AM.png

Putting Them Together

Compare those definitions of confirmation bias to our prior statement about efficient tutoring / learning:

  • Tutoring and learning involve the presentation of information in such a manner that confirms and extends the students’ existing knowledge in as efficient way as possible.

See the difference?  No?

With such a close definition, it’s actually hard to distinguish when you are really doing one or the other:

Screen Shot 2017-02-04 at 12.49.16 PM.png

In some sense, when you’re looking to make good headway in a new subject area, you want there to be some confirmation bias.  Otherwise you’re not learning in an integrative way, you have random facts disconnected by miles of ignorance, and your payoffs to learning might be too far out to be immediately useful.

The Sweet Spot

We can map any point in that Venn diagram to one’s “slope of learning” as they try to progress in a subject area or grasp a new piece of information.  We can base this slope on the subjective rating by the individual for how easy or hard it is to understand the new piece of knowledge.

We’ll use a ski slope metaphor for how cognitively burdensome it is to integrate a new piece of knowledge or fact:

Screen Shot 2017-02-04 at 2.47.15 PM.png

If you stick to the green Hard Going slope you will make slow progress, but the difficulty may help reassure you that you’re not just combing over territory you’ve already covered.  If you stick to the Easy Going path, you may move a lot more quickly towards tangible intellectual progress, but it is likely to be incremental at best.

Confirmation bias is still learning, it’s just learning in a way that most economizes cognitive effort.

So as you learn new information you should ask yourself: how cognitively challenging was it to integrate that new piece of knowledge?  What slope do I perceive myself as on?  You can use this as a litmus test for how likely it is that you’re doing real hard learning, or simple confirmation.

Knowledge Economies of Scale

There’s another paradox that comes after this process picks up steam.  Take someone like Tyler Cowen.  Wikipedia identifies him as an economist, but as his interviews and blog make clear, his knowledge is much broader than that, covering a wide field in areas like sociology, psychology, and global culture.

We might imagine that his personal “Knowledge Cloud,” inside the space of all possible knowledge, looks something like this:

Screen Shot 2017-02-05 at 9.03.53 AM.png

 

Let’s pretend there are economies of scale at work in knowledge acquisition.  This means that the more knowledge you have in your initial repertoire, the easier it is to bring in knowledge that directly neighbors it.

In the case of some new idea at the crossroads of sociology and economics, Tyler Cowen’s effort to grasp that idea might look like this:

Screen Shot 2017-02-05 at 9.04.05 AM.png

Now take someone else with a very different background, like Elon Musk.  Elon’s knowledge cloud, and his effort required to grasp the same concept, might look like this:

Screen Shot 2017-02-05 at 9.06.06 AM.png

 

Now we have a scenario where the “ski slopes” for each individual differs greatly, for that particular new idea:

Screen Shot 2017-02-05 at 8.57.48 AM.png

Tyler’s slope is much closer to what we’d be tempted to call confirmation bias.  But is it still confirmation bias?  Is it a bad thing?  Can we accuse domain experts that publish and adopt research at the cutting edge of their field as simply engaging in confirmation bias?

Bias Bad, no Bias Good

I don’t pretend to have the answers here, though I think this is a much richer description of what we mean when we talk about confirmation bias.  Confirmation bias is always at or near the top of any list of cognitive biases, and it makes sense why this is so.  We are learning all the time and we’ll mostly gravitate towards the easy slope, towards learning with the least effort or in the most economical way.

If people continually economize their learning, they will build models solely based on what’s the least cognitively burdensome knowledge for them to acquire.  That’s a big way to miss out on other important models, and why we should recognize confirmation bias as a potential pitfall.

But if we take the simple approach that says “I shall not commit any of the biases on this list,” we are misunderstanding what that list is telling us.  Behind every “bias” is some truth to the way we operate, some core insight about how we evolved as intelligent beings.  It would be a shame to turn our backs on thousands of years of evolution just because we have a fetish for what’s supposedly rational.  Instead of fighting ourselves and seeking to rid oneself of biases, you might try to take the idea a little deeper and use it to work with, instead of against, your brain.

Advertisements

What I read in 2016

2016 was an excellent year for reading.  IMG_20161206_164236.jpg

All of the books were great.  Below are some brief summaries.

Unique and worthwhile: (these titles are excellent and don’t appear on many other lists)

Peripheral Visions, by Mary Catherine Bateson

A really great book by the daughter of the famous anthropologist, Gregory Bateson.  She’s a brilliant researcher and writer in her own right, and in this work she examines many of the cultural norms across the world around learning, tolerance, and perspective.  A few choice quotes:

“We reach for knowledge as an instrument of power, not as an instrument of delight, yet the preoccupation with power ultimately serves ignorance.  The political scientist Karl Deutsch define power as ‘the ability not to have to learn,’ which is exemplified by the failure of empathy in a Marie Antoinette or the rejection of computer literacy by an executive.  Ironically, in our society both the strongest, those who have already succeeded, and the weakest, those who feel destined for failure, defend themselves against new learning.”

“The pitfall of fundamentalism… is that when some item is held constant while the context varies, constancy is an illusion, and those who resist change often suffer themselves.”

“Sometimes you meet people who have learned their way around the culture of the ‘other’ well enough to have access to a second way of seeing the world.  They then have a unique capacity to pick and choose among behaviors and assumptions that would otherwise have remained unquestioned, and even to invent new ones.”

The Sign of Three, edited by Umberto Eco and Thomas Sebeok

I have a private obsession with semiotics and in particular, Charles Sanders Peirce, one of its founders. This book is a collection of somewhat academic essays by scholars about the practice of abductive reasoning, something Peirce defined.  Abductive reasoning is different from deductive or inductive reasoning, in that you’re required to make a leap from observations/data to a theory, with no guarantee that your hypothesis is correct.  It’s the real process behind Sherlock Holmes’ methods, or any good hypothesis in the Scientific Method.

The discussions in the book range from detecting art forgeries, the methods of Sherlock Holmes and Dupin (the detective in Poe’s Murder in the Rue Morgue), Voltaire’s Zadig, and of course, Peirce himself.  Much of it was used as inspiration for my post on Ribbonfarm.

Charles Sanders Peirce, by Joseph Brent

Peirce lived something of a tragic life.  He was an early prodigy but had habits and a communicative style that put him at odds with many of his contemporaries, who often worked to block him from academic posts or opportunities.  Thus, much of his work was left to relative obscurity, even though he was a very modern thinker with idease that are at the core of present day probability theory, psychology, physics, mathematics and logic.

The Systems Bible, by John Gall

Though it’s a bit quirky, this is a great book and summary of practical advice when dealing with systems of all kinds.

Great: (these are a little more esoteric, while still good)

Reasoning and the Logic of Things, by Charles Sanders Peirce

One of the few accessible compilations of Peirce’s work.

Everything and More, by David Foster Wallace

A fantastic guide to modern mathematics, tracing the story of infinity throughout the ages.

Thinking and Deciding, by Jonathan Baron

Worth pushing through.

The House of Morgan, by Ron Chernow

Awesome history of global finance.  Chernow picked the perfect thread to follow in order to give maximum resolution to the big trends and shifts.

Charles Schwab, by John Kador

Good read on the history of Charles Schwab, the brokerage firm.  Almost all of their struggles (capital needs, regulatory pressure, technology changes, etc.) rhyme with what the current fintech space struggles with.  It can read at times like it came from the Schwab marketing department, but Kador pulls no punches when it comes to pointing out flaws in Schwab’s culture or decision making through its history.

Well covered elsewhere: (great books which you can find recommended practically everywhere)

The Snowball, by Alice Schroeder

Damn Right, by Janet Lowe

Both of the above are great biographies of two amazing individuals (Warren Buffett and Charlie Munger).

Infinite Jest, by David Foster Wallace

In talking about the ending for this book, Wallace said “it’s supposed to stop and then kind of hum and project.”  I definitely felt that for days after I read this book.  It’s deeply human, modern, and a fun read.

Foucault’s Pendulum, by Umberto Eco

I wrote something about this.

The Name of the Wind and The Wise Man’s Fear, by Patrick Rothfuss

Great fiction – sort of a cross between Game of Thrones and Lord of the Rings.

The Inner Game of Tennis, by Timothy Gallway

Great book about performance.  Fast read.

I also listened to some great audiobooks:

The Graveyard Book, by Neil Gaiman

Fun, fast bit of fiction.  You may try to call it a kids book but it’s still great.

Caesar, by Adrian Goldsworthy

I’ll never forget the period I read/listened to this in: I rewatched HBO’s Rome and played Total War: Rome II, which has a “Caesar in Gaul” campaign you can play.  Totally fun way to immerse oneself in a subject.

Born Standing Up, by Steve Martin

Funny, of course.  Also a great story about hustle and working on one’s dream.

Kitchen Confidential, by Anthony Bourdain

More hustle.  My favorite part is Bourdain’s story about how he got into food – out of childhood spite.  During his childhood, on a family summer vacation in France, Bourdain and his brother acted like punks all the time.  His parents got so fed up with it that when they went to eat at the world’s best restaurant at the time, they left Anthony and his brother in the car for the whole 2-3hr meal.  That whole time Anthony stewed “what could be so good in there, that I can’t have it?!?  I’ll show my parents, I’ll show everyone – I’ll be the best food-lover there is!”

The Subtle Art of Not Giving a F*ck, by Mark Manson

Great book, don’t let the title fool you.  There’s more to Mark’s message than being carefree.  I picked up a lot from this book.

The Power of Vulnerability, by Brene Brown

More great content for self work.  This isn’t a book per se but a recording from one of her courses.  It’s packed with a lot of wisdom and ways to become more emotionally intelligent.  I intend to listen to this again someday.

The Upside of Your Dark Side, by Robert Biswas-Diener and Todd Kashdan

Good focus on an issue that the two previous titles cover – how our weaknesses or “bad” emotions (like anger) can be better harnessed.  You can skip this one if you were to read those others, but it’s still an important topic.

10% Happier, by Dan Harris

Really entertaining.  Dan does a great job discussing his struggles and approaching meditation in a way that most of us (who might be skeptical) probably would.  He’s a self described neurotic/worry-wart, and was afraid meditation and mindfulness would cause him to lose his “edge” he perceived he had from worrying.

Rap citations

I am a Hip-Hop fan and ever since I saw the Fleshmap project show how the booty is the most mentioned body part in Hip-Hop I have wanted to accomplish a pet project whose goal is to answer the question – Who is the most quoted rap artist of all time?  If you pay attention to the lyrics of your average song, you’ll hear a lot of the same phrases again and again: grown ass man, me and Lorenzo rolling in a Benzo, shorty, crunk, etc…  There was likely one artist who first used this phrase in a song, and afterwards it spread.

The analogy I’ve always used is citations in academia.  If you’re an academic researcher, your credibility and importance is generally based on the number of other scholarly works that cite your own papers.  Herbert Simon had over a thousand citations which makes him a pretty big deal.  My goal with this project is to find who is the most cited artist – who coined a phrase that has been used more than any other?

Finally, after probably 2 years of letting this project sit on the sidelines, and after dragging a colleague at Infochimps away from work over enough weekends, I have started to hack away at my questions.  I have imported a database of around 45,000 songs with lyrics into MySQL and am running queries against it.  The data is not perfect, if you’ve ever managed an iTunes collection then you know how frustrating the metadata can be to organize, but these initial queries are probably within a reasonable ball park.

I am not yet proficient enough to run anything too complicated, so I’ve started with a dozen or so words and phrases that I know are common.  What follows is the phrase, the number of artists who mention it, and then (if interesting) the top artists who mention it and their total songs that contain the word/phrase.

Ass
10,255 Artists
– Insane Clown Posse – 204
– 2Pac – 195
– Nas – 194
– Too Short – 190
– Lil Wayne – 190
– Busta Rhymes – 180
– Snoop Dogg – 180
Nigga
9,684 Artists
– JayZ – 257
– Lil Wayne – 256
– Nas – 220
– 50 Cent – 214
– 2Pac – 210
– Master P – 210
– Snoop Dogg – 210
– Busta Rhymes – 201
Fresh
2,658 Artists
– Big Tymers – 77
– Lil Wayne – 63
– 2 Live Crew – 58
– Juvenile – 38
Diss
1,774 Artists
– KRSOne – 35
– Chamillionaire – 32
– Royce Da 59 – 31
– Canibus – 28
– Insane Clown Posse – 26
– IceT – 25
Shorty
1,767 Artists
– Mobb Deep – 61
– 50 Cent – 55
– Busta Rhymes – 49
– Nas – 37
– Lil Wayne – 33
– LL Cool J – 28
– Twista – 27
Sucka
1,120 Artists
– Lil Flip – 30
– E40 – 25
– TI – 24
– Playa Fly – 24
Booty
1,053 Artists
– Too Short – 26
– Kool Keith – 22
– Twista – 21
Crunk
776 Artists
– Three 6 Mafia – 21
– Ying Yang Twins – 20
– Li’l Jon – 19
My nine
567 Artists
– 2Pac – 18
– Master P – 11
– Nas – 9
– ZRo – 9
– Ice Cube – 9
– Cormega – 9
Throw your hands [in the air]
401 Artists
– Busta Rhymes – 10
– Ol Dirty Bastard – 6
– KRSOne – 6
– Method Man – 5
Benjamins
229 Artists
– Puff Daddy – 6
– Notorious B.I.G. – 6
– Ja Rule – 4
Dollar Bills
267 Artists
Dead Presidents
186 Artists
Half steppin
107 Artists
Lorenzo [rollin in a benzo]
92 Artists
Grown ass man
88 Artists
Gift of Gab
81 Artists
Like a jungle [out there no man is safe from]
78 Artists

I am already really pleased with the results, but there is a lot more that I want to do.  I have a friend that’s a brilliant designer who will work up a visualization for some of the final results.  Currently my data doesn’t have any years attached to the albums, so once I manage to get that in there then I can understand who said each of these phrases first.  It could be nice to attach geography to each artist, so we could possibly plot the progression of the phrase across the US.

If you have any questions you want to suggest I’d love to incorporate them into the project, and if there are any popular phrases that stick out in your mind please suggest them to me.  There is also another project out there doing something similar called the Hip Hop Word Count, which looks really promising.

Can a person change?

Is there at all a formula for how old a person gets vs. how hard it is for that person to change? I feel like for quite some time now I’ve struggled with the same things. I look at my co-founders and after the last year I can recognize their deep flaws, their most difficult thought processes to deal with and it’s very obvious they’re not going to change anytime very soon. I’ll have to adapt.

What about the things in me that I need to change? Those glaring flaws that they will see in me. How able am I to actually change those?

It’s like once a month the same black dog comes out to misdirect my thinking, to distract and steer me towards self-pecking. It’s always for the same causes – laziness, inexperience, and a hesitation to confront somebody. But is there a better root to focus my efforts on? The resolution to change any of those only ever lasts a few days.

The importance of mission in business

It is becoming clearer and clearer to me that almost nothing is as important in business as mission.  Boyd had a German word for this, auftragstaktik, and he defined it as a contract between leadership and subordinates that defined why that relationship even existed.  There is no way you can expect effective work from smart people unless they enter this contract.  They have to understand the organization’s mission and then their work will just flow towards that.  If you don’t make explicit the purpose the motivated people will come up with their own version, and the not-so-motivated will give up and do less work.

I get the feeling that many organizations only give singular missions to teams or subordinates and forget to unify everything.  It’s apparent when we’re dealing with a company that is making enormous mistakes on the ground, but everybody thinks they’re doing their job correctly because their individual mission is following through.  Nobody there has taken a step back to see how little actions may going to tear that company down in the long run.  Maybe business development’s success is linked to number of logos on a partner page, PR’s success is tied to number of press articles, and development’s success is based on the rate they can crank out new features.  Great!  But what ties it all together?  What is the point?

This mistake happens internally sometimes for us, too.  We grew so fast over the last few months that when a new employee came on they were told a bunch of tasks and they weren’t given a sense for how they fit in the overall picture.  The people I work with are so bright that whenever they come up with something we could do it is almost always a good idea, but not always is it central to our mission.  Or maybe it does relate to our mission, just not yet.  With a small team and a limited amount of cash in the bank we can only expect to inch our way to our goal as a unit.  There can be almost no splintering of functions or projects across our tiny organization, every move has to count.

The word (another German one) for an individual mission or focus is schwerpunkt, and it must be born of the auftragstaktik.  At Infochimps we have one goal – to be the plumbing for the data web.  To that end, our organization has 3 teams – a business development team, a data team, and a site team.  Business development’s job, or schwerpunkt, is to listen to as many people as possible for which data is valuable and to evaluate opportunities for the company.  Data team’s schwerpunkt is listen to this feedback and build products, and the site team’s schwerpunkt is to build the best site possible that enables this pipeline.  So long as everyone is working with their schwerpunkt, the company will always be in motion towards its goal.  Partner logos, press articles, and site features are only means to this end.

The Crafstsman

I am reading this book called The Craftsman by Richard Sennett.  Craftsmen are people who work for quality and seek to become masters in their trade.  The book draws from history’s archetypal potters and goldsmiths as easy examples of crafstmen, but your modern day software developer or manager can both have roots in this same quest for a standard of quality.

Prehension is when your hand knows what to shape itself to before it picks up a familiar object, say a lighter or a hammer.  Consider what this is like for the master blacksmith who has struck his hammer 10,000 times.  His mind, shoulder, and arms expect and conform to every exact action he takes with that tool.  Or consider a mind of prehensility, a mind that shapes itself to new ideas and challenges.

Sennett highlights challenges as an important step in the process of mastery but gets more technical than saying that this is just “to learn.”  Challenges aren’t only a ground for finding solutions, but are where we go to find problems.  You have to break some things or identify when they do not work to understand them.  An auto mechanic will know an engine after having broken it down to pieces and put it back together.  To get your startup business right, you must discover and understand why you’re not selling enough or developing the right product.

You can use the process outlined in The Craftsman in any pursuit.