human judgment: good or bad?

-- like saying human intelligence, or speed, or spitting ability: good or bad?

 

What standard do we judge it against?

 

However, there is a standard for assessing over- or under-confidence:

-compare people=s confidence levels with accuracy.

Ex. I=m 80% sure

- I should be right 80% of the times I say this

 

Not just a problem of using probability (although that contributes to it).

 

Are we overconfident by constant amount?

No -- correlations between confidence and accuracy are often near 0.

Examples:

eyewitness identification

general knowledge questions

Not simple to correct overconfidence; can=t just subtract a constant from confidence estimates.

 

When are we underconfident?

-- For judgments we=re very unsure of.

Ex: Words people said they were 0% sure of the correct spelling (i.e., thought they had NO idea how to spell them)

-- were actually right 12% of the time.

(Adams & Adams, 1960)

 

Expertise can inflate overconfidence:

(Arkes, Dawes, & Christensen (1986)

- Subjects guess who won baseball=s MVP award in the 40's and 50's.

- Gave them clue to help them.

- S=s who reported knowing more about baseball DID worse than those who reported knowing less.

-- Used rule less, thought they Aknew@

 

Can we reduce overconfidence by simply trying harder?

No: Some evidence that we become MORE confident when we expend more effort

(Paese & Sniezek, 1991)

 

If we=re so inaccurate, how come our lives aren=t all disasters?

-- Not necessarily bad at judging, just judging how likely we are to be right.

Being right may be more important that being right about being right!

 

Who is well-calibrated?

People for whom being right about being right matters

People who get feedback about confidence and accuracy

= weather forecasters

 

Why the feeling that forecasters are not well-calibrated?

1) Can have good correspondance between confidence and accuracy but still be wrong a lot:

A75% chance of rain@ - could say this every winter day in Eugene; doesn=t mean you have done good job distinguishing rainy from non-rainy days.

2) Tendency to convert 30% chance of rain into Ano rain@

-- but it does rain -- on 30% of those days.

3) More likely to remember when forecasters were wrong (more salient).

 

confirmation bias -

tendency to look for confirmatory rather than disconfirmatory evidence.

Scientific method has built-in procedure for disconfirming --

look to DISPROVE null hypothesis, while at the same time finding supporting evidence for experimental hypothesis

Looking for confirmatory evidence (on diagonal)

Does taking vitamins make me feel better? 

          took vitamin   didn't take vitamin
felt good          255            50
felt bad           51             10

ratio of good to bad days is same, whether we take vitamin or not (1 in 6 days are bad)

 

Does choc. make our skin breakout?

 
              eat choc.           don't eat choc.
skin bad       50                  10
skin good      50                  10

Our skin=s just bad!