Perceptions of Paradox
Illusions of Professional Skepticism
When I saw Beverly Hall's picture, I was reminded of my favorite elementary school teacher, Mrs. Foreman. Hall was Jamaican- American and her smile, like Mrs. Foreman's, seemed extraordinarily kind. She was the superintendent of the Atlanta public schools and named the national superintendent of the year in February 2009 by the American Association of School Administrators. The honor had been earned by raising test scores, including those on standardized tests that had been designed to hold teachers accountable as part of the No Child Left Behind Act.
By October the test scores were under scrutiny after being deemed statistically improbable. The truth was, there was cheating going on. A lot of cheating, but this time it wasn't by students. Erasure patterns were discovered on answer sheets that made it appear as though educators had corrected student's answers immediately after the testing. Students that had learning disabilities suddenly became proficient in math and reading, and students who were considered gifted went from exceptional to perfect.
In the fall of 2010, fifty agents with the Georgia Bureau of Investigation started questioning teachers about the conspiracy to falsify test scores. They considered Hall to be the ringleader. An accusation she vehemently denied. She allegedly offered cash bonuses to teachers who could meet the minimum score targets. The culture in the schools became one of unrelenting pressure to raise scores, and the cheating had reportedly been an open secret for years.
All in all, investigators identified 178 teachers and administrators that had fraudulently corrected student's answers. Hall was indicted in March of 2013 and on April 1, 2015, eleven teachers were convicted on racketeering charges and led from the courtroom in handcuffs in what had become one of the largest standardized test cheating scandals in U.S. history.
As I was reading research into the Atlanta cheating case I felt a little twinge of pushback when I learned that Beverly Hall was a target in the investigation. What I'd imagine it would feel like if someone had suggested to me that Mrs. Foreman was a cheater. This is a tell-tale sign of implicit bias that can sabotage your reasoning.
I speak for experts that make high-stakes decisions as a routine part of their jobs and when I talk with them about professional skepticism, they generally see it as the opposite of being gullible.
This view may seem intuitive but comes with serious risks of misjudgment when making critical decisions. In the real world, many of our judgments force us to choose between competing alternatives, not to simply evaluate a single item. If you need to make a choice, simply doubting all of the factors isn't helpful. Also, consider that the way we apply doubt is biased, we apply it unevenly. Evidence that meets our assumptions generally gets a free pass, while evidence that challenges our preconceived beliefs receives more scrutiny.
If your view of professional skepticism is simply setting a higher bar before you believe something, your judgment process is like a ticking time bomb. You're bound, at some point, to make a critical error and here's why. When we talk about skepticism, we almost always mean avoiding believing something is true when it isn't. This is a false positive or a type 1 error. We forget that a false negative, believing something is false when it's actually true, can be just as harmful. Effective skepticism should be a constant process. A screening process kind of like a signal to noise ratio that acts as a gauge for the relevancy of evidence and how we update our beliefs.
Experts have a variety of tools designed to guide their judgment. Physicians use algorithms that inform them which treatment regimen could be the most effective for a particular patient. Engineers use root cause analysis to separate symptoms of problems from their actual cause. Investigators have forensic methods and profiling tools. A problem encountered with many of these tools in real-world practice though is that they have only been developed on a superficial level, and people are not taught how to adapt their use in context. The result is that the tool can disguise dangerous biases and become part of the problem.
A common decision-aid used by investigators is the Fraud Triangle. The fraud triangle was designed to be a framework to help explain the reasoning behind someone's decision to cheat. The theory behind it is that if we can understand peoples reasoning for cheating, deterrence programs can be developed, and the information can be used to identify likely suspects. The triangle describes three categories of effects on decision making: Pressure, or what motive or need exists. Rationalization, the mindset used to justify the behaviors. And opportunity, the situation that allows the behavior to occur.
Assume you have been asked to work up a profile of the suspects in this case while considering the fraud triangle. The year is 1996. There is evidence of a cheating conspiracy underway at a Connecticut high school. Someone is manipulating grades and scores on standardized tests making the results appear better than they actually are. You are shown evidence of the cheating that includes reported grades that don't match original documents, and an analysis that shows the test scores are statistically abnormal. Particularly, many of the best students are scoring even higher than expected. What is the likely motive to cheat in this instance? Who would gain from this cheating, and how would they rationalize it? Who would have the best opportunity to alter test scores? Take a moment and try to build a picture in your mind.
When I present these facts to an audience, here's what they usually come up with: The suspects are likely to be teachers and school administrators that would receive promotions or bonuses by showing improvement in students' scores. Perhaps there's pressure to move more students into gifted programs. Alternatively, they may feel over-pressured to meet certain minimum standards. School officials would have the ability to cheat in this way due to their access to the records. Look for the teachers most affected by the pressure to improve scores, and who have the opportunity to change scores without being checked. Someone higher up the ladder of authority is probably involved.
Now, I'll introduce you to the main conspirator, we'll call her Stephanie.
I'm an Asian female. I was I think around 13 when this all happened.
Picture an upscale suburb in Connecticut where Martha Stewart might live. That's the setting for this story. One day, Stephanie said something happened.
I remember one time my friend…she was kind of freaking out.
She was freaking out because she was frightened. Afraid of her parent's reaction and the punishment she would receive for getting a "B" in the third quarter. Stephanie, seeing just her how upset her friend was, wanted to help.
I just kind of said offhand like well just, just change it to an A-. She was like what are you talking about?
Stephanie was talking about literally cutting the "B" out of the grade report and replacing it with an "A-"and then photocopying the doctored report and sending the copy to her friends' parents in a school envelope. Stephanie happened to work as a student aid in the school's attendance office.
She was so grateful. And my other friends started asking me to do that.
The stakes for the tests just got higher and higher, and her friends need for her help grew more and more important.
We took AP calculus in our like junior year, and it was a big, big test and we were all kind of worried about it…24:02 "my wheels started turning…24:17 "We all had those TI-80 calculators, and it was a time you were downloading games into them, and you could connect it to another calculator and pass notes or whatever, and a light bulb went off in my head…what if during the test…
She came up with a plan to plant a calculator in the bathroom trash can with the test answers on it so her friends could access them during the exam. There would be a wire that could be fished out, so they could connect it to their own calculator and download the answers when they went to the restroom. Each of the co-conspirators was responsible for a small portion of the exam, and one by one go to the restroom and upload the answers to the portion they had worked on. Once all of the answers were loaded into the calculator in the bathroom, they would go back one-by-one and download those answers onto their calculator. They wore cargo pants, and each of them had two identical calculators, one in their pocket, and one on the desk. When the proctor wasn't looking, they would swap the calculator on the desk for the one in their pocket and go to the restroom, always leaving a calculator on the desk, so no suspicions were raised.
One of the reasons it's difficult to predict people's motivations is because it's hard to imagine how the world looks through someone else's eyes. In Stephanie's case, the motivation to cheat can be harder to understand because the students who cheated were not struggling with low grades.
We all had probably if not 4.0 GPA's then like 3.9 GPA's.
And she told me she could see why it would be hard for someone to understand unless they had seen things through her eyes.
I knew that some parents hit their kids …I knew kids that got kicked out of their house. I knew that my mother would say ‘Your only job is to be a student…with all we do for you…we came to this country and left everything behind for you to be able to study here for you to be able to get into a good college, how DARE you get a B.' I knew the conversations they were going to have an I knew the guilt and shame you were going to feel, so I felt no problem doing that. I understood how it feels to think that (sic) there's nothing good enough and that you can't be proud of how you put your best effort in. They were coming to me because they were ‘oh my god I'm legitimately nervous' they were like defeated.
You might find it hard to relate to someone feeling defeated when they have a 4.0. And that might stop you from recognizing a pressure that's very real in someone else's mind.
Let me paint you a picture, I was a straight-A student, and I still applied to 26 colleges because I was convinced I wasn't good enough to get into a college….growing up I thought everybody else was doing better than me.
Eventually, parents that were worried about their children's performance on the SAT started to ask Stephanie's mother if she could tutor them.
She actually would meet mothers of international students who were so worried about test scores and getting into college and she would…introduce them to me, and I would tutor them. So, it was kind of funny that she introduced me to my own quote-unquote clientele. I would talk to these students, and they were very, very bright…but the English portion of the SAT's was really getting to them…And then I thought of a way to help them where I was like well all right well if you pay me, I will go take the SAT for you. I would only do it if they were an Asian female, I couldn't do it for an Asian male obviously. You could take the SAT at any high school…so I would make sure that the girls registered for the SAT's at another high school with people who don't know me and they would give me their license and I would go, and I would take the exam for them.
Here's my favorite part of this story. Some of Stephanie's friends started paying for the help, and over time that money started to add up. One day her mother was cleaning and came across a surprise.
I think when my mom found the money I had underneath my bed I think it was $3,000. I came home one day, and she's sitting at our dining room table, and she had like stacks in front of her, and I was like really confused. I remember walking towards her, and she's like ‘I am trying to stay calm, and I'm trying to understand what could happen because I don't understand how you have this much money.' And then it clicked, oh my god this is the money she found under my bed. And she's like ‘You are 14 years old I know you can't be a prostitute, you can't be selling drugs, I don't understand how you have this much money'. And me, panicking and also being 14, the best lie I thought of was that I said oh, I just saved all the birthday and Christmas money I ever got, and she was like ‘You're telling me that you somehow saved $3,000 from like 14 birthdays if that?' I admitted to her that I was doing this and she was at a loss for words…She kind of understood why I was quote-unquote helping those other students, but she was also like what kind of sick person thinks of this, how could you do this? I felt morally ok to do this though. The money wasn't the reason I was doing it.
The suspect profiles drawn up about Stephanie's case generally fall into one of two categories, those who recalled the Atlanta cheating scandal, and those who didn't. People who recalled the Atlanta case were twice as likely to predict the main suspects were teachers or administrators, although they didn't think their memory had influenced their decision. People that didn't remember the Atlanta case were much more likely to suggest alternative suspect profiles, such as students who were desperate to raise their scores. This is called availability bias. The tendency to be influenced by examples that are most readily available in our memory.
Let's go back to the fraud triangle. Pressure, rationalization, and opportunity. An obvious mistake even experts make when building a profile is to interpret things from our own point of view, or that of a similar example we can recall. We have to be able to think of pressures from the perspectives of those involved, not our own. We have to use our imagination and become unstuck from rigid thinking models. Many people have trouble imagining the pressure Stephanie felt because we haven't experienced it and don't understand it. Our assumptions about pressures, false as they may be, will lead us to predict rationalizations that make sense in the context of that pressure. Then to fulfill the third leg of that triangle, almost like magic, we perceive an opportunity that allowed the pressure to be acted upon. Assumptions begin beneath our awareness before our conscious reasoning process even begins. We then seek information that supports those assumptions, updating our beliefs while becoming ever more confident as we move forward. We entrap ourselves in a circular reasoning process motivated to support our assumptions. We manufacture patterns that don't exist and hold steadfastly to them. In the end, we construct a complex story held together only with the mortar of our illusions.
Here's an example.
Just before 1:00 am during the Olympic Games in Atlanta on July 27, 1996, an anonymous caller dialed 911 and said: "There is a bomb in Centennial Park, you have 30 minutes." Before the warning could reach the AT&T sound and light tower where security guard Richard Jewell was stationed, he noticed an unattended backpack and notified an agent with the Georgia Bureau of Investigation. The agent called bomb technicians while Jewell began evacuating the area. A 40-pound pipe bomb exploded earlier than the caller had reported, killing two people and injuring 111.
Jewell was hailed as a hero on all of the networks. His actions had undoubtedly saved a lot of lives. The very next day, however, the president of Piedmont College called the FBI in Atlanta to report that Jewell had experienced some problems in his record as a policeman there. The FBI decided to run a background check. Agents recalled a recent case where a volunteer firefighter in California had set fires, so he could extinguish them and become a hero. They started to wonder if Jewell had done the same. Maybe he had himself made that warning call, then reported the backpack to the agent, and then began evacuating the area.
Their suspicions seemed to be confirmed when Jewell's background investigation came in showing he had been arrested in 1990 for impersonating a police officer. On July 29th, profiling unit agents said Jewell "fit the profile of a person who might create an incident so he could emerge as a hero." 72 hours after the explosion, news reports were no longer painting Jewell as a hero, but as the focus of the FBI's investigation into the bombing.
Richard Jewell however, was not the bomber. The investigators' judgment had been infected with availability bias very early in the investigative process with their recollection of the firefighter case. This followed a motivated reasoning process where they sought evidence that matched their initial assumptions (known as confirmation bias) and interpreted information in ways that made it feel as though their intuitions about Jewell had been right. The investigators also attributed false personality characteristics and motivations to Jewell because they helped explain their assumptions of why he did it. A bias we aptly call attribution error.
These troubling biases are stronger when investigators have a high need for certainty, which is a common trait amongst auditors, investigators, and many other professionals. It will lead them to see patterns more quickly that match their assumptions and to reject evidence that doesn't fit coherently into their picture. This is when we enter into a strange realm, a paradox of behavioral profiling, where the profile can tell us more about the profiler than the one being profiled. It is apparent when a profiler draws quick assumptions, rejects non-matching evidence, and has a strong aversion to changing their mind.
It took almost nine years for Jewell to be completely exonerated when on April 13, 2005, Eric Rudolph pleaded guilty to the Centennial Park bombing.
Determining the motivation behind someone's behavior has long been a cornerstone of profiling. I find that we are too preoccupied with searching for motivations, however. While they can undoubtedly provide valuable insight, they are also highly subjective and prone to the influence of our assumptions. The case of Stephen Paddock the shooter in the Las Vegas massacre is a good example. Investigators and reporters were obsessed with finding a motive that never materialized. After almost 2,000 investigative leads had been followed, over 21,500 hours of video reviewed, and 250,000 images analyzed, the Las Vegas Police Department released an 81-page report that couldn't identify anything about motive. He left no manifesto. He acted alone. There was no history of mental illness. He had no known radical affiliations. He didn't support or follow any hate groups. Interviews with family and acquaintances told of nothing. These crimes are typically committed by young men. At 64 years old, Paddock didn't fit the expected profile. It would be tempting to say he just "snapped," but it was apparent he had been planning his acts for some time.
The longer we went without finding a motive, the greater the pressure became. A National Review article from January 2018 said "the more I think about this case, the more I think the motive could be the simplest and most chilling of all. He killed just to kill." Statements like this are commonly known as Barnum statements. They are not just unhelpful, they are dangerous because they can give you the illusion, that you've found an answer when you haven't. You become overconfident. You stop being curious. Behavioral experts know this as the Forer effect. The Forer effect is a psychological phenomenon where people give high accuracy ratings to vague statements, such as those used in fortune telling or astrology, even though it could apply to just about anyone.
Overconfidence is so powerful that it can poison our interpretations even in clear hindsight. Consider the case of George Matesky, who came to be known as the Mad Bomber of New York, one of the most famous cases in criminal profiling history. The profile of Matesky was so accurate that it seemed magical. Matesky had been injured in a boiler explosion while working at the Consolidated Edison power plant in Manhattan and his claim for disability benefits was subsequently denied. As revenge, he planted 33 bombs around New York City between 1940 and 1956. Messages were attached to some, and other times letters were mailed to the NYPD directly, stating his anger with Con Edison, saying they would be "brought to justice." The letters were very formal and printed in perfect block letters. One of the letters referred to the ‘dastardly deeds' of Con Edison. They were mailed from locations between New York and Connecticut. Frustrated with their inability to apprehend the bomber after 16 years, the NYPD hired psychiatrist James Brussel to profile the suspect.
Almost all bombers were male so the suspect would be male. The suspect displayed paranoia in his letters which peaks in the thirties, and since it was 16 years later the suspect would probably be middle-aged. The locations from where the letters had been mailed indicated he probably lived in Connecticut. The language used in the letters seemed to be written by someone foreign-born, and there was a large Eastern European population in Connecticut. The formal printing in the letters indicated he would be neat, meticulous and precise. The suspect would have a grudge against Con-Edison, probably a former employee. The most surprising and famous detail in the profile was investigators were told "When you catch him…and I have no doubt you will…he will be wearing a double-breasted suit, and it will be buttoned."
A month later, police arrested Matesky at his Waterbury, Connecticut home. They arrived in the middle of the night, and he was wearing pajamas. When they asked him to change, he returned wearing a double-breasted suit, buttoned. The accuracy of this prediction may not be what it seems though. The profile, including the detail of the double-breasted suit, had been published in the New York Times. It's likely Matesky read this, possibly leading him to, consciously or unconsciously, fulfill the prediction. The police had been using the newspaper to communicate back and forth with Matesky. They had the newspaper print a letter to the bomber asking him to confess, misstating information that they thought would prompt him to respond. Matesky took the bait, eventually contacting Dr. Brussel directly. The critical factor leading police to Matesky was not the profile. It was due to an administrator at Con Edison, who while looking through employee files for a suspect, came across angry letters written to the company after denial of a disability claim. One of the letters used the familiar but unusual term "dastardly deeds."
I teach experts to ferret out invisible influences that poison their judgment using a three-tiered approach. The process isn't complicated, but it can be challenging because we take our deepest assumptions for granted. There are many times people get surprised that they have an emotional attachment to their ideas, and then challenging them becomes a threat.
Start by exploring your logical processes. This level is tier 1, and it's less about subjective assumptions and more about objective facts. Are you starting with the same set of facts as others? Do these facts match up or can you find gaps or inconsistencies? In the Centennial Park bombing case, this would be the location where the backpack was found. In Stephanie's case, it would be the test scores.
Once you clear through the conscious and logical thoughts of tier one, move on to the pre-conscious and subconscious assumptions of tier two. Pre-conscious thoughts include those that you remember once you are reminded of them. Kind of like having an "oh, yeah" reaction. Subconscious thoughts are those beneath your awareness. Tier 2 processes are why, in the world of placebo's, pain relievers that cost $20 a pill are more effective than a pill costing only $1, even though they contain identical ingredients.
You're generally unaware of tier 2 processes, why they happen or how to control them. They are based on your subjective beliefs that we can become emotionally attached to. This is why it helps to have someone else to work closely with you in exploring this level. Other people will be able to bring up ideas and recognize biases you can't see in yourself. An example of a tier 2 assumption would be the motivations you assume someone would have for their actions. In the Centennial Park bombing case, if you believe the motivation was to look like a hero, you would focus on Richard Jewell and are more likely to interpret subsequent evidence to support that assumption. In Stephanie's case, if you recall the Atlanta school cheating scandal and believe the motive was raises or bonuses, you're going to focus on teachers and not students.
Working through tiers 1 and 2 prepares you to deal with the most challenging level, tier 3. These implicit, unconscious assumptions are the most difficult to tease out but are the source of many of our most serious misjudgments. When the source of a problem feels like a complete mystery, it is frequently rooted in tier 3. On the placebo scale, think of Pavlov's dogs. The brain networks involved don't handle language, so you have to do more than talk, you have to pay close attention to the feelings attached to your reactions to thoughts or ideas or evidence.
Do a lot of exploration of your most fundamental beliefs. Many are so fundamental as to be invisible to the believer, so it's important to work with one or more partners. An example of tier 3 would be the assumption that Beverly Hall isn't likely to cheat because she reminds you of your favorite first-grade teacher.
You'll also need to question tools such as the fraud triangle in ways you never have before. By analyzing how they integrate with and affect your judgment, not as a process that is separate from you. Finally, you should question the assumptions that are automatically built into the methods themselves. The most fundamental assumption of profiling is that behavior can be predicted by analyzing personality characteristics. We now know that personality characteristics display very differently in different situations, which makes them an unstable predictor of behavior. The field of psychology recognizes that situational factors rather than personality characteristics are a determining factor of behavior.
If you relied only on the personality traits of Stephanie and her friends, you may have thought that punishment would be a driving factor. If you only looked at the superficial evidence, you might have thought that she was motivated by money. You would have missed a much deeper story.
I can't stress enough…no punishment is worse than like the guilt and shame that you feel…there's not a hit in the world hard enough that even compares to that kind of feeling. If you put someone in the corner what do you expect them to do, they're going to fight back, whether it be wrong or right.
It turns out the best use of professional skepticism isn't to examine others, but ourselves. To be more self-aware. To question our assumptions. To challenge overconfidence and overcome the illusion.
© Toby Groves 2018 All Rights Reserved
Written by: Toby Groves, PhD
Podcast produced by: Kate Messer
Pocast editing: Sam Groves