Sunday, January 24, 2016

Confusion About Correlation and Causation ... in a Research Methods Textbook?!

Every so often, textbook publishers send me free copies of their books. Usually these are books for courses I teach, but sometimes they aren't. This week I received a copy of Discovering the Scientist Within: Research Methods in Psychology from Worth Publishers, a new title in its first edition. I don't teach a research methods course, but I flipped through the table of contents anyhow, and I noticed an entry called "Research Spotlight: The Upside to Video-Game Play" on page 31. Since the question of how playing video games might affect cognition and behavior is a controversial one, I was curious to see what the authors had to say about it in a research methods context. Unfortunately, their discussion has some problems.

First, they claim that "there are some real advantages to playing video games," citing a finding that "more time spent playing video games coincided with greater visual-spatial skills." Stating that playing video games has advantages is a statement of causality. If people who played video games just happened to have greater visual-spatial skills (maybe because their visual-spatial skills were greater than those of non-gamers to start, or because their visual-spatial skills were improving faster than those of non-gamers), there would be no "advantage" to the game-playing. The abstract of the underlying paper by Jackson et al. (2011) makes no mention of random assignment of participants to different amounts of video game play, so there's no justification for inferring causality. (Additionally, it notes that the video-game players had lower GPAs.)

Second, they say that "it just so happens that surgeons benefit from video-game playing as well," citing a study by Rosser et al. (2007) that found that "surgeons who played video games for more than 3 hours a week made 37% fewer errors and were 27% faster in laparoscopic surgery and suturing drills compared to surgeons who never played video games." This is followed by speculation as to the mechanism by which game playing could cause these differences. However, the evidence of causation here is even weaker than for the study of children cited above. It's not even a longitudinal study—it's just a cross-sectional finding of an association between video game play and performance on (computerized) tests of surgical skill. Again, one need only read as far as the Rosser et al. abstract to find the statement "Video game skill correlates with laparoscopic surgical skills." There is no evidence of causality, but the textbook authors have said that surgeons "benefit" from playing video games.

Finally, the caption below the stock photo reinforces the thrust of the boxed text by asking "If video games can make you a better surgeon, what other ares of your life could playing video games improve?" As they say in courtroom dramas, "Objection! Assumes facts not in evidence."

Sure, this is a run-of-the-mill mistake that laypeople make all the time: Confusing evidence of correlation (video game playing co-occurring with increased spatial skill or surgical proficiency) for evidence of causation (playing video games making your spatial skills and surgical proficiency better than they were before). But this is a textbook on research methods in psychology. If the authors of such books have the proverbial "one job to do," it is teaching their readers what conclusions can be drawn from what kinds of evidence. That's what education in research methods is all about: learning to design research studies that have the power to permit certain inferences, and learning which inferences can and cannot logically follow from which designs. You can think of analogies to other fields—a nutrition book reversing the properties of carbohydrates and fat? An algebra textbook getting the quadratic formula wrong? A history book that confuses the Declaration of Independence for the Constitution? Correlation versus causation is not a nuance or side issue; it's at the heart of the behavioral science enterprise.

The authors of Discovering the Scientist Within must understand the distinction between correlation and causation, and I am sure they can generate the plausible alternative (non-causal) explanations for these video game results that I mention above. I know this because on page 30, in the paragraph immediately before the "Research Spotlight" box, they write, "often there is not a set direction of how one thing influences another ... News coverage, such as in cases of school shootings, often portrays playing video games as the cause of aggressive behavior. Yet it is equally likely that aggressive individuals gravitate toward violent video games" [emphasis added].

The fact that mistakes like this can turn up in a book meant to educate its readers to avoid them is remarkable, and I think it goes to show just how confounding sound causal inference can be for the human mind. As Daniel Simons and I argued in The Invisible Gorilla, human beings are susceptible to an "illusion of cause" that leads us to jump to particular causal conclusions in all kinds of situations where the evidence we have doesn't logically justify them—indeed, where other explanations are equally or even more likely, and where the assumption of causality can get us into big trouble. The ease with which we can generate mechanisms to explain a particular causal inference can contribute to the illusion. For example, being aware of the "neural plasticity" concept could make it seem more likely that intensive cognitive work (e.g., video-gaming) might "train" some more fundamental underlying cognitive capacity (e.g., spatial skill) or transfer to some other practical task (e.g., surgical proficiency). None of us, not even psychology professors who write textbooks on research methods, are immune to these fundamental thinking pitfalls.

Hopefully the second edition of Discovering the Scientist Within will correct these correlation/causation errors, as well as any other issues that may lurk in the text. My quick flip-through picked up one more passage the authors might want to think about rewriting:
... the author Malcolm Gladwell, a self-described "cover band for psychology," is known for his ability to summarize and synthesize psychological findings so that the general public can benefit from the exciting advances in knowledge that psychological researchers have made.
Accompanying this sentence on page 41 is a photo of Gladwell's book Outliers. As many readers of this blog will know, I don't agree that the general public is benefitting from Malcolm Gladwell's writing, precisely because he doesn't summarize and synthesize as well as people think he does. Since correlation, causation, and statistical thinking are among the things Gladwell has difficulty with, it doesn't seem like a research methods textbook should be endorsing his work.

27 comments:

  1. They might also revise phrases like "a random person on the street." Students use this phrase colloquially even though it's unlikely that the person they find on the street would truly be random. When we teach RM it is worth teaching students to use that term really carefully.

    ReplyDelete
    Replies
    1. This is an excellent point; thanks for raising it. I had noticed it myself, but didn't want to lose focus from the issue I wanted to address. Since randomness in general is another thing human nature has trouble grokking, we should make sure to deal with it carefully in research methods (and in the rest of psychology education).

      Delete
  2. There's worse, but at least not in a textbook:

    http://meangreenmath.com/2016/01/26/what-happens-if-the-explanatory-and-response-variables-are-sorted-independently/

    ReplyDelete
    Replies
    1. That's awesome. And it goes to show how counterintuitive some simple quantitative ideas can be. (Assuming there really was a "manager" who thought that sorting the variables separately before doing a regression was a good idea.)

      Delete
  3. I have never had a psychology course, and am only a lay consumer of mass media with no training in logical thinking. And yet even I am constantly annoyed by statements in the press that obviously confound correlation with causation, or at least do not clearly establish that an evident correlation has risen to the level of logical causation. Since I am not a psychologist, this annoyance is merely personal and not professional. I group this particular annoyance with other personal annoyances, such as when the pronoun "I" is used as an object instead of the correct "me," and the helping verb "may" is used as a past tense instead of the correct "might" (few people nowadays seem to realize that "might" is the past tense of "may").

    ReplyDelete
  4. Motivational article! This information is an especial bounce by you and the ways you write love it.

    The walking dead jacket

    ReplyDelete
    Replies
    1. You get a prize for creative advertising in spam! Your prize is that I'll leave your link to Rick's jacket here, rather than delete it.

      Delete
  5. Researching is not that much easy task. It need a concentrated mind. That is why students are feeling lazy with their research work. They are not ready to research for completing their dissertation. They were choosing the dissertation writing help for guidance.

    ReplyDelete
  6. Keep in mind to attempt and complete this as an amazement to your kid. watch Roland Garros live

    ReplyDelete
  7. Possibly one weekend when they can invest some energy with a companion, dependably utilize outside assistance to shock a youngster in the event that you can.
    Roland Garros Direct

    ReplyDelete
  8. Understand that immaculate football room set up today.french open live streaming

    ReplyDelete
  9. Write more, thats all I have to say. Literally, it seems as though you relied on the video to make your point. You clearly know what youre talking about, why waste your intelligence on just posting videos to your blog when you could be giving us something enlightening to read? French Open 2016 Live Stream

    ReplyDelete
  10. keep informing us like this and try to write more. When yuo have new stuff to read then please inform us on buy coursework online

    ReplyDelete
  11. Read this very important review regarding the Quantum Code SCAM, before you throw your money away!

    ReplyDelete
  12. Here is our detailed review of the Quantum Code software and the so-called Michael Crawford, you have come to the right place.

    ReplyDelete
  13. We found some FACTS about Quantum Code software that proves that it ...

    ReplyDelete
  14. College essay may be based on such sources as: textbooks, books, journals, etc. have a glimpse at this link to find more.

    ReplyDelete
  15. This is extremely helpful info!! Very good work. Everything is very interesting to learn and easy to understood. Thank you for giving information.

    defend your nuts 2 | cooking fever |cat mario | bloons tower defense 5

    ReplyDelete
  16. This article shared a better knowledge about the confusion about the correlation. I like to share more details about this in thesis writing service

    ReplyDelete
  17. This is very nice. We are also provide online education at nationwide, so many people have no time to attain daily basis class and going university take admission online and Buy A Degree and complete your reaming education. get Admission fast.

    ReplyDelete
  18. I enjoy a couple of from the Information which has been written, and particularly the comments posted I will definitely be visiting again
    Dissertation writing services

    ReplyDelete
  19. As I wandered around ASQ I noticed all of the gray-haired old guys like me. We learned the charts before there was software to draw the charts. We had to do it manually. Student started doing reseach about this confusion and going to write a thesis taking help with coursework by coursework bliss We've confused correlation with causation. You don't have to draw a chart manually or calculate a formula manually to learn Six Sigma.

    ReplyDelete
  20. A hand written letter can be a highly prized possession, especially in this era of emails and online chatting. But a hand written letter on high quality writing paper can make it even more prized, enhancing the reading experience.
    purchase essays for college

    ReplyDelete