KEY THEME: Psychological science relies on empirical evidence and adapts as new data develop.
The key integrative theme for this chapter is “Psychological science relies on empirical evidence and adapts as new data develop.” Like the Ethics theme, this one applies to the work of all psychologists. Although it seems mostly related to psychologists who are actively conducting research, it also includes psychologists who are consuming research, including health service psychologists, teachers, and psychology students.
First, let’s break this theme down into its two components:
- Relies on empirical evidence
- Adapts as new data develop
Psychological science relies on empirical evidence
Western psychology relies heavily on quantitative research (using numbers and statistics) and the scientific approach to build its theoretical foundations. There are three basic features to using the scientific approach.
- Research strives to be systematic – As Lilliard suggested, researchers must be organized and consistent in their methods of inquiry so they can trust their results.
- Research is focused on an empirical question – In the pretend play example, the question was “is pretend play related to social skill development?” Empirical questions can be answered through observation and experience.
- Research creates public knowledge – Psychologists, like other scientists, publish their work. This usually means writing an article in a peer-reviewed professional journal targeted for other scholars. The article explains their research question in the context of previous research, describes the methods used to answer the question, and presents their results and conclusions.
Psychological science adapts as new data develop
Psychological knowledge is mostly developed through theories. Theories are a well-developed set of ideas that propose an explanation for observed behaviors we see in the world – and they are often in a continuous cycle of development and testing. Theories are typically based on multiple studies conducted by different researchers over many years. No one study can lead to full understanding – we must consider different contexts and populations using a variety of methods. Additionally, despite researchers’ best efforts, sometimes methods are flawed and/or conclusions are incomplete. Publishing the research allows others in the scientific community to contribute to the conversation, detect and correct errors, and develop new ways of answering the question. Over time, theories become more refined and accurate.
As we can see in Figure 4.1, knowledge that is built through the scientific process often changes over time.
Psychological scientists ask empirical questions and make predictions (hypotheses) derived from theories and then test those hypotheses. If the results are consistent with what was expected, then the theory is supported. If the results are not consistent, then the theory might need modification and further studies are needed.
Theories are too complex to be tested all at once; instead, researchers create hypotheses to test specific aspects of a theory. A hypothesis is a testable prediction. For example, in a 2014 study by Mueller and Oppenheimer,[2] researchers hypothesized that when learners take notes by hand, it leads to better learning than taking notes on a laptop. Then they set out to test their prediction by collecting and analyzing data. They found support for their hypothesis, which sent a ripple through the education world. Following this, other researchers set out to replicate these results – some in controlled research labs, and others in live classrooms.[3] Some results suggested that the way notes are taken is less important than how deeply the information is processed. This further contributes to the theories about how people learn. This is good news for folks who struggle with physically writing – they may be able to learn just as well as someone who is writing longhand, as long as they process the information in a deep way and avoid distraction.
Adding an additional layer to this is the context in which theories are developed and research is conducted. Consider technology such as predictive text and tools to edit and organize your typed notes. Since the time the Mueller & Oppenheimer studies were conducted, technology has rapidly advanced, and the ways that students interact with technology and note-taking has changed quite a bit. Although the results of the 2014 study are considered robust, they may not be fully generalizable to today’s learners. This is why psychological science must continually re-visit theories and adapt as new data develop. A recently published meta-analysis (a method where the results of many studies are aggregated) suggests that handwritten notes are superior to typed notes for learning, but calls for more research.[4]
The debate around “laptop versus longhand” is not quite settled. Perhaps this is no longer the most relevant question, however. Maybe we should be asking how technology and handwritten notes interact for optimal learning. The effect of instructor-created supplements such as slides and lecture outlines has not been studied, nor has the unique characteristics such as note-taking skill and motivation. Further, many college instructors no longer rely on lecture + student note-taking as the primary activity in class. Student-driven discussion, small group activities, and physical movement are additional forms of learning. As with most evidence-based decisions, when instructors make classroom policies or students choose note-taking modality, they must consider the research as well as teaching style, learning goals, accessibility, and other individual differences in how humans learn.
Link to Learning
Determining Cause and Effect
Did the results of the note-taking study surprise you at first, or did you feel skeptical about it? You might have thought—
doesn’t learning depend on a lot of different factors—including a person’s attention and desire to learn, time spent studying, their prior knowledge, or life circumstances? What about the topic or level of the class and one’s personal interests? What if some people were bored by the lectures and didn’t pay attention? Wouldn’t people who are having a bad day perform worse on the quizzes?
You are absolutely right and you are thinking like a psychologist! All of these factors (and many more) can easily affect a person’s performance on quiz questions. So, how can we take all of these different variables into account and be sure that it is the different note-taking that is causing the change in quiz scores?
Although there are many valid ways of learning and knowing about the world, if we want to understand cause and effect, we need to use a specific kind of empiricism called an experimental design to control for those variables that can get in the way of the one we want to study. There are two major required features of an experimental research design:
random assignment and manipulation or control of variables. The Mueller and Oppenheimer (2014) study is an example of this.
In random assignment, participants do not pick which group they are in, the experimenter assigns them to a particular condition or group randomly. This might be based on the flip of a coin, the roll of a die, or the generation of a random number by a computer. Random assignment makes it so the groups are relatively similar on all characteristics, except the one that the experimenter wants to manipulate. In Mueller and Oppenheimer’s study, participants were randomly assigned to either computer or longhand note-taking, because of that random assignment, there should be equal numbers of people with strong attention skills and equal numbers of people who were having a bad day in each group.
The requirement of manipulation or control means researchers are able to give each group a different task, or change the conditions under which participants are doing the task, so they can compare participants’ performance on some measurement. In the note-taking study, the researchers manipulated whether or not students took notes using their computer or by hand (independent variables) and measured the outcome (dependent variable). Because these researchers manipulated the conditions by randomly assigning students to computer or hand notes, they can make some causal claims about note-taking impacting test scores in those experimental conditions. In contrast, in other types of research design, our claims about cause and effect are much weaker because we don’t have experimental control over the groupings.
How could the note-taking effect be tested in a more realistic setting? If you said “test it in a real classroom” you’re thinking like a psychologist again! Students typically choose their own method of note-taking – so studying “reality” might mean that rather than assigning the groups, we study the grouping that already exists to compare laptop to longhand. The study design used here is known as a quasi-experiment. Even though a quasi-experimental design is similar to an experimental design (i.e., the researcher controls or manipulates important variables), because there’s no random assignment you cannot reasonably draw the same conclusions you could with an experimental design. However, quasi-experiments are still very useful and often necessary when we want to gain information about groups of people that already exist and cannot be manipulated, such as culture, income, religion, or family history.
It is sometimes difficult to know whether experimental findings would apply in real-world settings. This is one of the reasons why it is important that psychology as a discipline includes multiple types of research methods to build knowledge about the mind and behavior. In another type of design, correlational studies, scientists do not intervene and measure changes in behavior. Instead, they passively observe and measure variables and identify patterns of relationships between them. So, they can make predictions about relationships between two variables, but they cannot make conclusions about cause and effect.
We call this the golden rule of research; correlation does not equal causation. For example, if researchers collected data in a real classroom, recorded the note-taking methods, and compared information about grades – but did not intervene at all – that data would inform the theory but we could not make a cause-effect claim based on it. Correlational studies show the
strength and direction of the relationship between two constructs or variables of interest but not whether one causes the other.
Analysis Questions
To assess whether you understand the strength of claims made based on empirical research, you should ask yourself the following:
- What was the question being addressed by the investigator(s)?
- Did the study design allow the researchers to answer the question?
- What were the results and conclusions regarding the question?