4 | The Psychological Scientist

Theme: Psychological science relies on empirical evidence and adapts as new data develop.

Research story: A Problem with Play

Part of being a good researcher is being able to acknowledge the areas where your research area needs strengthening. Sometimes that means taking a critical lens to your field and your own past work. One such example happened in 2013, related to researchers who study children’s pretend play. In a published paper, developmental psychologist Angeline Lillard called out previous research on the way it was conducted and the results were interpreted. She makes suggestions about methods and statistical analysis to improve the rigor in child development research in general, and on pretend play specifically. She also calls for researchers to replicate (repeat) prior studies so that more definitive conclusions might be drawn.

It is important to note that the researchers mentioned in Lillard’s paper did not do anything wrong or unethical. Science is an ever evolving practice. Oftentimes, methods and statistics that were acceptable and regular practice even just five years ago can become outdated or replaced by better, more rigorous, or more valid practices. Psychological scientists and practitioners should always be looking to improve and not get stuck in certain practices just because they have always been done that way. Sometimes it takes being called out by another researcher like Lillard and her colleagues to reevaluate our methods and to improve our field.  

One critique in this paper questions some conclusions that were made about how pretend play is related to social skills. The research design in these studies was correlational. With this type of design, the strongest claim that can be made is an association claim. For example, we can say that children who engaged in more pretend play had more developed social skills than children who didn’t play this way as often. However, we cannot say that pretend play caused an improvement in social skills. Some of the research reports made statements implying causal relationships – and this is what Lillard and her colleagues were concerned about. Correlation does not equal causation – a more rigorous research design is required to support causal claims. Additionally, there are several variables that can influence the development of complex functions such as social skills, and some were not considered. Lillard’s paper reminds readers to be critical consumers of information. 

Another critique that Lillard and her colleagues pointed out in their 2013 paper was about how the researchers worked with children and approached different questions in their studies. For example, one critique was that researchers weren’t always masked. Masking means that the researchers do not know which children are in which condition. When the researchers were masked, the significant findings went away or were not as strong as before. This suggests that strengthening scientific rigor in studies of pretend and play is likely necessary. 

Although this paper sent shock waves in the pretend-play research community, it ended up being an overall positive ripple. Now, almost every paper about pretend play addresses this article and most of them follow Lillard and colleagues’ recommendations for how to better conduct science in this area. Overall, it helped strengthen the field by pushing researchers to approach studying play in a way that ensures their results are real and conclusions have a strong scientific backing.^[1]

KEY THEME: Psychological science relies on empirical evidence and adapts as new data develop.

The key integrative theme for this chapter is “Psychological science relies on empirical evidence and adapts as new data develop.” Like the Ethics theme, this one applies to the work of all psychologists. Although it seems mostly related to psychologists who are actively conducting research, it also includes psychologists who are consuming research, including health service psychologists, teachers, and students of psychology.

First, let’s break this theme down into its two components:

1. Relies on empirical evidence
2. Adapts as new data develop

Psychological science relies on empirical evidence

Western Psychology has relied heavily on quantitative research (using numbers and statistics) and the scientific approach to build its theoretical foundations. There are three basic features to using the scientific approach.

1. Research strives to be systematic – As Lilliard suggested, researchers must be methodical and consistent in their methods of inquiry so they can trust their results.
2. Research is focused on an empirical question –  In the pretend play example, the question was “is pretend play related to social skill development?” Empirical questions can be answered through observation and experience.
3. Research creates public knowledge – Psychologists, like other scientists, publish their work. This usually means writing an article in a peer-reviewed professional journal targeted for other scholars. The article explains their research question in the context of previous research, describes the methods used to answer their question, and clearly presents their results and conclusions.

Psychological science adapts as new data develop

Psychological knowledge is mostly developed through theories. Theories are a well developed set of ideas that propose an explanation for observed behaviors we see in the world – and they are often in a continuous cycle of development and testing. Theories are typically based on multiple studies conducted by different researchers over many years. No one study can lead to full understanding – we must consider different contexts and populations using a variety of methods. Additionally, despite researchers’ best efforts, sometimes methods are flawed and/or conclusions are incomplete. Publishing the research allows others in the scientific community to contribute to the conversation, detect and correct errors, and develop new ways of answering the question. Over time, theories become more refined and accurate.

As we can see in Figure XX, knowledge that is built through the scientific process often changes over time.