Observation, Case Studies, Archival Research & Meta-Analysis
Inside Research: Dan McAdams, Department of Psychology, Northwestern University
Media Matters: An American Family Observational Methods
Observational methods involve gathering information by observing phenomena as they occur. Particular rules and structures shape how the observation is constructed, depending on the type of technique. There are two
broad categories of observational methods: naturalistic and structured. Naturalistic Observation
Naturalisticobservationinvolves watching and studying phenomena as they naturally occur in the environment. While the researcher may be interested in specific behaviors, in naturalistic observation the researcher does little to control the setting. Most naturalistic observations start with a relatively focused area of interest that may be constrained by the researcher’s
background and theoretical orientation. The researcher’s background and theoretical orientation often constrain the focus of the research. As with other areas of research, ethical dilemmas can arise during the course of observation. Advantages of Naturalistic Observation
First, naturalistic observation may be the only practical way to study certain topics. Second, by its very definition, naturalistic observation is high in external validity. Disadvantages of
Naturalistic Observation
The lack of control in naturalistic observation is an asset when you are still developing your questions but becomes more problematic if you have already defined them. Additionally, natural observation may not be the best choice for studying relatively infrequent behaviors. A third limitation of naturalistic observation is the problem of bias from researchers who are participant observers. Reactivity also occurs when the individuals being observed know they are
being watched and, as a result, change their behavior. There are two imperfect strategies for dealing with reactivity: concealment and habituation. In concealment, the observer finds some credible way to prevent subjects from knowing they are being watched. In habituation, individuals become accustomed to the experimenter’s presence. Example of Naturalistic Observation: Parent and Child Sportsmanship
Behavior
Arthur-Banning, Wells, Baker, & Hegreness (2009)
examined the relationship between adult behaviors and the behaviors of third- through sixth-grade athletes in a community basketball league. Through naturalistic observation of games and coding behaviors, they discovered that positive spectator and coach behaviors served as predictors of positive athlete behavior, while negative spectator behaviors predicted negative athlete behavior. Example of Naturalistic Observation: Child Care Quality
While thoughtfully designed
naturalistic observations can yield useful data, creating a workable observational scheme and training researchers to use that scheme reliably require a substantial investment of time and effort. These challenges were present in the National Institute of Child Health and Human Development (NICHD) Study of Early Child Care in the mid-1990s. A major task for this study was to figure out ways to use observation to systematically measure the characteristics of care being received by children.
Reaching adequate proficiency and accuracy in completing observations with this measure involved many months of difficult, often tedious training. Example of Naturalistic Observation: Scale Errors
The work of Rosengren and colleagues (2009) on scale errors involved a data collection method called event sampling, where researchers count each occurrence of a particular behavior or event. The design involved participant observation, where an individual who is part of the
environment (i.e., a parent) observes the behavior of interest (i.e., scale errors by their children). Experimenters trained parents in recording their children’s behaviors. Based on parents’ observations, children were found to commit scale errors in their everyday lives, with parents reporting an average of 3.2 scale errors per child over the 6-month period. Structured Observation
Researchers use more structured observation when they are ready to exert greater control over the
setting in which the observation occurs. Structured observations are often more useful when you have already identified and carefully defined the phenomena of interest. Advantages of Structured Observation
Structured observations can reveal the ways in which specific factors influence the context of interest in your research. Because well-designed structured observations involve a process of carefully defining the variables of interest and controlling the environment in which
these variables occur, they have the potential for both high external and construct validity. Disadvantages of Structured Observation
In order to effectively use and benefit from the structured observational approach, you need to be able to specify the dimensions where the structure will be applied. Depending on what you are observing, it may not be obvious where control should be applied. Example of Structured Observation: Scale Errors
Rosengren, Schein,
and Gutiérrez (2010) used a more structured observational approach in a different study of scale errors in young children. The structured observational design was intended to increase the likelihood of observing scale errors by placing miniature replica toys in the preschool classroom during pre-determined observational periods. This systematic approach revealed substantial individual differences in frequency of such errors decreased during the 10-week observation period. These discoveries would
not have been possible in the naturalistic parent report method used previously. Videorecording
Many observational approaches involve video recordings. However, the visible presence of recording equipment may increase the artificiality of the observational situation. Concealment of the observer or recording device and habituation by the subject to the observational situation are often the best solutions available. As a general principle, you can record without the consent of
those being recorded only if you are observing in a public setting; other types of recording require the explicit consent of those being observed. Recording data facilitates the use of multiple raters to assess levels of agreement on the coding of behaviors because it can be viewed repeatedly. Coding of Observational Data
Coding refers to taking qualitative information gathered during observation and distilling it into a form that is more readily amenable to statistical
examination. There are three broad types of sampling that relate to the process of coding: specimen record, event sampling, and time sampling. In a specimen recordapproach, all of the behaviors in a given time period are recorded. In this approach, it is essential to have access to technology like video recording to assure that nothing that occurs is missed. An event sampling approach involves recording all the instances of the behavior of interest within a specific time frame. In time
sampling, the observer records whether certain behaviors occur during a sample of short time intervals. Regardless of the type of sampling implemented to gather your data, developing a good coding system to deal with that data involves creating a careful set of categories that are both valid and reliable. Case Studies
A case study is a detailed examination of a single individual over a period of time. A case study can provide rich data about that unique circumstance, but it
cannot provide information that can necessarily be generalized to other situations or individuals. Instead, case studies can suggest themes and directions for future systematic study. Drawing Insight from the Exceptional Drawing of Nadia
A well-known case study involves Nadia, a young girl with autism, who at the age of 3½ began to create highly realistic drawings well beyond the ability of most children her age. Many researchers have used the case of Nadia to examine the
relation between autism and giftedness. When evaluating case reports such as these, you should consider whether the inferences are based on actual studies of the individual or drawn from reports of the original case study. The Memorable Case of H.M.
A famous case study in psychology and neuroscience is that of the patient H.M. When H.M. was 27, he underwent brain surgery as a treatment for debilitating seizures. However, the surgery also had the effect of rendering him unable to
remember any recent events. The many case studies of H.M. were instrumental in determining the hippocampus’s role in memory; the impact of amnesia on everyday functioning; and the relationship between memory and language processing. Advantages of Case Studies
Case studies provide incredibly rich data sets that can address interesting theoretical or applied research questions. In many circumstances, insights obtained from case studies would not likely have been obtained using
other types of research methods. Disadvantages of Case Studies
Drawing inferences from a case study to the more general population may not be valid. Case studies can also include both accurate and inaccurate information, and it may not be very easy to determine what information is the most accurate. In historical cases or cases studied by only a few researchers, you should consider both the validity and reliability of the reports, because they may be incomplete or biased by the
researcher’s expertise. Archival Research
Archival research refers to studies of data that have been previously collected. Many archival databases are available for use by any researcher interested in analyzing the data. More relevant to researchers in psychology is the General Social Survey (GSS), which contains demographic and attitudinal data. While this type of research may seem quite simple, to do it properly requires the creation of detailed coding schemes and long hours
poring over transcripts and other existing materials. Advantages of Archival Research
An advantage of archival research is that researchers do not need to collect the data themselves, which saves a lot of time and money. The nature of archival data also enables the researcher to obtain a relatively large, representative sample. Large samples enable investigators to use complex, sophisticated statistical approaches to test different models or theories against one another. Some
databases also enable researchers to access data that would be very difficult and costly to collect on their own and might be beyond the scope of what an individual researcher could collect in a reasonable amount of time. Disadvantages of Archival Research
A disadvantage of archival research is that your research questions are limited to the specific data that have been collected. Meta-Analysis
Meta-analysis involves summarizing and synthesizing past a body of
research to compare results across a number of studies. You could read the entire body of research and write a review paper that captures the overall conclusions from the individual reports. You could also count the studies that show one outcome, the studies that show the opposite outcome, and the studies that are inconclusive. Finally, you could opt for a third and more effective method by conducting a meta-analysis that compares the effect sizes across different studies. Effect size refers to
the strength of the relationship between or magnitude of the difference of two variables. When conducting a meta-analysis, researchers need to figure out how to classify and combine data from different scales that assess either similar or somewhat different variables, how to code data into meaningful categories that can be used in the analysis, and what to do with low-quality or missing data. Advantages of Meta-Analysis
You do not need to collect data to conduct a meta-analysis.
Meta-analysis also provides an objective way to compare findings across many studies. Researchers can sift through a large number of studies to determine an overall effect that, without the use of meta-analysis, might have remained unknown due to the variety of results reported in multiple studies. Disadvantages of Meta-Analysis
The quality of a meta-analysis depends on finding high-quality research reports, which may simply be lacking in some areas of interest. In addition,
only studies that find statistically significant effects tend to be published, leading to biased reports. Another disadvantage of meta-analysis is that it is time consuming to search for studies, code each study, and conduct the analysis.
What is the greatest benefit of naturalistic observation?
Naturalistic observation will allow the researcher to identify patterns and behaviors that may have been difficult or impossible to observe in a lab setting. This gives better insight into normal social behavior as subjects may react differently in a natural environment than when in a clinical setting.
What is the advantages of naturalistic observation when compared to surveys interviews and case studies?
It reduces the possibility of observer bias. It takes comparatively less time. Naturalistic observation is the cheapest method of data collection among all the other methods. It provides the first hand or fresh data, makes it more reliable.
What is the main advantage of the naturalistic observation technique quizlet?
The advantage of naturalistic observation is that it represents exactly what happens naturally. The primary disadvantage of naturalistic observation, however, is that it is not standardized, and so variables cannot be isolated.
How are case studies different from naturalistic observations?
Case studies, surveys and naturalistic observations are all descriptive research methods. Case studies analyse in great depth, whereas surveys analyse in less depth. Naturalistic observations like case studies and surveys only describe behavior without explaining it.
| 
