Survey and Observational Assessments Not Correlated in CPS & Court-Involved Families: Implications for Services

In a previous KIPS Blog, you learned about the MOVE program, an innovative, 13-week mandated parenting program for families affected by intimate partner violence (IPV) and involved in the court or child protective service systems.  Practitioners working with these families and the courts influence life-changing decisions on parent-child visitation and custody.  However, this critical work is hindered by two major difficulties.  First, there is a lack of research on how to effectively support families who have experienced IPV.  Second, few of these decisions are based on reliable evidence regarding the strengths and needs of these families. This is why I’ve been eagerly awaiting the results of the MOVE program evaluation study.

Parenting Assessment Observations Enhance MOVE Evaluation

Initially, the evaluators of the MOVE program relied only on self-report questionnaires from the participating mothers to assess their parenting attitudes, skills and parenting stress. Because previous research has shown that self-reports are poorly correlated with actual behavior, they decided to add an observational parenting assessment, KIPS, to enhance their understanding of the parent-child relationships at entry and completion of the MOVE program. Paul Lanier and the research team at the School of Social Work at the University of North Carolina at Chapel Hill recently published some of their preliminary findings.  As part of their larger evaluation study, they examined the acceptance and feasibility of observing and filming the play interactions of mothers and their young children who were participating in mandated MOVE services.  In addition, they compared data from observational assessments to self-reports of parenting. 

At entry to the MOVE program, 24 mothers completed a packet of questionnaires on parenting, as well as a variety of other topics (e.g., demographics, mental health, intimate partner violence, substance use, stress and coping). The diverse group of mothers averaged 27 years old (range = 19-42) and were African American (46%), White (29%) and multiracial (21%), with 13% Hispanic heritage.  They reported educational levels as 17% less than high school/GED, 21% high school/GED, 33% some college and 29% completed college.  These mothers completed 3 parenting questionnaires: the Adolescent-Adult Parenting Inventory (AAPI), the Healthy Families Parenting Index (HFPI) and the Parenting Stress Index-Short Form (PSI-SF).

To observe parenting behavior, mother-child pairs (children under age 5 years) were filmed during 15 minutes of parent-child play, which were then assessed with the KIPS tool.  The researchers departed from the usual KIPS instructions to “play as you normally do”, by prompting the pairs through three 5-minute sequences of child-led play, parent-led play and clean-up time.  To decrease unease with an observer, the camera was set up in the play space and the researcher left the room during the three 5-minute sequences of play. 

What Did They Discover From Multiple Parenting Assessments?

Overall, the results showed relatively positive parenting by mothers affected by IPV as they entered the MOVE program.  The average KIPS score for the 25 mothers’ observed quality of parenting was 3.98 (SD = .66, range = 2.3 to 4.9).  This average score indicated moderately high quality parenting behavior on the KIPS scoring system where the maximum score is 5.  Because these scores reflect prompted parent-child play, they cannot be generalized to typical free play interactions.  

The research team concluded: “the KIPS measure is a useful source of information regarding system-involved mother-child dyads”   [Lanier, et al. (2016) Journal of Child Custody, vol. 13(1), page 45]

On the parenting questionnaires, mothers self-reported very positive parenting attitudes and skills, with less than 25% falling into the risk area on the subscales of the AAPI and less than 5% into the area of concern on the subscales of the HFPI. In striking contrast, on the PSI-SF, parenting stress scores for 100% of the mothers exceeded the clinical cut-off on the Parent-Child Dysfunctional Interaction subscale.

When comparing the scores for observed and self-reported parenting, only 4 of the 104 exploratory correlations were significantly associated, which is below what we might expect to see by chance alone. This is a clear indication of little association between observations of parenting behavior and mothers’ self-reports of their attitudes, skills and behavior, which confirms what others have previously found.

Lanier and his colleagues commented on their study’s parenting outcomes as follows:

“… Although these mothers were court- and/or CPS-involved and mandated to parenting services as a result of IPV, they had positive parenting skills, attitudes, and interactions with their children.  As a stark contrast, all mothers in this study were in the clinical range for parenting stress overall and stress related to the interactions with their children.”

I was surprised by how high the parenting quality was in this group of families. 

As the authors interpreted, perhaps:

“. . .  mothers in this sample have developed positive coping strategies that buffer the relationship between the internalized parenting stress and the behaviors and attitudes externalized in interactions with their child.”

[Lanier, et al. (2016) Journal of Child Custody, vol. 13(1), page 46]

Another explanation for the moderately high picture painted by observations of parenting behavior is that mothers who participated in this study were more confident in their parenting skills.

What Can We Learn About Assessing Parenting From this Study?

This admittedly small study offers valuable insights regarding parenting assessment of mothers affected by IPV who are involved with the courts or CPS system. It suggests that an observational parenting assessment is a promising strategy for collecting reliable and valid evidence regarding the parent-child relationship, as well as the parenting strengths and needs. This study also shows, as have others, that parent self-report instruments do not correlate significantly with validated observational measures.  This underscores the importance of using valid observational tools to inform the decisions and interventions with these families. 

One of the hurdles in observational assessments is the fear of low acceptance by court- and CPS-involved families. Such apprehension is understandable.  However, the UNC research team found that most mothers were willing to participate and have their parenting assessed by structured observation (see publication for details).  Considering the vulnerable life circumstances of these families, the fact that a majority of mothers chose to participate in observations of parent-child interactions is encouraging.  The research team also found that, given their traumatic histories, engaging families in the evaluation required heightened efforts to build trust with the families. Building rapport between the MOVE practitioners and well-reputed evaluators laid helpful groundwork. Finally, thoughful planning and resources were needed to offer tangible supports (e.g. transportation, childcare, convenient locations) for family engagement.

These preliminary results suggest that family support practitioners would do well to identify and reinforce vulnerable parents’ strengths, while acknowledging any changes needed to improve nurturance of their children.  Practitioners can then build upon the strengths to address the stresses of understanding individual children’s needs and emotions and managing challenging child behaviors that may be linked to the violence they witnessed in their families.

The study findings also confirm previous research that observational measures offer a different view of parenting than parents’ self-reports.  Of course, taking both perspectives into account can enrich service plans to match the needs and preferences of individual families.