How to Rise to the Parenting Assessment Reliability Scoring Challenge
The research is clear, parenting is critical to a child’s successful development. That is why so many of you work so hard to promote high quality parenting in the families you serve. Since parenting is a major goal in family service programs, it makes sense to adopt a parenting assessment to guide services with families and to document their outcomes. While being trained to assess parenting, many family service providers tell us that careful observation of parent-child interaction is a very new experience for them, and that it really opens their eyes to valuable insights they hadn’t thought of before training. These insights then guide their work with families.
Parenting Assessment Scoring Challenge
Learning to use a parenting assessment tool reliably is challenging work, and afterward, if skills are not polished periodically, there is a natural tendency to drift away from accurate scoring. For example, in one instance we found that after just four months without feedback, 60% of a program’s staff required refresher training to regain reliable scoring on an observational parenting assessment. We find that the first time staff face annual recertification on the KIPS observational parenting assessment, it sometimes comes as a surprise that their scoring skills may have slipped a bit. It’s like speaking a foreign language. Once learned, you can do it; but to speak well and with confidence, you need to practice.
Likewise, observational parenting assessments require a little extra effort to keep your scoring skills sharp. That’s why top-notch supervisors have devised various ways to support their staff in maintaining accurate scoring skills on the KIPS parenting assessment. Elizabeth Willingham at Northern Virginia Family Services Healthy Families notes,“When recertification comes around, if you have not been focusing on the tool, you will find that staff have lost their skills. Staff need support to maintain those skills.” Let’s look at several strategies that have worked.
Parenting Assessment and Supervision
Some programs maintain accurate scoring by reflecting on the parent assessment results in individual supervision. When a supervisor occasionally views a parent-child video and compares her scoring with the home visitor’s scoring during supervision, it offers an opportunity to provide feedback to help retain accurate scoring. It also offers the opportunity to discuss interpretation of the assessment, which deepens the understanding of the parent’s behavior and enriches the design of individualized interventions.
At NORWESCAP Early Head Start, Alison Fennell schedules quarterly meetings with each home visitor to reflect on their most challenging families. They view, score and discuss the KIPS videos together. Alison finds, “This consistent review in supervision helps Home Visitors look at the full range of markers that describe KIPS ratings and feel more comfortable with assessing parents’ skills. . . . Together we reflect on the observed interaction, identifying the parenting strengths and the opportunities for growth. With an understanding of the strengths and opportunities, we move to planning the next steps with the family.”
Online Parenting Assessment Reliability
Since retaining reliability for a parenting assessment tool is so important, we built an online resource for ongoing support. The KIPS Library provides a searchable collection of parent-child videos with experts explaining the scoring of each item on the scale. Some programs have their staff log in to the Play Video Collection 3 or 4 times per year to observe the parent-child videos, score, and compare their KIPS ratings to the expert ratings. This approach encourages staff to bring any assessment questions they have to supervisors or co-workers throughout the year, rather than waiting until it’s time for annual recertification.
Parenting Assessment Reliability through Peer Support
At MyChild’sReady PAT, within the Child Crisis Center in Arizona, Erin Cowan took support a step further by developing a KIPS Template for Family Review to guide staff during peer supervision meetings in interpreting parenting assessment results and planning their interventions. After completing the template, staff members discuss their plans with their co-workers. This template is available for others to use in the KIPS Library Resources.
Parenting Assessment Reliability through Staff Meetings
Other programs maintain accurate scoring by incorporating practice on observation and scoring skills into their staff meetings. Videos can be selected from the KIPS Library Play Video Collection so staff can observe together and discuss their ratings, compared to the expert scoring. This approach may also bring up puzzling questions about parenting assessments done with local families. Katie Sykes, Child and Family Resource Center in North Carolina suggested, “It’s good to get together your Parent Educators…about once a month, a quick 15-20 minute session, to make sure that you are all on the same page. . . . It helps to have that sense of community . . . so you’re all looking at behaviors in the same way.”
Parenting Assessment Reliability through Role Play
Offering hands-on support for recertification, Tammy Smith at Kershaw County First Steps in South Carolina developed an in-depth review session. She explains, “In providing oversight to staff on becoming certified on KIPS, I had a staff that really struggled with the process. In speaking with her, she was able to get some of the items, however not all of them. I would ask her different questions about where she felt she was struggling and would try my best to explain the behaviors to her. This however did not seem to help, even though she would do okay on the practice test. In watching her view the videos and seeing how she scored them, I found that she had a general understanding of the behavior descriptions for the ratings, but was becoming confused when it came to decide the difference in a rating of 2 or 3/4 or 5. In thinking of ways that I could help her to better understand the difference in the ratings, I remembered that I myself am a visual learner, and do better when I can see what I am looking for. So I sat down with her and with another staff member who was certified and role played the different behaviors on the 5-point scale. I would show role plays of what behaviors you would see that would lead you to rate the parent’s behavior on the video as a 1 or 3 or 5, and then we discussed the differences. This seemed to help her understand the scoring a little better and she was able to obtain her certification.” After success in supporting my own staff, I was asked to do a training session as a refresher for staff in other SC programs who needed to be re-certified. For this group session, I videoed the role plays and created a PowerPoint slide presentation.”
Tammie’s slides for the group session reviewed the 12 KIPS items and ratings, with hints derived from the KIPS Library handouts, KIPS Behaviors in Detail in the workbook, and KIPS Coaching Tips. Role plays of two staff acting as parent and child illustrated ratings of 3 and 5 for selected items, such as Supportive Directions and Promotes Exploration/Curiosity, to help differentiate the ratings. Discussions of the role plays led up to independent scoring and group discussion of two of the more challenging parent-child play videos in the KIPS Library.
You work hard to promote high quality parenting in the families you serve. Programs adopt a parenting assessment to guide their services with families and to document their outcomes. Plus, learning to use a parenting assessment tool reliably is challenging work. Isn’t it worth a bit of extra effort to maintain those hard-won scoring skills?