It’s not just about numbers: Language as data for child welfare
Child Welfare Initiative | 
Blog | 
4 mins. Child Welfare Initiative | Blog | 4 mins. 
Child Welfare Initiative | Blog | 4 mins. by Elisabeth S. Wilson, Senior Impact Analyst
While we often look at the numbers in child welfare to tell us how the kids are doing, there may be another source. Child welfare is full of conversations, discussions, and decisions all through word of mouth, and this language is often forgotten when calculating child welfare outcomes. A new technique called text analysis can unlock the day-to-day language used in the child welfare system to help us all truly understand children’s wellbeing.
Text analysis allows researchers to look through massive amounts of unscripted text without reading every line. In this study researchers used the reports of child abuse and neglect reported to the Indiana Department of Child Services.
After we upload the reports to the software, the program reads every word, assigns a positive or negative value to the word, and marks where it is in the narrative. This type of analysis answers questions like:
Published in Children and Youth Services Review, and led by Brian Goodwin and myself, “Text mining and the examination of language used to report child maltreatment: How language impacts child welfare intake reports” wanted to examine how language varied across child abuse and neglect reports. Specifically, we wanted to understand if language used to report child abuse and neglect differed by relationship to the child reported, in urban vs. rural communities, and by the reported child’s race. Researchers uploaded all the child abuse and neglect reports from 2019 across all 92 counties in Indiana to answer these questions.
These questions are pivotal to the field because 6 million children are reported to child abuse and neglect hotlines, but only about 656,000 children are determined to be victims. This means millions of families are subjected to an investigation by DCS that do not need DCS’s services. This also means agencies are spending valuable time and resources investigating families, when funds could be used to support children currently in care. Finally, across the country child welfare caseloads are high. Providing more effective techniques to evaluate the need for a child welfare investigation could support both family privacy and reduce caseloads.
The text analysis study found that this can be a great tool for child welfare if a specific child welfare dictionary could be developed. Currently, text analysis uses a regular dictionary to assign words a positive or negative value, but that doesn’t always work in child welfare. In addition, we found that using text analysis can show differences between counties on what is being reported. For example, counties vary in drug abuse allegations and between urban and rural jurisdictions. We did not find any substantial differences in language between children reported of different races, but our dictionary was not sufficient in the context of child welfare.
Identify upstream services. By using text analysis you can analyze a lot of data, without asking people to divulge personal information. Through looking at the allegations individuals are reporting across your state you can add specific services to support that community. In practice: leadership could see that county A is seeing a high number of alcohol abuse allegations, thus, specific services to support this addiction can be added/expanded to that area.
Add practitioner voices. Defining a child welfare dictionary will not be easy, and it won’t be useful without the voices of the field, and those with lived experience. Developing a dictionary is an opportunity to center the voices of the field to develop a tool for the field.
Evaluate how hotline policies impact reporting. By analyzing the frequency of words throughout the call you can see how hotline policies affect reporting. Indiana found that many policy questions asked at the end of calls were influencing the reporting of additional allegations. Through determining when allegations are reported throughout the call, can help intake workers define their questions to improve reports.
Gain additional data points. Language is data. All of the notes you take in child and family team meetings, supervisor meetings, and everywhere else is collecting data. Through taking clean notes you can contribute to data collection. Remember it’s not just numbers that lead to outcomes, it’s the voices of those in the system.