Evaluating the Statewide Implementation of the FAIR System for Child Maltreatment Substantiation in Alaska

Child Protective Services (CPS) is responsible for keeping children and families safe through a variety of activities (e.g., receiving, investigating, and making substantiation determinations about allegations of child abuse and neglect; monitoring and managing risk; connecting families with resources to reduce risk and promote child welfare). CPS workers play a critical role in keeping children safe, yet the decision-making systems they rely on to guide their actions are prone to inconsistency across providers and locations (e.g., Fluke et al., 2020; Font et al., 2020; Munro, 2019). Varying definitions and policies regarding maltreatment (Day et al., 2022; LaBrenz et al., 2023) and standards of proof used (Kahn et al., 2017) contribute to location-based disparities. CPS workers acknowledge inconsistency as a significant issue and identify a lack of definition clarity and information gaps as key contributors (Fluke et al., 2020; Mitnick et al., 2016).

Research has documented contributors to these inconsistencies, including family income (e.g., Drake et al., 2023), CPS workers’ perceptions of parental cooperation (e.g., Bartelink et al., 2015; Munro, 2019), and the amount available for investigation and assessment (e.g., Fluke et al., 2020). Although older studies had found differences due to race, more recent U.S. national studies have failed to find them (e.g., Drake et al., 2023; Kobulsky et al., 2021; LaBrenz et al., 2023), perhaps due to improvements in CPS decision-making. Often, the contributors are cognitive factors of which workers are not consciously aware. For example, CPS caseworkers’ decisions are influenced by common human cognitive biases and heuristics, such as confirmation bias that discounts evidence contrary to first impressions (e.g., Munro, 2019), overconfidence (e.g., Benbenishty & Fluke, 2020), and making intuitive decision-making (Bartelink et al., 2015).

These human decision-making errors affect CPS system judgments and subsequent actions. For example, interrater reliability — or the extent to which trained individuals agree on decision-making when given the same information — regarding caseworkers’ risk assessments, decisions about foster care placements, and even agreement on substantiation of sexual abuse allegations, is strikingly low (e.g., Bartelink et al., 2015; LaBrenz et al., 2023). This indicates that individual caseworkers make key determinations differently and without uniformity throughout agencies. Previous studies of substantiation decisions (e.g., Heyman & Slep, 2006) have shown a chance-like agreement between typical field decisions and those of master reviewers.

These inconsistencies limit the reliability of substantiation decisions and open them up to influence by extraneous factors. When any factor influences a decision beyond the facts of the incident, unfairness is introduced for the children and families CPS is charged with protecting and serving. When the line of substantiation is faint or movable, children who need protection are likely not to receive that protection—that is, when cases ought to be substantiated but are not. Conversely, parents may be inappropriately labeled as maltreating—that is, when cases ought not to be substantiated but are (Benbenishty & Fluke, 2020). Unsurprisingly, inconsistent decision-making has been found to foster perceptions of unfairness and capriciousness by families and community agencies who interact with CPS (e.g., Benbenishty & Fluke, 2020; Drake et al., 2023; Fluke et al., 2020; Font et al., 2020; LaBrenz et al., 2023). The lack of decision-making structure and guardrails fosters unfairness that undermines CPS’s ability to achieve its mission of protecting children and preventing child maltreatment.

In part due to the unreliable nature of such determinations, some advocate moving away from dichotomous decisions (i.e., maltreatment vs. no maltreatment) and instead focusing on risk and service needs (e.g., Font et al., 2020; Fuller, 2022). Proponents argue that this “differential response” would lessen CPS’s potentially adversarial role and divert energy from decisions that do not predict recidivism and toward more effective intervention plans (Fuller, 2022), although the data on this are mixed (Sedlak & Evans, 2022). However, states are legislatively mandated to focus on identifying and tracking the prevalence of substantiated abuse (Child Welfare Information Gateway, 2019) and making those determination decisions. In addition, states need to (a) make fair and consistent decisions, (b) communicate those decisions to families, and (c) educate the community regarding the line between discipline and maltreatment. However, previous efforts at structured decision-making that focus on summarizing the risk of future maltreatment have been criticized for poor variable choices and weak prediction of future reports (Feely & Bosk, 2021).

Development of a Consistent Determination System

Given that inconsistency in substantiation determinations is a substantial problem that undermines effective policy and treatment, Heyman and Slep began a program of research in partnership with the United States Air Force Family Advocacy Program (FAP) over 20 years ago. The research program aimed to develop a system that would enable caseworkers to make consistent, fair substantiation determinations (Heyman & Slep, 2006, 2009; Slep et al., 2023; Slep & Heyman, 2006). FAP serves a similar function in military communities as CPS does in civilian ones, receiving reports of family maltreatment, making determinations about whether those allegations are over a threshold, managing risk, and providing families with appropriate services. There are a few notable differences between FAP and CPS. FAP handles both partner maltreatment and child maltreatment and is staffed by Masters-level, credentialed social workers, which is not always the case in CPS. Finally, substantiation determinations within the military are made by interdisciplinary committees, whereas in CPS, they are often made by social workers and their supervisors (Military OneSource, n.d.).

This research program included multiple stages. The first was an evaluation of the content validity of then-in-force Department of Defense’s (DoD) definitions for physical, emotional, and sexual abuse and neglect of children and spouses, as well as field interviews with clinicians who had vast experience using those definitions (Heyman & Slep, 2006, Studies 1 and 2). Results indicated a need for detailed operationalizations of terms, creating clear delineations between forms of maltreatment, focusing on assessing acts/omissions and their impacts, and stipulating the standard of proof (i.e., preponderance of information).

Heyman and Slep then created criteria for deciding maltreatment allegations. They split the then-in-force criteria into act and impact components and conducted an extensive literature review of theoretical and empirical articles, federal and state laws and legal guidance, working group reports, and coding manuals for maltreatment criteria. They then conducted focus groups with field social workers and field-tested the criteria on over 500 cases, iteratively refining them (Heyman & Slep, 2006, Study 3).

During a second field trial (Heyman & Slep, 2006, Study 4), the university-Air Force partnership (a) implemented a structured assessment that corresponded to the criteria, (b) modified the committee structure, and (c) instituted a computerized decision tree to guide decision-makers through criterion-by-criterion judgments. Taken together, these changes resulted in rates over 90% agreement between master reviewers and committees (Heyman & Slep, 2006, Study 4), which was retained when the system was disseminated to Air Force sites worldwide (Heyman & Slep, 2009). The substantiation process was perceived as fair to alleged offenders and victims by FAP social workers and other voting stakeholders (Heyman et al., 2010; Slep & Heyman, 2006). Furthermore, rates of recidivism were greatly reduced; one-year re-offense rates for those who had met criteria initially were cut in half (Snarr et al., 2011), thus suggesting a tertiary preventive effect of a fairer and more consistent system. Subsequent studies in the U.S. Army replicated the findings regarding fairness (Heyman et al., 2022), decreases in recidivism (Slep et al., 2023), and local sites’ high agreement with master reviewers (Zaninovic et al., 2025).

The multi-component approach — comprising assessment, training, criteria, and a computerized decision tree — is known as the FAIR (Field-tested Assessment, Intervention-planning, and Response) System. The criteria were subsequently adopted by all military services, the Diagnostic and Statistical Manual of Mental Disorders, 5^th edition (American Psychiatric Association, 2013), and the International Classification of Diseases, 11^th edition (World Health Organization, 2020).

Adopting FAIR for Alaska’s Maltreatment Assessment

In 2014, the Office of Children’s Services (OCS) in the State of Alaska instituted the first statewide implementation of the FAIR System, which they named the Maltreatment Assessment Protocol^[1] (Alaska Office of Children’s Services, 2015). OCS adopted the maltreatment criteria in the FAIR System, built them into their regulations, and aligned them with state statutes. OCS then created a civilian-system adaptation of the FAIR System training and built the computerized decision matrix into their statewide software. Finally, OCS formatted a written version of the criteria in the FAIR System, printed color copies, and distributed them to all workers and supervisors.

The aim of this study was to conduct an implementation science investigation of the perceptions of the OCS staff charged with implementing Alaska’s adapted version of the FAIR system. We assessed their views on the fidelity of usage, utility, and impact of the FAIR system approximately one year after implementation using a mixed-methods approach, comprising focus groups, interviews, and questionnaires.

Study 1: Online Survey

This evaluation component was approved by Penn State University’s Institutional Review Board and conducted by an independent research team unaffiliated with OCS operations.

Method

Participants

Participants were 54 staff members from OCS (e.g., Family Services Workers, Incident Assessment Workers, Supervisors, and Management from headquarters). Participants were mostly female (87.0%). More than two-thirds of the respondents were 35–54 years old (13.0% 18–34 years; 37.0% 35–44 years; 29.6% 45–54 years; 18.5% 55 years or older; 1.9% unknown), and 74.1% were white (1.9% Black, 7.4% Alaskan Native, 5.6% Hispanic, 7.4% other race, 1.9% unknown). Approximately 44% of the respondents were (a) family service line workers, who serve the ongoing needs of families in the system, followed by (b) initial assessment workers (20.4%), who conduct interviews to determine the safety of the child and risk of maltreatment when OCS receives reports of alleged maltreatment, (c) initial assessment supervisors (20.4%), who oversee the initial assessment workers, and (d) family service supervisors (14.8%), who oversee the family service line workers.

Measures

A study-specific measure of staff’s perceptions of constructs critical to initial implementation — (a) fidelity of usage and (b) utility and impact of the FAIR system — was drafted by an implementation science expert at Penn State (DFP); implementation science experts at NYU (AMSS and REH) then independently reviewed the measure, with 100% agreement that the items were face and content valid.^[2] Subsequently, OCS officials reviewed them and concurred on the face and content validity regarding the Alaska implementation. The full measure can be found in Online Supplement 1.

Fidelity of usage. Participants rated how often they used and accepted aspects of the FAIR system (e.g., “How often do you use the relevant criterion in [the FAIR system]?” “How often do you accept the final verdict of [the FAIR system]?”) on a 5-point scale (0 = “Never” to 4 = “Always”). The average of the 7 items constituted the Fidelity of Usage score (Cronbach’s α = 0.89).

Utility. Participants rated the usefulness of various elements (e.g., FAIR system criteria, decision of each criterion, risk or threat of harm) as a worker/supervisor to make the maltreatment determination. Responses were on a 5-point scale (0 = “Not useful” to 4 = “Extremely useful”). The average of the 6 items constituted the Utility score (Cronbach’s α = 0.95).

Impact. Participants rated the impact of FAIR-system usage on various outcomes (e.g., “To what extent has the use of [the FAIR system] impacted…the consistency of your decision-making to determine maltreatment?” “…the perceived fairness by the families?”). Responses were on a 5-point scale (0 = “Not at all” to 4 = “A great deal”). The average of the eight items constituted the Impact score (Cronbach’s α = 0.86).

Procedure

Staff from each OCS unit were invited via listservs to participate in a 28-item online survey (see Online Appendix 1) about Alaska’s version of the FAIR system. Participants were recruited by the evaluators and were informed that the information regarding (a) participation and (b) the responses they gave was only available to the evaluation team, and OCS would not have access to it. Thus, participation was voluntary, and there was no penalty for not participating. The survey link remained active for four weeks post-distribution, and reminder emails were sent at two and three weeks after the invitation. Participants provided consent for the online survey by clicking an option on the introductory page or by submitting the survey.

Results

Descriptive statistics were calculated to summarize the central tendency and variability of the study variables—fidelity, utility, and impact.

Fidelity of usage. Among initial assessment supervisors, use of the FAIR system or its language was reported most of or all of the time in 82% of respondents, compared with 50% of initial assessment workers, 43% of family service supervisors, and 44% of family service line workers. Only 9% of initial assessment supervisors, 20% of initial assessment workers, 14% of family service supervisors, and 39% of family service line workers reported “never” or “sometimes” using the FAIR system or its language. On average, initial assessment supervisors reported using the FAIR system or its language most of the time, and other respondents reported using it about half of the time (see Table 1). Thus, the majority of respondents reported using the FAIR system or its language most of the time.

Utility. Among initial assessment supervisors and family service supervisors, the FAIR system was perceived to be very to extremely useful by 64% and 60% of respondents, respectively, compared with 40% of initial assessment workers and 50% of family-service line workers. Nine percent of initial assessment supervisors, 30% of initial assessment workers, 20% of family service supervisors, and 13% of family service line workers reported that the FAIR system was not or only slightly useful. On average, initial assessment supervisors reported that the FAIR system was very useful, and other respondents reported that it was moderately useful (see Table 1). Similar to the finding on fidelity, most respondents reported that the utility of the FAIR system was high.

Impact. Among initial assessment supervisors and family service supervisors, the FAIR system was perceived to have had “quite a bit” to a “great deal” of impact by 64% and 60% of respondents, respectively, compared with 33% of initial assessment workers and 47% of family service line workers. Only 18% of initial assessment supervisors, none of the initial assessment workers, 20% of family service supervisors, and 20% of family service line workers reported that the FAIR system had no or very little impact. On average, all respondents reported that the FAIR system had somewhat of an impact, and most respondents reported that the FAIR system had quite a bit of impact (see Table 1).

Study 2: Focus Groups

The focus group evaluation component was approved by Penn State University’s Institutional Review Board and conducted by an independent research team unaffiliated with OCS operations.

Method

Participants

Sixteen face-to-face focus groups (n = 36 participants) were conducted from three large regional offices (Wasilla, Anchorage, and Fairbanks) with OCS professionals (20 IAWs, 8 FSW, 5 Supervisors and Managers, and 3 OCS headquarters members) and 3 Administrative Law Judges. Six telephonic focus groups were conducted with professionals (n = 12) from distant regional OCS offices and with Assistant Attorneys General. (Management from state OCS headquarters was not a part of the telephonic groups to promote openness.)

Procedure

Focus groups were limited to 6 participants and were conducted separately with professionals with specific roles (i.e., Family Services Workers, Incident Assessment Workers, Supervisors, leadership at state OCS headquarters). Consent for the focus group was provided before the group meeting (via email or phone) when the participant was scheduled to participate, and again at the start of the focus group by the facilitator. Focus groups were approximately 60 minutes. Notes were taken, but no recordings were made, and no names were captured. The investigators recorded reaching saturation when the last day’s focus groups produced no new information. All focus groups were facilitated by two senior researchers. Telephonic focus groups included an additional notetaker. After each focus group, facilitators checked notes for completeness and comprehensiveness (e.g., writing out points that were abbreviated when taking notes). All notes were compared immediately after each focus group to ensure clarity and accuracy.

Facilitators used a structured set of questions to guide discussions. Participants were asked about the amount of time they spent using the FAIR system, what facilitated its launch and implementation, what some challenges were, what could improve its usability, and its impact on families and workers. (See Online Appendix 1 for the full list of questions.)

The final author’s team at Penn State completed coding to maintain independence from the FAIR developers at NYU, one of whom (AMSS) was among the focus group facilitators. Coding was based on a Grounded Theory approach (Charmaz, 2006; Thornberg & Charmaz, 2012). The first step is identifying primary themes. All members of the coding team read all notes from all notetakers and met to discuss themes. Once consensus on the list of themes was finalized, formal coding was conducted. A minimum of two coders read and coded each transcript. Any coding discrepancies were resolved via discussion with the coding group as a whole. Themes that were identified were as follows: (a) the FAIR system was being heavily used during initial assessments and when making substantiation decisions; (b) the FAIR system was viewed as having a positive impact in several ways, especially with initial assessment workers and supervisors; and (c) targeted additional training sessions could further improve implementation.

Results

The results are summarized in Table 1 and described in detail below.

The FAIR system is heavily utilized. IAWs consistently reported using the FAIR system daily. Some reported that using the FAIR system initially presented a challenge due to its comprehensiveness, but that the provided examples were helpful, and once they used it, they felt like it “connected the dots.” One worker reported that she had “been using it for 9 or 10 months now and [she had] used it for about 400 to 500 cases in that time.” Many IAWs were able to quote from the FAIR system without looking at it, and all knew where their laminated copy was located and reported taking it with them when conducting in-home assessments and using the computerized version as standard practice.

Initial assessment supervisors were also quite familiar with the FAIR system, again often being able to quote the criteria from memory. They reported consistently reviewing cases with the IAWs using the FAIR system as a framework for discussion when making case determinations. They reported that it had become standard practice to incorporate the specific language from the FAIR system in the write-up of decisions.

Family service line workers were less familiar with the FAIR system and noted that they rarely use it in the context of ongoing case management. Those who addressed new allegations had used it and had some familiarity with it. Those who had worked with the FAIR system had said it was helpful in that it provided a more structured way to approach determinations, and that that was useful given that making determinations was rarely part of their work with families. Family service supervisors indicated a clear understanding and utility of the FAIR system.

Staff perceived the FAIR system as helpful. The overwhelming majority of workers using the FAIR system reported that it was extremely helpful to them in (a) conducting a thorough assessment, (b) providing consistency across the agency, and (c) establishing objectivity in the process. More specifically, IAWs and their supervisors stated that the FAIR system provided a framework for structuring their assessments. The FAIR system was described as “very useful,” “enormously helpful,” and providing “an ah-ha moment.” One worker stated, “It helps me make decisions with more confidence…Things are clearer and I can use the tool and the language in it to make the decision clearer.” Participants mentioned it helped “organize their thoughts” when it came to isolating critical aspects of an assessment. Supervisors expressed that the FAIR system facilitates staff achieving a high standard in assessment and that the examples provided in the FAIR system offer concrete direction. Assessment workers who were relatively new to OCS said that the FAIR system helped clarify portions of their jobs. IAWs and their supervisors noted that using the FAIR system gave them more confidence that determination decisions were based on a systematic examination of the facts. The FAIR system was likened to “a medical algorithm for treatment.” A supervisor noted that the FAIR system “allowed a lot of stress relief” for workers through the increased structure it provided.

An additional theme was that the FAIR system improved IAWs’ documentation and allowed them to provide clear, criterion-based justifications for decisions to substantiate. A supervisor commented that her workers, even new ones, could easily explain why cases were substantiated and document “specific reasoning for their decisions.” Another noted that the FAIR system “tied the findings directly to the statutes, which makes it less of a judgment call…It allowed the staff to make more concrete decisions and explain and cite why they were able to make this decision.” In addition to improving documentation regarding decisions, participants noted that the structure of the FAIR system led to more comprehensive notes, and that these clearer notes, in turn, facilitate case reviews.

Administrators noted that another strength of the FAIR system is that its language (a) provides clarity, which can be especially useful if a decision is appealed, and (b) is understood by parents and judges. This facilitates communication regarding consistency and OCS procedures and policies. One administrator aptly noted, “Now every point is getting a review based on the law and not perception, feeling, and gut.”

Administrators noted that the implementation was straightforward and efficient, and that substantiation rates had been evened out across different teams and different regions of the state. They attributed this finding to the FAIR system solving a problem of consistency with some teams having more stringent thresholds for substantiation that persisted over time and could be seen in routinely collected statistics, but that training had failed to even out. However, administrators did note the challenge of mastering all FAIR criteria without daily use and that their supervisors would benefit from greater familiarity with the less-often-employed branches of the FAIR system.

Assistant attorneys general also noted the FAIR system had led to several visible improvements within their system. They stated that the work products received from OCS were of higher quality than before the introduction of the FAIR system. For instance, all assistant attorney general participants noted that when the workers use verbiage from the FAIR system in their documentation, the decisions were clearer and the cases were handled better. One participant noted that documentation “used to be about feelings, and workers were all over the place. [With the FAIR system], they see [the case] more clearly and more concretely and articulate findings more clearly.” The administrative law judges saw the FAIR system as bringing the necessary consistency to OCS processes.

When specifically asked about concerns regarding using the FAIR system, experienced IAWs said that the FAIR system’s structure and the required assessment had the potential to lead to an inappropriate narrowing of IAWs’ focus (i.e., missing other important information that would be relevant to understanding risk or future needs). However, they believed this concern was outweighed by the FAIR systems’ benefits and that the inappropriate narrowing could be addressed sufficiently through supervision and training. Administrators noted a few areas in which the FAIR system could be better aligned with other regulations. They also noted that because of the FAIR system’s complexity, mastery cannot come from a single training.

Family court judges expressed concern that although Alaska OCS and the assistant attorneys general had worked to ensure the FAIR system was consistent with the existing statutes, the statutes had not, in fact, been altered. This is exacerbated by the fact that two sets of statutes cover child maltreatment in Alaska, and the FAIR system is more aligned with one of them. Thus, there were times when something might be inconsistent with the FAIR system but still fall within the scope of the statutes. Judges thought that additional efforts to more completely harmonize law and the FAIR system, as OCS policy, would be helpful. They also thought that many of the FAIR system’s more complex portions that help it fit with the law were not yet consistently mastered across OCS and would benefit from additional training.

Participants had suggestions for enhanced training. When participants were asked about possible areas for improvement, all suggestions focused on training and implementation rather than the FAIR system’s criteria. Suggestions for trainings included a standalone skills training session and ongoing training activities. The standalone training was recommended to focus on the theoretical underpinnings of the FAIR system, the evidence that it was needed in Alaska, and how the FAIR system met that need. Furthermore, this training should include a detailed review of the relevant state statutes and then guide workers through applying the FAIR system to complex cases. This is especially relevant because relevant definitions appear in two chapters of the Alaska statutes (i.e., Alaska Child in Need of Aid Statute, 2015; Alaska Child Protection Statute, 2014) and although the Attorney General’s office reviewed the FAIR system before its implementation to ensure it was fully consistent with Alaska law, if workers are not certain how to convey that in their written summaries, then the judicial system is limited in its ability to infer the information they require to make optimal decisions.

Ongoing trainings were also recommended. Some thought that OCS should have additional online, scenario-based trainings with competency quizzes. Another training need was a just-in-time training tool that family service workers could use, as they usually do not use the FAIR system or conduct assessments. Thus, the tool would enable them to refresh their skills when the need to use the FAIR system arises.

Universally, the greatest training need noted concerned applying the impact criteria of the FAIR system, most notably, how to assess and document psychological distress and psychological and developmental impacts. IAWs reported that they had limited expertise in determining the impact of non-physically injurious acts and omissions by caregivers. Although psychological and developmental consequences were included in the FAIR system, the IAWs reported that the assessment of those consequences was beyond their expertise without more training. The Administrative Law Judges believed that more training was needed for the less frequently used portions of the FAIR system that were critical to syncing determinations based on FAIR with the statutes (e.g., exclusions, less frequently documented impacts such as psychological and developmental consequences).

Discussion

Through these evaluation components (i.e., surveys and focus groups), quantitative and qualitative findings paint a steady picture of the impact of this first civilian, statewide implementation of the FAIR System. The FAIR system significantly influenced OCS’s initial assessment systems, with widespread adoption across the agency. Moreover, the findings point to growing confidence in the OCS operations. Indeed, some parts of the FAIR system have quickly become established within Alaska’s determination systems, and this is consistently seen as having a positive impact on OCS operations.

Supervisors use the FAIR system protocol slightly more than their workers do, with widespread utilization generally. This pattern of findings is consistent with what one might expect from a strong initial implementation and is encouraging about the ability to implement such a protocol elsewhere. As supervisors continue to support implementation, FAIR-system usage will become further established, and the parts of the FAIR system that are relevant to only a subset of cases will gradually become more familiar. Nevertheless, quality assurance and ongoing training are required.

Similarly, our findings suggest that the adoption of the FAIR system was perceived by staff as beneficial (e.g., for standardization and objectivity). Quantitatively, participants rated their perception of the FAIR system’s impact as quite positive. Focus group findings elaborated on that impact. Participants shared that the FAIR system increased confidence in workers’ decision-making, provided a consistent rationale for substantiating (and not substantiating) allegations of abuse and neglect, improved clarity, thoroughness, and purposefulness of assessments, increased uniformity across the agency, as well as increased objectivity in assessments and decision-making. Participants reported that as a result of the adoption of the FAIR system, case closures occurred more efficiently because of the straightforward protocol being applied to decisions. Future research could extend these findings based on stakeholder perceptions by empirically testing whether the FAIR system has a measurable positive impact on workers (e.g., burnout; Stanley et al., 2025), systems (e.g., faster case closure), and families (e.g., reduction in reoccurrence; Slep et al., 2023; Snarr et al., 2011).

It was particularly encouraging that judges perceived that the information they received was more objective and evidence-informed. From their perspective, the system supported case workers in clearly gathering information and articulating findings, such that judges could follow and make decisions more easily and confidently. This is promising because it may mean that judges’ decisions, too, are more reliable, and that relations and trust between these aspects of the welfare system would be improved. Again, these possibilities would need to be empirically tested.

This evaluation of Alaska’s efforts provides clear direction as to what is required for a successful implementation of the FAIR System. The process findings are supported by implementation research from the prevention science field regarding the need to attending to the myriad implementation processes to ensure high quality and relevance, including a balance among the fidelity, adaptation, tailoring, and cost-effectiveness of the procedures and processes (Bumbarger & Perkins, 2008; National Academies of Sciences, Engineering, and Medicine, 2019).

Other states contemplating adopting the FAIR System should consider some important lessons from Alaska OCS’ successful implementation. First, thoroughly engaging and coordinating with all key stakeholders is essential for both strong buy-in and faithful FAIR System implementation (thereby avoiding the need to do additional testing). Second, OCS built a parallel computerized decision tree that mirrored the Air Force computer system, including the detailed definitions. Third, OCS developed a standardized training with case examples coupled with systematic launch trainings. Fourth, OCS established an accountability process by regularly checking in with supervisors and actively monitoring implementation to ensure quality. This careful attention to supporting implementation enabled OSC to achieve the positive impacts of the FAIR system, similar to those found in the Air Force and Army investigations.

As with all systems being implemented in new settings for the first time, the implementation of the FAIR System in Alaska highlighted a critical difference between the workers in the military and those in Alaska, and in turn, many other child protective agencies. The FAIR System includes criteria that operationally define psychological distress, fear reactions, and developmental disruptions. The language used was developed on and tested with military family protection workers. In all branches of the U.S. military, these workers are required to be credentialed providers and usually have a master’s degree in social work. During the focus groups, we found that IAWs in Alaska struggled the most with these elements of the FAIR system criteria and required more support and training around these elements, due to the differences in training and education. To address the identified challenges in applying psychological and developmental impact criteria, state implementations should develop structured trainings modeled after those our team has designed for the military to address similarly complex constructs. These trainings should include a combination of delivery formats to meet varying learning needs and implementation contexts and may include (a) asynchronous interacting online modules that guide practitioners through common errors in assessments and reports, offering examples and corrective strategies; (b) topic-specific PowerPoint presentations that focus on nuanced issues, with voiceover narration and integrated quiz questions; and (c) synchronous virtual or in-person workshops that use simulated scenarios and deliberate practice (e.g., Ericsson, 2018) to build skills in applied decision-making and clinical judgment. A staged rollout over several months, beginning with foundational asynchronous content and followed by live skills-based sessions, facilitates scalable implementation while developing, reinforcing, and maintaining key competencies over time.

As for implications for other states, Alaska OCS’s experiences are relevant to other large systems considering the implementation of FAIR. Before any level of implementation, they gathered key persons throughout the state systems to discuss and engage different agencies in the transition. They planned and prepared carefully, ensuring that training materials and automation were ready before launching the FAIR system. They partnered with their local resources to incorporate their perspectives and expertise without modifying the core of the FAIR system. They encountered. but were ready for, barriers, such as the tremendous workload carried by supervisors, many of whom were so busy that they were unavailable for more than minimal training before launch. However, the consistency of the training materials and the automation of the FAIR system itself took hold and supported the shift.

Next steps include implementation in other states’ child welfare systems. Future implementations should consider the match of the FAIR System with their workforce and use Alaska’s suggestions for training enhancements to both initial and ongoing training. Training activities and their impacts should be documented to create a package of the minimally sufficient activities that must be executed to achieve positive effects in child welfare systems adopting this protocol.

Second, research is needed to test whether the FAIR System reduces repeat offense in civilian communities as it has been documented to do in military communities (Slep et al., 2023; Snarr et al., 2011).

Third, studies are needed to investigate the degree to which the FAIR System, a structured substantiation process with operationalized criteria, reduces extraneous bias in substantiation determinations. Within the military, Heyman, Slep, and colleagues (2016) found that the FAIR System was applied without bias on multiple factors (e.g., officer/enlisted status, military/civilian status, gender). Perceptions of Alaska’s OCS staff imply less systematic bias in decisions after the FAIR system’s implementation (i.e., substantiation rates became more consistent across different teams and jurisdictions). A recent investigation of trends in CPS outcomes from 2005–2019 indicated no current differences for substantiation or placement for Black and Hispanic children (Drake et al., 2023). Given the importance of this question, the considerably different racial makeup of Alaska (e.g., higher proportions of American Indian or Alaska Native and Asian/Pacific Islander populations compared with the continental U.S.), and the implementation of FAIR, quantitative outcome research is necessary to investigate its impacts on disparities in Alaska’s decisions. Future researchers may consider scrutinizing these data through a structural lens (e.g., Feely & Bosk, 2021) when investigating the FAIR system’s impact on racial/ethnic outcomes. Although the FAIR system is not a risk predictive, structured decision-making model (e.g., Schwalbe, 2008) criticized by Feely and Bosk (2021), it operates within a broader child welfare system that may still differentially impact historically disenfranchised subgroups. The system’s focus on individual acts and impacts, while improving consistency, does not address upstream factors rooted in systemic racism — such as poverty, housing instability, and differential surveillance of communities of color — that can lead to disproportionate referrals and substantiation (e.g., Feely & Bosk, 2021). Thus, future evaluations might employ a racial equity audit to assess whether FAIR may perpetuate racial/ethnic inequities (Dettlaff & Boyd, 2020; Feely & Bosk, 2021) despite its intent to increase fairness. For example, poverty, environmental dangers, and housing instability — often rooted in historical systemic racism (e.g., Dettlaff & Boyd, 2020; Feely & Bosk, 2021) — may disproportionately lead more Alaska native or other minority groups to meet criteria for child neglect. If this were found, a critical examination of the connections between those conditions and individual behaviors would be necessary to focus response not only on families’ behavior but also on the underlying structural causes of maltreatment (Feely & Bosk, 2021). In essence, although the FAIR system has achieved more consistent decision-making in past research (e.g., Heyman & Slep, 2006, 2009; Zaninovic et al., 2025) and was perceived to increase consistency in Alaska, achieving true equity also requires attention to the structural conditions that may create differential exposure to child welfare involvement across racial and ethnic groups (e.g., Johnstone, 2020.

Strengths and Limitations

This evaluation has two major strengths. First, it represents a real-world, statewide implementation directed by child maltreatment policy makers and members of the state attorney general’s office, as opposed to external researchers. This has positive implications for the portability and usability of the FAIR system across state-level CPS. Second, both the breadth of stakeholders and the mixed-method design of this evaluation foster confidence in the validity of the findings.

This study also has some limitations. First, the evaluation only measured stakeholders’ perceptions of the system and its impact. Additionally, participants may have responded in ways they believed evaluators or supervisors would find favorable, potentially inflating positive perceptions of the FAIR system’s effectiveness. To determine if FAIR led to objective improvements in the consistency and accuracy of workers’ and supervisors’ decisions, one would have needed to experimentally introduce the FAIR system to randomly assigned teams, which was not practical. Such research should be considered in future state implementations. Quasi-experimental approaches could also be pursued, examining changes in maltreatment recurrence, appeals, or substantiation rates before and after FAIR-system implementation. Such an investigation of Alaska’s administrative data was beyond the implementation focus of this paper but would be possible.

Second, the evaluation was cross-sectional and involved retrospection about the implementation. Future studies should collect data both before and after implementation.

Third, although participation rates in the interviews and focus groups were good, stakeholders who participated may have had different perspectives than those who decided not to participate (i.e., self-selection bias). This finding is consistent with the Leverage-Saliency Theory of Survey Participation (Groves et al., 2000), which posits that people participate through internal cost-benefit analyses that emphasize the research topic’s personal importance (i.e., leverage) and perceived relevance (i.e., salience). Given that the FAIR system implementation impacted the daily work of most stakeholders invited, it would be a highly salient attribute for a research invitation. These strong opinions about FAIR — whether very positive (e.g., perceived improvements in consistency and fairness) or very negative (e.g., perceived challenges or disagreements regarding its adoption) — would be highly influential in decisions to participant. Alternatively, stakeholders with more neutral opinions might have less pull to participate. Thus, our mixed-methods results may have overrepresented the more extreme ends of the opinion spectrum.

Fourth, Study 1’s questionnaire was developed for this study to assess initial perceptions of the implementation. It was rated as face and content valid. The high internal consistency values of each subscale strongly imply that each subscale is unifactorial, although further testing with exploratory factor analysis is needed to document their underlying structure. A clear limitation is that the scales have not been tested for construct or criterion validity (Cook & Campbell, 1979); thus, researchers looking to replicate our findings should consider further psychometric testing. Fifth, the focus groups were conducted face-to-face in Alaska’s three largest regional offices (Wasilla, Anchorage, and Fairbanks), with distant regional offices represented through telephonic sessions, which may have influenced the depth and quality of responses compared with face-to-face sessions. Sixth, because we did not collect data about the racial or ethnic background of OCS staff of participating versus non-participating staff, we cannot determine if our sample was representative of the broader OCS workforce. Finally, given that the FAIR system aims to reduce systemic bias, future studies should test how (a) worker and family race/ethnicity and (b) region impact substantiation trends before and after implementation of the FAIR system.

Conclusion

The FAIR system was initially developed when a military family protection agency was concerned that its substantiation decisions were not reliable — that is, the same incident that met criteria at one site would not meet criteria at another site, or an incident that met criteria at one meeting would not meet criteria at another meeting at the same site. The agency’s concerns were not military-specific or maltreatment-specific, as unreliability is endemic in every field of human judgment (Kahneman et al., 2021). However, unreliability is not immutable. Agreement between local-community committees and expert master reviewers went from 50% (chance agreement, no better than a coin flip) to over 90% (chance corrected κ=.80 –1.00; Heyman & Slep, 2006) and stayed above 90% even when disseminated worldwide (Heyman & Slep, 2009; Zaninovic et al., 2025). Alaska’s OCS leaders read these research findings and wanted to implement the system in their child protection agency; any human services domain (e.g., behavioral health, education, juvenile justice) could adopt a similar, evidence-informed approach to developing, field-testing, and disseminating an evidence-informed structured decision-making process (e.g., Heyman & Slep, 2019; Kahneman et al., 2021).

Following the first statewide implementation of the FAIR system, child welfare system social workers and supervisors, state assistant attorneys general, and judges reported positive perceptions of FAIR on feasibility, usability, and impact on workers and families. The FAIR system was perceived to (a) be easy to use, (b) require a manageable amount of training, and (c) increase the consistency and accuracy of child welfare decisions. Next steps should include research on the effect of the FAIR system on observable outcomes for families (e.g., recidivism) and workers (e.g., burnout), as well as implementations in other states with training enhancements. Future studies should also examine long-term impacts, ensuring that improvements in decision-making persist over time and lead to meaningful reductions in child maltreatment rates.