For over 40 years, risk-needs assessments (RNAs) have been deployed as a key tool for determining risk and providing programming recommendations for individuals under correctional supervision. The benefits of RNAs are substantial, providing mechanisms of standardization, reducing biases and, in turn, decreasing justice system involvement. As a result of the positive outcomes when adopting RNAs, many states have mandates in place for state correctional populations. The increased popularity and demand for RNAs has resulted in agencies needing assessment tools in a short period of time, thus opting for tools that were developed elsewhere, outside of their jurisdiction, tested on a different population altogether. For these off-the-shelf (OTS) tools, the belief was that they would offer the same predictive accuracy and function similarly regardless of location or population. Unfortunately, we now know this is not true. Another regrettable circumstance at the rise of OTS tools is that, in some cases, even the developers convey their validation statistics as stable and unwavering, thus many agencies have been misinformed. This means that in the last 20 years, the vast majority of agencies throughout the United States have been using RNAs not built for their jurisdiction, limiting the possibility of successful outcomes. Despite this being the circumstance for many agencies in the U.S., Dr. Zachary Hamilton, University of Nebraska, Omaha, and his team have been working diligently with Vant4ge to develop the Static Risk and Offender Needs Guide-Revised (STRONG-R). Dr. Hamilton drew from Tennessee’s population to create a highly advanced risk and needs assessment tool. What we developed together was astounding.
Where Prediction Shrinks
Risk-needs assessment researchers have a standard metric when calculating the quality of an RNA, this is referred to as the Area Under the Curve (AUC). When predicting someone’s recidivism rate, a score of .50 would be as accurate as a coin toss—50/50. This is unacceptable and ineffective. On the other hand, a score of 1.0 would be a perfect knowledge of the future, which, simply put, is science fiction. The standard range to evaluate the strength of a tool’s predictive accuracy is .50-.55 as negligible, 0.56-0.63 as weak, 0.64-0.70 as moderate, while 0.71 is considered a strong level of predictive accuracy. When examining the results of OTS tools not built for a specific jurisdiction, historically, the scores are not great. One OTS tool, when applied in Ohio, produced a moderate AUC score of 0.70. When that same tool was applied to an out-of-state jurisdiction, the result was a weak score of 0.59. Similarly, when an early version of an RNA tool was applied in Ontario, the result was an extremely strong score of 0.77. When this same tool was moved to a jurisdiction in the U.S., for which the tool was not specifically built, the score dropped to 0.61—this is known as “predictive shrinkage”. As it stands, this is the standard for OTS tools being frequently adopted by agencies throughout the country.
A Tennessee Proxy
In 2016, Tennessee passed the Public Safety Act with the intent of improving not only case planning and community supervision, but to build a customized risk and needs assessment tool individuated to their state. When TDOC reached out to Vant4ge, they had little usable data to build an assessment tool, much of which was antiquated or inconsistent. This made it difficult for Vant4ge and Dr. Hamilton to draw from a reliable sample. To surpass this hurdle, Dr. Hamilton developed a proxy sample from the Washington Department of Corrections (WADOC), where STRONG-R had already been implemented.
Dr. Hamilton’s team’s method was to find proxy samples in Tennessee that were nearly identical to those in WADOC based on race, gender, prior offenses, etc. and build a tool based on those overlapping traits. The TDOC STRONG-R 1.0 was weighted in 4 categories: VIOLENT, PROPERTY, DRUG, and FELONY. The average AUC result for men was 0.71 and for women, 0.68. Based on a proxy sample size that was originally drawn from over 56,000, but pared down to 33,000 for higher accuracy, these results were better than we could have imagined.
Not Over Yet
According to Dr. Hamilton and his team, RNA development is commonly completed in stages, where 1) developers generate a pool of potential items, 2) an initial tool is created and piloted with a development sample, 3) unnecessary items and responses clarified, 4) a final model is established and deployed, and 5) after sufficiently tracking recidivism outcomes (i.e. two-plus years), the tool is validated. In two years’ time, after the TDOC 1.0 was implemented, Vant4ge had gathered enough usable data to validate the tool based solely on TDOC’s population. TDOC 2.0 was weighted for the same four categories, for both genders, and the results drastically improved. For males, the AUC improved from 0.71 to 0.75, and for females the score jumped from 0.68 to 0.74.
When Tennessee reached out to Vant4ge and Dr. Hamilton, they required a working assessment tool ready to test in six months. In these situations, most agencies opt for an OTS tool to meet tight deadlines thinking there is no time to create a customized assessment that meets the local needs of their population. Tennessee’s experience proved this is not the case. With Vant4ge’s STRONG-R tool and Dr. Hamilton’s team’s proxy method we were able to build a highly predictive, customized tool in under six months.
Validated Now and for Longevity
The proxy method used in the Tennessee STRONG-R 2.0 is a foundational advancement in the capacity to quickly develop a highly predictive, customizable and localized assessment. Furthermore, it is made more effective through its use in a platform that uses risk and needs to drive case management decision making. Through a proven implementation model, Vant4ge works with agencies and their leadership to support long-term and sustainable use. By automating the use of the risk and needs outputs of the highly predictive assessment, case management efficacy is vastly improved. Accurate risk and needs prediction improves safety and supervision, which enhances program matching and program access, increasing efficiency and outcomes. Thus, more accurate tools help agencies improve rehabilitation practices, increase public safety, and reduce recidivism. Administering an assessment may be the commencement of corrections or community care, but implementing Vant4ge’s STRONG-R improves every step across the continuum of supervision.
Download the article, “Never Going to Let You Down Preventing Predictive Shrinkage via the STRONG-R Assessment Method”
