|
Recent Over-the-Road Validation Trials
WayPoint has been extensively validated in the trucking industry by the very largest TL and LTL carriers. Here we describe how the validation trials were done; the results of the trials; and how the results translate into a 20% decrease in fleet-wide collisions. Typically, that is an ROI of 35:1 in the first year.
Five carriers each did blind, retrospective trials with their OTR drivers. The S and J trials were "case control" studies, where drivers were chosen for their collision records over a specified time period. Drivers in the C trial were randomly sampled: they included drivers attending a Saturday morning safety meeting. After the test was administered, booklets were scored "blind" by WRI and and Driver Assessment Reports were returned to the carrier. The DAR provides a risk level (High risk vs. low/average) for preventable and non-preventable collsions. In addition, the DAR includes a list of driving habits typical of the WayPoint profile along with specific countermeasures for avoiding collisions.
The valididity of WayPoint in particular, and predictive tests in general, is quantified in several ways. The first method, called signal detection, computes the ability of the test to identify the crash prone driver without falsely targeting the safe driver. Two percentages tell the story:
- Hit Rate
. A Hit is a true positive, the case where WayPoint identifies a driver as High Risk when in fact his or her crash performance is in the upper 15th percentile. The Hit Rate is the percentage drivers correctly identified as poor performers. The Miss Rate, then, is simply the percentage of poor performers NOT identified as High Risk (100%-Hit Rate) Obviously, the higher the Hit Rate, the lower the Miss Rate.
- False Alarm Rate
A False Alarm is a false positive, the case where WayPoint identifies a driver as High Risk when in fact he or she does not have crash performance in the upper 15th percentile. The False Alarm Rate is the percentage drivers identified as High Risk who are, in fact, good performers. The Correct Acceptance Rate, then, is the percentage good performers who the test accepts (100-False Alarm Rate).
Obviously, the higher the hit rate and the lower the false alarm rate, the better the test. For OTR drivers, typical hit and false alarm rates are shown in the table below. When a combination of collision frequency and severity are used as a critierion measure and corrected for tenure, WayPoint has a hit rate of 65% and a false alarm rate of 9% or less.
The S trial allowed us to validate WayPoint on a criterion measure that did not involve crashes: Complaints [mostly to the 800 "How's my driving?" number] from the motoring public. The Hit Rate by WayPoint for drivers receiving one or more complaints was 52% whereas the False Alarm Rate was 20%, a statistically significant outcome.
|
|
