April 16, 2018In this article forthcoming in the Ohio State Law Journal, Quattrone Center affiliate David Rudovsky and David Harris argue for a larger role of empirical analysis in court rulings regarding Terry stops.
Abstract: The investigative detention doctrine first announced in Terry v. Ohio and amplified over the past fifty years has been much analyzed, praised, and criticized from a number of perspectives. Significantly, however, over this time period commentators have only occasionally questioned the Supreme Court’s “common sense” judgments regarding the factors sufficient to establish reasonable suspicion for stops and frisks. For years, the Court has provided no empirical basis for its judgments, due in large part to the lack of reliable data. Now, with the emergence of comprehensive data on these police practices, much can be learned about the predictive power of suspect conduct and other predicates for law enforcement interventions. And what has been learned calls into question a number of factors that have been credited over many years.
No observer of the legal system can fail to notice the growing role of data and empirical analysis in the courts. A disparate set of cases have turned in large part on rigorously analyzed data. Yet this trend has not taken root in an important set of cases involving the widely used practice of stop-and-frisk. When stop-and-frisk practices become the subject of litigation, courts generally either have no data to review or have failed to engage in empirical analysis of the data that are available and which could be used to test the claims of reasonable suspicion. Rather, the courts invoke the conventional wisdom that as a matter of common sense certain conduct, for example, furtive movement, flight, bulges in clothing, and suspect location, indicates criminal conduct.
We have no argument with common sense propositions; we have no aversion to clear, straightforward thinking. But what this phrase often reflects is a set of unexamined (even if widely held) assumptions. The proliferation of data on these basic questions provides the means for empirical analysis, and it is our argument that courts should do so in assessing reasonable suspicion factors in the same manner that they have engaged in empirical judgments, using both big and targeted data, in other areas.
April 6, 2018The groundbreaking study by Quattrone Center affiliates Charles Loeffler and Greg Ridgeway, along with Jordan Hyatt, presents a new methodology for measuring the incidence of wrongful conviction.
Objectives: Estimate the frequency of self-reported factual innocence in non-capital cases within a state population of prisoners.
Mehods: We conducted a survey of a population sample of state prisoners who were asked to anonymously report their involvement in the crimes for which they were most recently convicted. To assess the validity of verifiable responses, prisoner self-report data were compared to aggregate conviction and demographic information derived from administrative records. To assess the validity of unverifiable responses, we developed a non-parametric test to estimate the probability of false innocence claims.
Results: We estimate that wrongful convictions occur in 6% of criminal convictions leading to imprisonment in an intake population of state prisoners. This estimate masks a considerable degree of conviction-specific variability ranging from a low of 2% in DUI convictions to a high of 40% in rape convictions. Implausible or false innocence claims are estimated to occur in 2% of cases.
Conclusions: The present investigation demonstrates that survey methods can provide bounded estimates of factual innocence claims within a discrete and known population. The resulting estimates, the first to formally separate claims of legal and factual innocence and to incorporate a formal measure of response plausibility, suggest that prisoners themselves are very often willing to self-report the correctness of their convictions. At the same time, a considerable minority indicate that procedural weaknesses with the administration of justice occurred in their cases. And, a distinct minority, with considerable offense variation, maintain that they are completely innocent of the charges against them.
February 26, 2018Citywide Cluster Randomized Trial to Restore Blighted Vacant Land and Its Effects on Violence, Crime, and FearIn this article in the Proceedings of the National Academy of Sciences, co-authored by Quattrone Center faculty affiliate John McDonald, provides rigorous evidence on the crime-reducing effects of urban infrastructure improvements.
Abstract: Vacant and blighted urban land is a widespread and potentially risky environmental condition encountered by millions of people on a daily basis. About 15% of the land in US cities is deemed vacant or abandoned, an area roughly the size of Switzerland. In a citywide cluster randomized controlled trial, we investigated the effects of standardized, reproducible interventions that restore vacant land on the commission of violence, crime, and the perceptions of fear and safety. Quantitative and ethnographic analyses were included in a mixed-methods approach to more fully test and explicate our findings. A total of 541 randomly sampled vacant lots were randomly assigned into treatment and control study arms; outcomes from police and 445 randomly sampled participants were analyzed over a 38-month study period. Participants living near treated vacant lots reported significantly reduced perceptions of crime (−36.8%, P < 0.05), vandalism (−39.3%, P < 0.05), and safety concerns when going outside their homes (−57.8%, P < 0.05), as well as significantly increased use of outside spaces for relaxing and socializing (75.7%, P < 0.01). Significant reductions in crime overall (−13.3%, P < 0.01), gun violence (−29.1%, P < 0.001), burglary (−21.9%, P < 0.001), and nuisances (−30.3%, P < 0.05) were also found after the treatment of vacant lots in neighborhoods below the poverty line. Blighted and vacant urban land affects people’s perceptions of safety, and their actual, physical safety. Restoration of this land can be an effective and scalable infrastructure intervention for gun violence, crime, and fear in urban neighborhoods.
December 21, 2017An Analysis of Racial and Ethnic Disparities in Case Dispositions and Sentencing Outcomes for Criminal Cases Presented to and Processed by the Office of the San Francisco District AttorneyThis report, prepared on behalf of the San Francisco District Attorney by Quattrone Center affiliate John MacDonald and Steven Raphael, examines sources of racial disparity in criminal justice outcomes in San Francisco, complementing prior work on this topic completed by the Center on behalf of the San Francisco Public Defender. It finds that substantial disparities exist, but most can be explained by preexisting factors occurring prior to the lodging of cases with the district attorney’s office. Moreover, racial disparities have narrowed since the passage of California Proposition 47 in November 2014.
December 15, 2017Root Cause Analysis: A Tool to Promote Officer Safety and Reduce Officer Involved Shootings Over TimeIn this Villanova Law Review article, Quattrone Center Executive Director John Hollway, Clavin Lee, and Sean Smoot discuss how Root Cause Analysis (RCA) can be used to address the problem of officer-involved shootings (OIS) of civilians.
RCA is problem-solving method designed to identify core underlying factors, including environmental or systemic factors, that contributed (along with individual decision-making) to an undesirable outcome, organizational accident, or adverse event. Once these core causative factors have been identified, system participants can reduce or remove them from the system, thereby preventing future occurrences of the undesirable outcome–in this case, an OIS. RCA is part of a prospective, non-blaming “systems approach” to preventing error in complex human systems that has successfully reduced errors in aviation, healthcare, manufacturing, nuclear power, and other areas.
Existing retrospective accountability measures for law enforcement, such as internal affairs, OIS administrative reviews, civil rights litigation, civilian review boards, etc., are necessary to ensure official accountability for OIS that occur, and to secure appropriate compensation for civilians who may be innocent victims of an OIS. However, these mechanisms have proven insufficient in reducing the incidence or prevalence of OIS.
Use of RCA to evaluate officer-involved shootings can permit a deeper understanding of all the contributory factors of a shooting, and provide a road map to guide policy reforms and reduce OIS over time. Many, and perhaps most OIS occur despite the fact that no one—neither the officer, nor the civilian, nor the general public—wants them to occur. Rather, a sequence of contributing events led the officer to feel that pulling the trigger is his or her best decision. An understanding of these multiple, contributing factors—environmental, informational, situational, supervisory, etc.—is essential to interrupting that sequence for the next officer and next civilian.
August 29, 2017This research paper by Megan Stevenson, a 2016-2018 Quattrone Center Research Fellow and now Assistant Professor of Law at George Mason University, provides one of the first empirical assessments of the effects of the implementation of modern risk assessment tools. Using rich data on more than one million criminal cases, the paper shows that a 2011 law making risk assessment a mandatory part of the bail decision led to a significant change in bail setting practice, but only a small increase in pretrial release. These changes eroded over time as judges returned to their previous habits. Furthermore, the increase in releases was not cost-free: failures-to-appear and pretrial crime increased as well. Risk assessment had no effect on racial disparities in pretrial detention once differing regional trends were accounted for. It concludes that Kentucky’s experience does not mean we should abandon risk assessment, but it should temper the hyperbolic hopes (and fears) about its effects. Risk assessment in practice is different from risk assessment in the abstract, and its impacts depend on context and details of implementation.
June 27, 2017Most studies of racial disparities in the justice system have focused on final case outcomes, such as conviction, incarceration, and sentence length. While important, these data points do not provide sufficient insight into the many points in the criminal justice process where cases against Black, White, and Latinx defendants could diverge. To fill this knowledge gap, the Quattrone Center, in collaboration with the San Francisco Public Defender , reviewed the charging and case adjudication process for 10,753 Public Defender clients in San Francisco with cases resolved between 2011 and 2014.
The raw data reveal Black/White and Latinx/White disparities. Black defendants are held in pretrial custody for an average of 30 days, 62% longer than Whites. Cases involving Black defendants also take longer to resolve–it takes an average of 90 days to process a case for a Black defendant, but only 77.5 days to process a case for White defendants, a delay of 14%. Black defendants are convicted of 60% more felony charges than White defendants, and 10% fewer misdemeanors, while Latinx defendants are convicted of a similar number of felonies to Whites, but 10% more misdemeanors. Custodial sentences received by Black defendants are, on average, 28% longer than those received by White defendants. Latinx defendants receive comparable custodial sentences to White defendants, but they receive probation sentences that are 55% longer.
Even though these disparities are occurring within the plea bargaining system, plea bargaining itself appears to neither contribute to the disparate outcomes, nor to reduce the disparities. Rather, these disparities are best explained by two factors that pre-date the case adjudication process: 1) People of color receive more serious charges at the initial police booking stage and 2) People of color have more extensive previous encounters with the criminal justice system in San Francisco County. Criminal history and booking decisions appear to have a “ripple effect” that impacts plea negotiations for subsequent charges.
March 29, 2017
This Stanford Law Review article by Quattrone Center researchers Paul Heaton, Sandra Mayson, and Megan Stevenson offers a first-of-its-kind empirical analysis of misdemeanor pretrial detention.
Abstract: In misdemeanor cases, pretrial detention poses a particular problem because it may induce otherwise innocent defendants to plead guilty in order to exit jail, potentially creating widespread error in case adjudication. While practitioners have long recognized this possibility, empirical evidence on the downstream impacts of pretrial detention on misdemeanor defendants and their cases remains limited. This Article uses detailed data on hundreds of thousands of misdemeanor cases resolved in Harris County, Texas—the third largest county in the U.S.—to measure the effects of pretrial detention on case outcomes and future crime. We find that detained defendants are 25% more likely than similarly situated releasees to plead guilty, 43% more likely to be sentenced to jail, and receive jail sentences that are more than twice as long on average. Furthermore, those detained pretrial are more likely to commit future crime, suggesting that detention may have a criminogenic effect. These differences persist even after fully controlling for the initial bail amount as well as detailed offense, demographic, and criminal history characteristics. Use of more limited sets of controls, as in prior research, overstates the adverse impacts of detention. A quasi-experimental analysis based upon case timing confirms that these differences likely reflect the casual effect of detention. These results raise important constitutional questions, and suggest that Harris County could save millions of dollars a year, increase public safety, and reduce wrongful convictions with better pretrial release policy.
Summary of Key Findings
March 13, 2017This article by Quattrone Research Fellows Sandra Mayson and Megan Stevenson reviews research identifying opportunities for improving pretrial processes in the United States.
Abstract: Our current pretrial system imposes high costs on both the people who are detained pretrial and the taxpayers who foot the bill. These costs have prompted a surge of bail reform around the country. Reformers seek to reduce pretrial detention rates, as well as racial and socioeconomic disparities in the pretrial system, while simultaneously improving appearance rates and reducing pretrial crime. The current state of pretrial practice suggests that there is ample room for improvement. Bail hearings are often cursory, with no defense counsel present. Money-bail practices lead to high rates of detention even among misdemeanor defendants and those who pose no serious risk of crime or flight. Infrequent evaluation means that the judges and magistrates who set bail have little information about how their bail-setting practices affect detention, appearance and crime rates. Practical and low-cost interventions, such as court reminder systems, are underutilized. To promote lasting reform, this chapter identifies pretrial strategies that are both within the state’s authority and supported by empirical research. These interventions should be designed with input from stakeholders, and carefully evaluated to ensure that the desired improvements are achieved.
February 5, 2017Using Root Cause Analysis to Study Prosecutorial Error: A Collaboration Between the Montgomery County (Pennsylvania) District Attorney’s Office and the Quattrone Center for the Fair Administration of JusticeIn this article published in Tolle Lege: The Villanova Law Review, Steven Raper leads a collaborative team of Penn researchers and Montgomery County (PA) District Attorneys in reporting on the results of an award-winning root cause analysis (RCA) conducted in the county.
February 3, 2017Comparing the Effectiveness of Henderson Instructions and Expert Testimony: Which Safeguard Improves Jurors’ Evaluations of Eyewitness Evidence?This paper, co-authored by Quattrone Research Fellow Amanda Bergold and published in the Journal of Experimental Criminology, compares the efficacy of expert testimony versus jury instructions as safeguards to assist jurors to accurately evaluate eyewitness testimony.
Link to published version
January 26, 2017
This paper, co-authored by Quattrone Research Fellow Amanda Bergold and published in the Journal of Forensic Psychology Research and Practice, evaluates the effects of reforms designed to improve juror assessment of eyewitness identification, a leading cause of error in the U.S. criminal justice system.
Abstract: Mistaken eyewitness identifications are a leading cause of wrongful convictions. Even with procedural safeguards (e.g. attorney argument, cross examination of witnesses) in place, jurors still have difficulty evaluating the reliability of eyewitness identifications. The purpose of the current study was to test the New Jersey Supreme Court’s assumptions that recently implemented research-based case-specific jury instructions sensitize jurors to unreliable eyewitness testimony. Four hundred sixty-eight jury-eligible adults watched a trial simulation in which estimator variables, system variables, and jury instruction were manipulated, and subsequently rendered a verdict. The Henderson instructions influenced mock jurors’ perceptions of the eyewitness identification, but these perceptions did not translate to their verdict decisions. Rather than sensitizing jurors, the instructions induced an overall skepticism of eyewitness identification. Taken together, results indicate that the current Henderson instructions should be modified to improve juror sensitivity to various witnessing and identification conditions.
Link to published version
November 30, 2016This article published in the Journal of Law and Economics, written by Paul Heaton and John McDonald of the Quattrone Center and other co-authors, measures the efficacy of private police in reducing crime.
Abstract: Over a million people in the United States are employed in private security and law enforcement, yet very little is known about the effects of private police on crime. The current study examines the relationship between a privately funded university police force and crime in a large US city. Following an expansion of the jurisdictional boundary of the private police force, we see no short-term change in crime. However, using a geographic regression discontinuity approach, we find large impacts of private police on public safety, with violent crime in particular decreasing. These contradictory results appear to be a consequence of a delayed effect of private police on crime.
September 19, 2016This study by John MacDonald and Ellen Donnelly, conducted at the request of the Delaware Access to Justice Commission’s Committee on Fairness in the Criminal Justice System, involved a statistical review of all adults arrested and charged with a criminal offense in Delaware between 2012 and 2014 to determine the extent to which race is a factor in explaining the likelihood or length of any subsequent incarceration sentence. While the authors caution that the study does not identify exact causes for any observed disparities, factors such as the offense charged at arrest, pretrial detention and contextual factors such as criminal history appear to be important contributors to Black/White differences in incarceration.
Link to Access to Justice Commission press release on the study
June 15, 2016This article by Stephanos Bibas in the William and May Law Review proposes ways to improve the plea bargaining process so as to improve fairness and reduce wrongful convictions.
Abstract: American criminal procedure developed on the assumption that grand juries and petit jury trials were the ultimate safeguards of fair procedures and accurate outcomes. But now that plea bargaining has all but supplanted juries, we need to think through what safeguards our plea-bargaining system should be built around. This Symposium Article sketches out principles for redesigning our plea-bargaining system from the ground up around safeguards. Part I explores the causes of factual, moral, and legal inaccuracies in guilty pleas. To prevent and remedy these inaccuracies, it proposes a combination of quasi-inquisitorial safeguards, more vigorous criminal defense, and better normative evaluation of charges, pleas, and sentences. Part II then diagnoses unfair repercussions caused by defendants’ lack of information and understanding, laymen’s lack of voice, and the public’s lack of information and participation. To prevent and fix these sources of unfairness, it proposes ways to better inform pleas and to make plea procedures more procedurally just.
May 4, 2016This working paper by Quattrone Center Research Fellow Megan Stevenson provides innovative new quasi-experimental evidence on the impacts of bail.
Abstract: Instrumenting for detention status with the bail-setting propensities of rotating magistrates I find that pretrial detention leads to a 13% increase in the likelihood of being convicted, an effect explained by an increase in guilty pleas among defendants who otherwise would have been acquitted or had their charges dropped. On average, those detained will be liable for $128 more in court fees and will receive incarceration sentences that are almost five months longer. Effects can be seen in both misdemeanor and felony cases, across age and race, and appear particularly large for first or second time arrestees. Case types where evidence tends to be weaker also show pronounced effects: a 30% increase in pleading guilty and an additional 18 months in the incarceration sentence. While previous research has shown correlations between pretrial detention and unfavorable case outcomes, this paper is the first to use a quasi-experimental research design to show that the relationship is causal.
April 25, 2016In this groundbreaking report, the Quattrone Center provides a first detailed look at the structure and functioning of Conviction Review Units across the United States.
U.S. policymakers and the public are increasingly concerned about wrongful convictions– situations where innocent individuals have been convicted and incarcerated for crimes they did not commit. The most prevalent institutional response by prosecutors to address post-conviction fact-based claims of actual innocence is the Conviction Review Unit (CRU), sometimes called the Conviction Integrity Unit. Since the creation of the first CRU in the mid-2000s, more than 25 such units have been announced across the country; more than half of these have been created in the past 24 months.
Most CRUs have grown up ad-hoc, often in reaction to a limited number of specific cases with unique circumstances. Very few have written protocols, policies, or procedures, and few of those have been made public. This paper reports results from the first ever national survey of CRUs designed to assist: (a) Current CRUs in understanding how their peers have approached common challenges; (b) Offices without CRUs in the creation of effective units; and (c) Communities in developing metrics to evaluate the units and their utility.
February 29, 2016This article co-authored by Quattrone Center affiliated faculty member Jon Klick and published in the Journal of Law and Economics examines ways that providing exonerees with compensation can improve the accuracy of the criminal adjudication process.
Abstract: A great concern with plea bargains is that they may induce innocent individuals to plead guilty to crimes they have not committed. In this article, we identify schemes that reduce the number of innocent pleas without affecting guilty individuals’ plea-bargaining incentives. Large compensations for exonerees reduce expected costs associated with wrongful determinations of guilt in trial and thereby reduce the number of innocent pleas. Any distortion in guilty individuals’ incentives to take plea bargains caused by these compensations can be offset by a small increase in the discounts offered for pleading guilty. Although there are many statutory-reform proposals for increasing exoneree compensation, no one has yet noted this desirable separating effect of compensations. We argue that such reforms are likely to achieve this result without causing losses in deterrence.
February 10, 2016Can a Criminal Justice Alcohol Abstention Program with Swift, Certain, and Modest Sanctions Reduce Population Mortality?This study, published in the Lancet Psychiatry and co-authored by Academic Director Paul Heaton, examines whether a program for handling substance-involved offenders in the criminal justice system that replaces traditional probation with a new approach based upon frequent monitoring and swift, certain, but modest sanctions can improve population health. Analysing county-level data, the study shows that implementation of 24/7 Sobriety in South Dakota—the largest such program targeting alcohol-involved offenders to date—is associated with a reduction in mortality of 4.2% (95% CI 1·5–6·9; p<0.01). The association is evident among causes strongly and often acutely associated with excessive alcohol use such as circulatory conditions. These results provide additional support for swift, certain, and fair sanctioning approaches to community supervision, and build upon prior research suggesting that such programs can reduce crime.
December 8, 2015The Quattrone Center, along with the Philadelphia Police Department, the Philadelphia District Attorney’s Office, the Philadelphia Court of Common Pleas, two private defense counsel, and a former reporter for the Philadelphia Daily News collaborated on a National Institute of Justice (NIJ)- sponsored event review using “systems principles” of error reduction that had been proven effective in other environments (e.g., healthcare, aviation, nuclear power management).
The Philadelphia Sentinel Event Review Team selected the Lex St. Massacre–the largest single mass murder in Philadelphia history, during which four men mistakenly confessed and were incarcerated prior to identifying the true perpetrators of the crime–to review. The team dedicated itself to the thorough review of the Lex St. case using principles of root cause analysis (RCA) to conduct a “just culture event review.” The review was designed to understand what happened in the Lex St. investigation and prosecution, and to generate precise recommendations that will help to prevent the mistakes that occurred in that case from happening again.
October 16, 2015The Access to Justice Commission, an initiative of the Delaware Judiciary, has requested analysis of several aspects of Delaware’s criminal justice system to determine whether reforms in policing, prosecution, adjudication and imprisonment can be accomplished in a way that would reduce racial disparities, while not increasing the incidence of violent crime. The Quattrone Center provided recommendations to the commission on ways to improve fairness in police searches, bail, charging and sentencing, and application of alternatives to incarceration.
April 20, 2015This white paper describes an internal Root Cause Analysis (“RCA”) conducted by the Montgomery County, PA District Attorney’s Office in partnership with the Quattrone Center. The Quattrone Center specializes in RCA regarding errors within the criminal justice system. RCA has been used in complex and fragmented industries (e.g., healthcare, aviation, nuclear power) to understand the various factors (direct and environmental) that lead to both individual and systemic-based errors, and to provide interventions that can effectively prevent future error.
February 1, 2014The “systems approach” has been used, improved, and refined over time to improve safety and reduce errors in a variety of complex, high-risk industries, including health care, aviation, and manufacturing, among others. While the challenge of preventing errors in well-meaning complex systems is neither new nor unique to criminal law, the need for error reduction in the criminal justice system is clear. This document advocates for the application of a systems approach to reducing errors in the criminal justice system, generating reform in a fashion that will unify well-intentioned but professionally adversarial participants around an objective shared by all: the integrity of investigations, prosecutions, and adjudications, and the elimination of known and currently unknown errors that undermine the fair administration of justice. It then sets forth requirements for the successful application of a systems approach, and a model for interaction among researchers, reformers, and practitioners in the criminal justice system – including prosecutors, defense attorneys, judges, and law enforcement officials – that will allow for more rigorous analyses of the criminal justice system and the design, testing, dissemination and implementation of successful best practices that will improve the fair administration of justice.