The following are recent articles by Lisbeth Schorr drawing on lessons learned over the last several decades from research, theory, and practice about multiple efforts to solve social problems, new opportunities to act on what we now know, and new approaches to assessing “what works”.
Click on each title for access to that article's full text.
"Expanding the Evidence Universe: Doing Better By Knowing More," December 2011
Executive Summary: "Expanding the Evidence University: Doing Better By Knowing More," December 2011
Thanks to the last two decades of research and experience, we now know so much more than ever before about what it takes to imporve outcomes for disadvantaged children and families -- whether measured in healthy births; school readiness; school achievement; robust physical, and mental health; or neighborhood support and safety. But our expanded knowledge has not led to better outcomes at a magnitude that matches the need. Among the reasons: we have too often failed to marshal the full extent of available evidence and to generate new, real-time knowledge from experience.
The paper explores how we might make use of all the evidence we now have from multiple sources, including research, theory, practice, as well as the findings from program evaluations, and how we might aggressively gather new evidence about the nuanced and powerful strategies for change that are currently emerging.
"Integrating Evidence and Practice to Reduce Disparities: Developing an Inclusive Framework for Effective Mental Health Interventions," Keynote Address to the Conference of the National Alliance of Multi-ethnic Behavioral Health Associations, May 11, 2010.
“Innovative Reforms Require Innovative Scorekeeping,” Education Week, August 26, 2009.Current experience around the country suggests that complex efforts to improve outcomes for disadvanteged children and families are best begun with a focus on results. When communities agree on outcomes and results for children and families, they can encourage innovation and local problem-solving that responds to unique conditions and cultures, by replacing rigid regulation of inputs with rigorous accountability for accomplishments. A results frame makes it easier to bridge diverse constituencies and points of view, to support collaboration across boundaries, to mobilize and sustain efforts over time, and to map backwards from agreed-upon results to determine what collection of interventions and supports are most likely to achieve the results.
New approaches to knowledge building do not reject -- on grounds of messiness or complexity -- information that can shed light on real-world efforts that promise to improve outcomes. Non-experimental evaluations must insist on rigor even in the absence of certainty or proof, in the belief that credible evidence of effectiveness can be attached to promising interventions if they are found to have a basis in strong theory; a converging accumulation of empirical evidence of effectiveness from similar or related efforts that may not rise to the level of causal proof; consensus amoung informed experts based on a combination of theory, research, and practice-based evidence.
“To Judge What Will Best Help Society's Neediest, Let's Use a Broad Array of Evaluation Techniques,” The Chronicle of Philanthropy, August 20, 2009.Serious social reformers today agree that rigorous efforts to determine “what works” are essential. But if these efforts are not to sabotage or marginalize the most innovative attempts to solve intractable social problems, funders must develop more inclusive ways of establishing effectiveness and accountability.
If government agencies and private grant makers support only those interventions that are shown effective by experimental methods, we will be robbed of (1) good programs that do not lend themselves to experimental evaluations, (2) reforms that are deeper and wider than individual programs, and (3) innovations of all kinds.
Mismatch between Complex Interventions and "Gold Standard" Evaluations, Chart, 2009.
Comparison of the attributes of effective complex interventions and the attributes associated with "gold standard” evaluations.
Co-authored with Katya Fels Smyth, “A Lot to Lose: A Call to Rethink What Constitutes ‘Evidence’ in Finding Social Interventions that Work,” January 2009, Malcolm Wiener Center for Social Policy,
The authors describe the characteristics of What It Takes organizations, which their work suggests support lasting change in the lives of highly marginalized and vulnerable people. They identify the risks inherent in the continued privileging of experimental designs over all others, and suggest that the risks are heightened in periods of great economic stress, when the pressure for accountability is increased.
“The O'Connor Project: Intervening Early to Eliminate the Need for Racial Preferences in Higher Education,”Judicature, September 2004
, Volume 88, Number 2 September-October 2004.“The O'Connor Project, Can We End Racial Discrimination Without Affirmative Action? Here's What it Will Take
,” The American Prospect, January 2004.
In her opinion in Grutter v. Bollinger, Associate Justice Sandra Day O'Connor declared her expectation that racial preferences in higher education will no longer be necessary 25 years from now. These articles lay out what we can and must do with present knowledge to reduce or eliminate racial disparities early in life and thereby eliminate the need for racial preferences at the university level.
“From Knowledge Management to
By taking a bold, generous, and inclusive approach to knowledge building, foundations can equip community organizations, service providers, and policymakers at every level with actionable information that will enable them to more effectively achieve their objectives.
Co-authored with Patricia Auspos, “Usable Information About What Works: Building A Broader and Deeper Knowledge Base,” Journal of Policy Analysis and Management, Fall 2003 , Vol. 22, No. 4.
The authors contend that to foster and support effective strategies that will actually improve lives, the way we think about and process knowledge in the social policy world must change:
• Prevailing conventions about what counts as credible knowledge must be reexamined and modified.
• Our focus must change from making yes-or-no judgments about individual interventions to discerning patterns from an accumulation of research and experiences.
• Our knowledge-building activities must cross systems and disciplinary boundaries, even if service delivery and funding continue to operate predominantly within self-contained silos.
Lisbeth Schorr argues that rigorous research on impacts is needed but is not sufficient. Too many programs are multidimensional, cannot or should not be standardized, evolve or adapt through time, require participants’ active involvement, or are dependent for success on good implementation, not just good design. Schorr argues for more flexible forms of evaluation that require experts and practitioners to hypothesize the linkages between actions and outcomes, identify interim indicators of success, and pay more attention to the attributes of programs and the institutional contexts that are essential to success. While recognizing the inevitable trade-off between knowing a few things very well and more things with less certainty, Schorr argues for greater efforts to understand broad patterns that connect activities to results, even if this means compromising the search for absolute truths.