Shelly Martinez
Executive Director
Commission on Evidence-Based Policymaking
Washington, D.C.
Re: Commission on Evidence-Based Policymaking Comments, Docket ID USBC-2016-0003-0001
Dear Ms. Martinez:
The First Five Years Fund (FFYF) strongly supports the Evidence-Based Policymaking Commission’s (“Commission”) mission to identify effective strategies for producing and using evidence to support federal programs and policies. FFYF supports the core early learning and care programs carried out across the U.S. Department of Health and Human Services, the U.S. Department of Education, and the U.S. Department of Health Resources and Service Administration, and is committed to a well-funded, high-quality continuum of affordable early learning and care. Given our work, FFFY urges Commission Members to carefully consider the following recommendations when evaluating and identifying future Commission activities, including possibly developing findings and recommendations for Congress.
Data Use in Program Design, Management, Research, Evaluation, and Analysis
FFYF shares the Commission’s interest in identifying strategies for better using data, statistics, research, and findings from evaluation to improve policies and programs. Strong government programs adopt these strategies as part of a culture of continuous improvement. Effective programs benefit from embedded evaluations, as referenced in the Commission’s Request for Comments. FFYF encourages the Commission to examine ways to better and more routinely embed evaluation as a core element of federal programs. For example, Congress recently provided crosscutting program evaluation authority and funding to the Secretary of Education in the new Every Student Succeeds Act (ESSA, P.L. 114-95). As you examine such models, however, we also encourage you to ensure your work does not lead to an inadvertent over-investment in programs with previously demonstrated effectiveness to the detriment of innovative efforts to develop and test new ideas. Policymakers must continue to invest in innovation, while also supporting efforts to scale programs supported by greater levels of evidence. Furthermore, we recommend an approach to the evaluation of early learning and care program models that takes into consideration the varied factors in children and families’ lives outside the scope of program service delivery that impact child outcomes, and more specifically, that program evaluation is intentionally tied to a theory of change that identifies intended outcomes and the underlying capacities or mechanisms on which those outcomes rest. Poorly designed studies that only focus on third grade assessment outcomes – using tests designed for State K12 accountability systems – do not satisfy this higher standard.
Striking a thoughtful balance between stimulating promising innovations and investment in practices with a greater evidence base is particularly important to strengthening early learning systems. Recent studies, including work by Nobel Prize-winning economist Professor James Heckman and his colleagues at the Center for the Economics of Human Development, demonstrate that early learning has moved beyond rudimentary inquiries to more sophisticated challenges related to achieving greater scale and quality. Professor Heckman’s work, and other well-designed studies, demonstrates that the most effective early learning programs provide high quality, developmentally appropriate, comprehensive services for children from low-income families from birth through age 5.
As we enter into the era of greater implementation based on this existing research base, we also need continued experimentation, research and working experience that informs practice, fosters innovation and provides elected officials and the public with even more confidence to invest in systems and programs that work — and continuously improve. Dr. Jack Shonkoff, M.D. at Harvard University’s Center on the Developing Child recently released a report, ‘From Best Practices to Breakthrough Impacts’, which highlights that while there exists a number of studies on the effectiveness of various early learning programs and interventions demonstrating the difference early childhood programs do indeed make, there is limited data available that could be used to inform replication and scalability that would result in improved outcomes at a population level. Furthermore, in evaluating program effectiveness, it is imperative that we graduate from using a broad brush in asking whether or not a program ‘works’, and begin asking ‘which features work for whom and why?’ This frame of program evaluation positions the field to advance replication of the elements that do lead to improved outcomes without being tethered to less effective elements of earlier intervention iterations. Dr. Shonkoff also recommends that in addition to conducting randomized control trials, which take several years to complete with no opportunities for mid-course corrections, program evaluation should include micro-trials that are small-scale, and short-duration field tests designed to catalyze rapid, shared learning across multiple projects in the field simultaneously.
Dr. Shonkoff and his colleagues summarized the particular challenge in the early learning field well when they wrote:
“On the one hand, many leaders in the field are engaged in critically important efforts to improve the quality of programs, increase the effectiveness and efficiency of service delivery systems, enhance the skills and compensation of a highly diverse early childhood workforce, and encourage innovation. These efforts are happening at multiple levels across a variety of sectors—and they must be sustained. On the other hand, most decision makers urge funding solely for programs with previously demonstrated effectiveness, regardless of the nature or magnitude of their impacts. This widespread preference for “evidence-based” programs, many of which have produced small effects on random categories of outcomes that have not been replicated, seriously limits the likelihood of achieving increasingly larger impacts at scale over time. Indeed, many of the most compelling challenges facing the early childhood field today are linked to the absence of sufficient professional and political incentives for developing and testing new ideas”
As a result, FFYF encourages the Commission to explore evidence-based structures that support and encourage federal programs and investments across a continuum from promising innovations to investments with a more robust research base. Some existing programs, such as the U.S. Department of Education’s legacy Investing in Innovation Fund (i3), and ESSA’s new Education Innovation and Research Grants, specifically utilize a tiered approach to investment, based on an initiative’s evidence base. Exploring the efficacy of such evidence based models, while also establishing other evidence and evaluation structures that identify and document best practices for sharing, could help to fundamentally improve federal policy and practice.
Thank you for providing this opportunity to help inform the Commission’s deliberations and work. We would be glad to answer any questions you may have about these ideas and FFYF’s work and would be pleased to participate in future Commission events.
Sincerely,
Kris Perry
Executive Director
First Five Years Fund
1010 Vermont Avenue, NW, Suite 810
Washington, D.C. 20005
(202) 730-0941
Download PDF