Jamie F Chriqui, Yuka Asada, Natalie Riva Smith, Aimee Kroll-Desrosiers, Stephenie C Lemon, Advancing the science of policy implementation: a call to action for the implementation science field, Translational Behavioral Medicine, Volume 13, Issue 11, November 2023, Pages 820–825, https://doi.org/10.1093/tbm/ibad034
Navbar Search Filter Mobile Enter search term Search Navbar Search Filter Enter search term SearchPublic policies have been essential in addressing many of the most pressing public health problems in the USA and around the world. A large and convincing body of multidisciplinary research has established the impacts or effectiveness of public policies, such as smoke-free air laws and nutrition standards, on improving health outcomes and behaviors. Most of this research assumes that because an evidence-based policy is adopted or takes effect, it is implemented as intended. This assumption, however, is often incorrect. Like with clinical guidelines and other interventions, implementation science has an important role to play in promoting the uptake and implementation of evidence-based public policies that promote public health. To realize this potential, there remains a critical need to first establish a common understanding of what public policy is, the role of specific policies in the context of implementation (i.e., is it the evidence-based intervention or the implementation strategy?), and to establish an appropriate methodological foundation for the field of policy implementation science. We recommend that the field must evolve to (i) include policy experts and actors on policy implementation science study teams; (ii) identify theories, models, and frameworks that are suitable for policy implementation science; (iii) identify policy implementation strategies; (iv) adapt and/or identify study designs best suited for policy implementation science research; and (v) identify appropriate policy implementation outcome measures.
Public policies are important to promote the health and well-being of the public. Many important health advances have been made because of policies designed to prevent or limit unhealthy behaviors (such as smoke-free laws) and promote access to medical care (such as health insurance mandates). However, just because a policy is “on the books” does not mean that it is implemented or implemented as intended. To improve how researchers study policy implementation, we discuss some challenges in the field, provide a call to action for researchers to continue developing the field of policy implementation science, and we recommend that scientists establish partnerships with experts in public policy and work together to develop scientific methods that will do a better job of putting policy into practice.
ImplicationsPractice: Policy implementation science can inform how health-related policies are implemented in practice, including barriers and facilitators to implementation and strategies to promote optimal implementation.
Policy: Decision-makers should consider whether and how policies are implemented in practice when determining whether such policies are working as intended and whether they should be continued or terminated.
Research: Researchers studying policy impacts or effectiveness should include an implementation science lens to really understand the mechanisms by which the given policy may or may not be effecting the desired outcomes.
Public policies have been instrumental in addressing some of the largest public health problems of our time [ 1]. From reducing tobacco use and secondhand smoke exposure (through taxes and smoke-free air laws); to reducing motor vehicle fatalities (through speed limits, blood alcohol content limits, and handsfree laws); to reductions in dental caries (through water fluoridation laws); public policies have led to major public health gains. Understandably, there is an extensive body of literature across disciplines such as public health, economics, public policy, political science, urban planning, and many others that examine the effects or impacts of public policies that have been adopted or enacted on communities, organizations, and individual behaviors and health-related outcomes. Studies of the impacts of public health-related policies such as those listed above have been instrumental in driving further policy and resultant behavioral changes to improve health outcomes.
To date, most studies of policy impacts or effects a priori assume that because a policy is “on-the-books,” adopted, or takes effect (e.g., a tax change), it is implemented (or enforced) in practice as intended or designed. These studies typically examine the impacts or effects of public policies without considering details of the content of the policy, how the policy was implemented after it was adopted or enacted, or strategies that facilitate or inhibit policy implementation. The priority placed on studies of policy effects with a lack of consideration of policy implementation in public health research is exemplified by a recent literature search that we conducted in PubMed for US-based papers published in the last 10 years that included the terms “policy OR policies OR law* OR legislation OR tax OR ordinance OR regulation” AND “implementation” in the title and/or abstract. Of the 3,110 records returned, only 20 truly met the criterion of “policy” and “implementation”; 506 (16.3%) were studies of the impact or effects of a given policy on a particular outcome and not studies of policy implementation; the remaining studies included commentaries, protocols, and other studies related to policy and implementation but not on point. In nearly all cases, the authors considered the fact that the policy took effect as equivalent to it being implemented (and referred to it as a study of policy implementation). Similarly, we conducted a search of the NIH Reporter for studies funded under the NIH Dissemination and Implementation Research in Health (DIRH) program announcements for the past 10 years. Of the 110 funded studies that mentioned “policy” or a related term (policy, policies, law, legal, legislation, ordinance, statute, regulation, regulatory, code, rule) in the abstract or title, only 16 (14.5%) were studies designed to understand factors affecting policy implementation or enforcement, understand the mechanisms of implementation, or test strategies to improve policy implementation.
However, like with evidence-based clinical guidelines and interventions, public health policies are not always implemented or enforced as intended [ 2, 3]. Because of the lack of consideration of implementation in studies of policy effects, researchers and decision-makers may conclude that when a policy does not demonstrate the desired or expected effects, the policy does not work. Yet, it may be that the policy is not being implemented at all or not being implemented or enforced as designed (see Implementation Is a Critical and Oftentimes Forgotten Stage in the Study of Public Health Policy section for specific examples). Such implementation failures can lead to erroneous conclusions that policies do not result in the desired outcome, often referred to as a Type III error.
Implementation science has emerged as a field commonly described as “the study of methods to promote the adoption and integration of evidence-based practices, interventions, and policies into routine health care and public health settings to improve the impact on population health” [ 4]. To date, much of the implementation science field has focused on studying the implementation of evidence-based practices or interventions in specific settings, most notably health care. The field has evolved to develop numerous theories, models, and frameworks (TMFs), such as the Consolidated Framework for Implementation Research (CFIR) [ 5], that guide studies of determinants of implementation, as well as frameworks such as RE-AIM [ 6] and Proctor’s Implementation Outcomes Framework (IOF) [ 7] to guide outcomes. Likewise, studies to examine implementation strategies have embraced compilations such as the widely used Expert Recommendations for Implementing Change (ERIC) project which identified 73 strategies to support the implementation of interventions [ 8]. Advancements in study design have included the development of hybrid models [ 9] and increased use of pragmatic methods [ 10].
At the same time, Dodson et al. define policy implementation science as “focused on generating knowledge to effectively spread research evidence among policymakers and integrate evidence-based interventions into policy designs” [ 11]. While several commentaries and papers have emerged in recent years on policy implementation science [ 12–18], there remains a lack of a common understanding and subsequent methodological advancement specific to policy implementation in the implementation science field. This gap has likely led to the disconnect that is seen in the public health literature and funded studies of policy “implementation” as well as the challenges that applicants and grant reviewers alike face when applying for or reviewing policy implementation science proposals. Similarly, many researchers studying policy implementation often are not formally trained in public policy or political science; rather, they seek to apply robust scientific methods from their disciplines (e.g., econometrics, decision and data sciences, medicine, etc.) and/or apply common implementation science frameworks, methods, measures, etc. to the study of policy implementation. Such scientific approaches may be inadequate or inappropriate, as they do not specifically address the unique drivers and/or context of policy implementation.
Thus, in the remainder of this commentary, we seek to provide a foundation on policy implementation research and science for the implementation science community writ large, for researchers interested in advancing policy implementation science, and for reviewers tasked with reviewing policy implementation science manuscripts or grant proposals. We first describe basic definitions of what “public policy” means and things that “governments do” that may fit within the domain of public policy. Second, we highlight the critical role that implementation places in the policymaking process. Third, we seek to differentiate what role the “policy” plays in a given implementation science study—is it the intervention, or the strategy—and provide examples. Finally, we provide a call to action of next steps to advance the science of policy implementation research.
Public policies are formal decisions made by federal, state, and local governments to act or not to act [ 19]; they are “whatever governments choose to do or not to do” [ 20]. Public policies include laws/legislation, ordinances, rules and regulations, executive orders, administrative procedures, and court decisions that carry the force of law in the given jurisdiction (e.g., USA, a specific state such as California, a specific local jurisdiction such as Chicago, or across jurisdictions). A decision to “do nothing” and maintain the status quo is as much a public policy decision as a decision to act or change a law or enact a new law [ 19]. Examples of public health policies include smoke-free air laws and ordinances; excise taxes on tobacco, alcohol, sugary drinks, and gasoline; nutrition standards governing foods sold or served in schools; state laws banning texting while driving; minimum drinking age laws; appropriations or funding of programs; insurance mandates (e.g., mammography screening mandates); legalization or decriminalization (e.g., marijuana laws); subsidies (e.g., EBT incentives for purchasing fruits and vegetables); motorcycle helmet laws; and many others. The common thread with all these examples is that they all were enacted or adopted by a government at the federal, state, and/or local levels (including through referenda or ballot initiatives in some states) and are intended to improve population health and behaviors.
It is important to note the distinction between big “P” policies, which are “things that governments do” (as noted above) and small “p” policies. Small “p” policies include “organizational guidelines, internal agency decisions or memoranda, social norms guiding behavior” [ 21] as well as noncodified policies, decisions, or actions made by governmental (e.g., an individual school’s practice governing recess before lunch) or nongovernmental bodies or institutions (e.g., a health care system, a nongovernmental accrediting body). For purposes of this commentary, we are intentionally focusing on big “P” public policies rather than small “p” policies because the definition of big “P” policies is fairly consistent regardless of whom one may consult; small “p” policies are often hard to define or explain and can easily be confused with programs or practices that governments or other agencies may implement.
The public policymaking process reflects a series of stages as illustrated in Fig. 1 [ 19]. Policy implementation (highlighted in the figure), including policy enforcement, is a critical stage in the process; however, the public health literature often assumes a priori that the “black box” of implementation occurs simply because a policy has been adopted or takes effect (e.g., just because a beverage tax or a smoke-free air law is enacted and takes effect, researchers equate adoption and effective date to implementation). Oftentimes, policies that are adopted or enacted, and implemented and/or enforced, may not reflect the best available evidence (due to partisan politics or other factors influencing the policymaking process); as a result, policy implementation may lead to implementation challenges (due to poor policy design) and/or less than desirable outcomes [ 21]. Policy implementation is also often not straightforward or linear, as depicted in the figure, and is usually messy. For example, Asada et al. conducted a series of qualitative studies to examine implementation of beverage taxes in Cook County, Illinois and Oakland, California as well as a narrative review of beverage tax implementation studies from jurisdictions across the country [ 2, 3, 22]. They identified four overarching and consistent challenges that researchers have identified with beverage tax implementation across jurisdictions including, but not limited to, a lack of (i) time to fully roll-out and implement the taxes post-enactment; (ii) preparation and planning for the administration and management of tax collection distribution; (iii) planning and processes to support the equitable allocation of tax revenues; and (iv) preparation for post-enactment opposition from the beverage and related industries and anti-tax opposition groups. Each of these factors has led to numerous implementation challenges with beverage tax implementation across the country; however, without studying the process of implementing and enforcing the tax across jurisdictions, policymakers, advocates, and researchers alike would not have a complete picture of why beverage taxes are not fully affecting beverage purchasing. Studying the process of implementing the policy (in this case a beverage tax) is critical to helping explain what happens in between policy adoption or enactment and policy outcomes or effects. This is where implementation science comes into play.
![A simplified depiction of the policymaking process [19].](https://oup.silverchair-cdn.com/oup/backfile/Content_public/Journal/tbm/13/11/10.1093_tbm_ibad034/2/m_ibad034_fig1.jpeg?Expires=1728559999&Signature=n97qi2Da5yzmAA5elDGIAUGvbcZk5AxAvfkN8TwiJ-BbL7oyui-Vrwo0TNP0N2LRvVPJAteo8EdPNgLxRSsd0inuwBXqHTapnHZXVz85fF~K5WEI0wHnGVZ97C2Ot3rWOiOvVxLXy2pbmzqEFrhtPS3vZuidb0U4wOPY3ETrCflnOTFRK8vqjUhQnZh~rSmg2cEkRF-3lTMbc1vBFjrcEQURmriffgfhc~wFSv36ObCjTdPwFzbObIFagjz-S-FCAyuKgAX9JTbHHMb6pdvT4OAEBE6LdlU32mlFXHBmFRDV-AWb~7UfbfLpqx6p2OMkViDDPnNtdR5NrdOu5yncyw__&Key-Pair-Id=APKAIE5G5CRDK6RD3PGA)
A simplified depiction of the policymaking process [ 19].
Public policies can be used as a means to directly impact the desired health-related outcomes and/or as a population-level strategy employed by governments through which to implement an evidence-based intervention that achieves the desired health-related outcomes. Along these lines, research assessing policy implementation must address the key conceptualization of what the role of policy is in a specific scenario: is policy the evidence-based “intervention,” the “implementation strategy,” or the basis or “vessel” for the intervention?
Studies assessing policy as the “intervention” seek to answer questions about how that policy can be implemented as intended. Such studies consider the policy as the evidence-based intervention and may attempt to test the effectiveness of a given implementation strategy or set of strategies on the successful implementation and/or enforcement of that policy. For example, smoke-free air laws and ordinances are an effective population-level public health policy approach to reducing tobacco exposure and are thus an “intervention.” An implementation science study in this context may seek to identify and test specific strategies (e.g., restaurant training, awareness campaigns, enforcement) to successfully implement such policies in given communities.
Studies assessing policy as the “implementation strategy” seek to identify how policy can be used to implement an evidence-based practice or intervention. For example, a fiscal policy (e.g., 1% tax on taxable household income exceeding $1 million) can provide a specific, earmarked funding stream for behavioral health services in a community (e.g., substance use and/or mental health treatment services) [ 23]. In this case, the policy itself is the strategy to improve access to evidence-based behavioral health services (i.e., it is itself the implementation strategy) [ 23]. An implementation science study in this context may seek to study how earmarked taxes as an “implementation strategy” could promote provider delivery and utilization of evidence-based behavioral health services [ 23].
Some policies also act as a “vessel” for specific interventions as elements in the policy design and/or implementation requirements. For example, the Affordable Care Act (i.e., the policy) authorized the Centers for Disease Control and Prevention to award grants to organizations to improve health and increase use of preventive services in underserved areas through community health workers (CHWs) [ 24]. In this case, the “intervention” is the use of CHWs; however, implementation of CHWs in communities is the result of policy language. However, the use of CHWs may vary greatly across communities which leads to the important question of how specific provisions or nuances within a given policy are implemented in practice and how such provisions or their implementation will greatly affect implementation outcomes.
The distinctions between policy as an intervention, as an implementation strategy, or as a vessel for identifying the “intervention” clearly has implications for all aspects of a given research study, starting with what the essential research question is. This conceptualization of the role of policy in a given study additionally should be clarified, and drive all aspects of the study, including selection of guiding frameworks, study design and associated methods, and outcome selection and measurement [ 25].
In this commentary, we highlighted the challenges with policy implementation science. There is much to be done to build this branch of the implementation science field. It is exciting that some of the more recent policy implementation science studies funded by NIH are starting to tackle some of these challenges and investigate policy implementation questions across a range of substantive areas; however, there is a lot more to be done. To mobilize the field into action, we have identified a few points for consideration as a starting point.
Much like the field expects economists to be part of implementation science studies involving costs or cost-effectiveness of implementing an intervention, we should equally expect policy or political scientists on studies involving policy implementation. The inclusion of such experts on policy implementation science teams will provide perspectives on the role of values, policy framing, power, and politics that otherwise may be missing from the study team [ 26]. For example, a policymaker or regulatory agency head’s position on the role of government intervention in behavioral decisions such as removing junk foods from schools will likely influence the nature of the policy that ultimately is enacted and implemented or enforced. Likewise, study teams should include policy actors involved with policy implementation (e.g., administrators in state departments of public health; entities who may be required to comply with the policy), similar to community-engaged or community-based participatory research approaches that study implementation of a community-based intervention. The inclusion of key implementers in the study team may help to ensure sustainability of the policy implementation effort once the study has concluded. However, it will also be critical to recognize the role that power and politics will play in nearly all policy implementation efforts. For example, when Oakland, California, was implementing their beverage tax, the revenue was to be allocated to the city’s general fund for reallocation for public health programming. However, the Mayor at the time attempted to reallocate the funds for their priority projects. After significant pushback from a range of stakeholders, the Mayor reversed course [ 22].
The implementation science field has exploded in recent years with TMFs. Increasingly, studies are applying these TMFs to study policy implementation (e.g., determinants, outcomes, etc.); however, TMFs have typically been designed for studying program or practice-based interventions and/or strategies. To our knowledge, only one policy implementation science framework exists within the implementation science literature [ 13]; more work is needed in this area. Studies are needed to determine if and when certain implementation science TMFs are more appropriate for studying policy implementation, align existing implementation science TMFs with policy questions, and/or determine if new TMFs are needed to specifically guide policy implementation science.
As noted earlier, there have been substantial efforts such as ERIC to outline common strategies that are used and tested in the implementation science field [ 8] as well as policy-related compilations of implementation strategies [ 13]. The latter compilation is an important starting point; it is organized around governance, financial, and service delivery arrangements as the intended “targets” of implementation [ 13]. More work is needed to identify how strategies may be tailored to the stage of public policymaking (see Fig. 1); for example, strategies that are effective during the policy adoption phase to ensure strong uptake may be different those that may facilitate implementation and/or enforcement. Importantly, future work should also begin to identify how and whether effective strategies may vary by policy domain or issue area (e.g., tobacco use, marijuana use, health care access, affordable food access, etc.).
By their nature, studies of policy implementation most likely will need to rely on pragmatic study designs [ 10]. However, from there, things get complicated. Many in the implementation science field are using hybrid study designs [ 9], particularly Hybrid Type III designs which primarily examine implementation outcomes and secondarily would examine policy (or intervention) effects, which may be well suited for studying policy implementation but have yet to be formally adapted for this purpose. There is a need to build the knowledge base regarding the types of study designs that may be well suited for policy implementation science research.
Commonly used implementation science outcome frameworks include RE-AIM [ 6] and Proctor’s Implementation Outcomes Framework (IOF) [ 7] and describe constructs such as reach, fidelity, and sustainability. Careful consideration must be given to the appropriateness of these constructs for policy implementation as well as to the distinctions in how they are operationalized and measured. There may be a need to refine our understanding of common constructs (e.g., what does fidelity mean in the context of policy implementation?) and/or develop new desired outcomes specific to policy implementation [ 15].
In order to realize the promise of policy implementation science for improving for population health, the field must first establish consistent understanding, language, and methods specific to policy implementation science. As we have detailed in this commentary, understanding the nuances of policy implementation science can facilitate the building of robust research teams, selecting a suitable TMF for the study, and identifying relevant implementation strategies, designs, and measures for the given policy. Using implementation science to our advantage for public health policy work will illuminate our path toward continued health promotion in the future.
The authors would very much like to thank the two anonymous reviewers for their helpful suggestions on an earlier version of this manuscript. Their suggestions have been incorporated into this final version of the paper. The views expressed here are those of the authors and do not represent the official policy or position of the National Institutes of Health, the US Department of Veterans Affairs, the US government, or the authors' employers.
Support for authors’ contributions to this manuscript was provided by the National Cancer Institute’s Consortium for Cancer Implementation Science (CCIS; under HHSN26120170003B, 75N91020F00002(TO 6), PI: J.F.C.); the Centers for Disease Control and Prevention’s Physical Activity Policy Research and Evaluation Network (PAPREN; U48DP006381; Multi-PIs: S.C.L. and J.F.C.); the National Heart Lung and Blood Institute (K12HL138049; S.C.L.); the National Cancer Institute (T32CA057711; N.R.S.).
Conflict of Interest: The authors declare that they have no conflicts of interest.
Authors’ Contributions: J.F.C. conceptualized and led the drafting and revisions to the manuscript. Y.A., N.R.S., A.K.-D., and S.C.L. all helped with conceptualization of the commentary and provided critical inputs and revisions to the manuscript. All authors approved the final version of the manuscript.
Ethical Approval: This article does not contain any studies with human participants performed by any of the authors.
Informed Consent: This study does not involve human participants and informed consent was therefore not required.
Welfare of Animals: This article does not contain any studies with animals performed by any of the authors.
Although a commentary, we will provide the search results from the PubMed or Reporter searches upon request from the corresponding author.
