We used the same process as used for Option Year 1 topics. Led by the Johns Hopkins University (JHU) Evidence-based Practice Center (EPC), we used a modified Delphi technique to obtain Technical Expert Panel (TEP) feedback on which patient safety practices (PSPs) have sufficient evidence to support widespread implementation, and the rationale based on the findings of the reports. We defined PSPs as interventions, strategies, or approaches intended to prevent or mitigate unintended consequences of the delivery of health care and to improve the safety of health care for patients.1 The TEP feedback form included brief summaries of the reports on the seven patient safety topics we reviewed, access to the full reports, and questions to capture their preliminary independent assessment of the specific PSPs covered in the reports.

We asked the panel to independently indicate a global judgment about whether health care facilities should adopt the specific PSPs included in our reports, and to indicate priorities for addressing limitations in the evidence on each PSP. By “global judgment,” we mean a summary judgment that takes into consideration the following factors:

• importance of the safety problem [frequency & severity];
• rationale for the PSP;
• evidence on whether the PSP can improve outcomes;
• potential for unintended consequences;
• difficulty of implementing the PSP based on reported barriers, facilitators, resources needed, and available toolkits to support implementation;
• each expert’s experience as a researcher, clinician, policymaker, or patient safety advocate.
  

We offered the following categorical scheme to guide judgements for the topics, including a category, “not enough evidence to rate,” to allow experts to not rate a PSP when the evidence is limited.

1. Strongly Encouraged for improving patient safety: Evidence is strong enough to be certain that, if we were choosing a hospital for care of loved ones, we would choose a hospital that was implementing this PSP over one which was not. Unless hospitals know their outcomes for this safety problem are already excellent, most organizations should be implementing this PSP.
2. Encouraged for improving patient safety: Enough evidence exists to determine that, if we were choosing a hospital for care of loved ones, we would choose a hospital that was implementing this PSP over one which was not, but we have uncertainty about the effectiveness of the practice or concern about implementation barriers or costs that keep us from putting it on the “strongly encouraged” list. Unless hospitals know their outcomes for this safety problem are already excellent, many organizations should be implementing this PSP. Each organization would need to consider whether it has capacity for implementation.
3. Discouraged for improving patient safety: Evidence on the effectiveness of this PSP does not justify concerns about potential harms of the PSP, costs, or implementation barriers. The balance of information suggests that harms and costs of implementation may outweigh the potential benefit.
4. Strongly Discouraged for improving patient safety: Evidence on this PSP indicates that it is harmful or ineffective, in which case the costs of implementation cannot be justified.
5. Not enough evidence to rate.
6. Prefer not to rate this practice.
   

Seven of 15 TEP members completed the online feedback form before the TEP meeting.

The JHU EPC collated the results of the initial feedback and prepared a series of slides to present to the eight of the 15 TEP members who were able to attend the virtual meeting on February 21, 2025. The slide presentation included an overview of the purpose of the project and the prioritization process, followed by a listing of topics and a summary of findings, including the pre-meeting TEP feedback.

During the TEP meeting, we asked the TEP members to share their thoughts about what to recommend based on the findings from the reports completed in Option Year 2. After the discussion of each topic, we asked the TEP members to independently submit their votes on whether to strongly encourage, encourage, discourage, or strongly discourage PSPs for adoption. At this stage, TEP members could decide to vote neutral, indicating they neither encouraged nor discouraged PSPs for a topic or there is not enough evidence to rate.

Two TEP members who were unable to submit their votes during the meeting were allowed to submit votes after the meeting. We then collated the results of the final voting. Five of the TEP members who participated in the first year of the project were not able to participate or provide feedback in the second year.

In this report, we present a short summary of our findings on each patient safety practice (PSP) topic, followed by a summary of the ratings and comments from the Making Healthcare Safer (MHS) IV Technical Expert Panel (TEP). The summaries of findings presented here are a condensed version of the summaries shared with the TEP prior to the meeting. The final MHS-IV reports will be available at: Making Healthcare Safer IV.

1. Patient Monitoring Systems To Prevent Failure To Rescue

1a. Findings From Previous MHS Reports

• MHS III (2020) concluded that patient monitoring systems (PMS) with continuous monitoring (CM) moderately reduced rescue events and moderately improved hospital length of stay but had inconsistent results and did not significantly impact mortality. PMS with intermittent vital sign input inconsistently affected mortality.

1b. MHS IV Rapid Review Findings

• Literature Search
  • Searched PubMed and Cochrane Library for systematic reviews and primary studies (2018–2024)
  • 3 reviews, and 30 additional studies covering 3 types of interventions:
    • New standard of care (e.g., modifying criteria for using telemetry in order to reduce unnecessary telemetry use)
    • Change in PMS modality (e.g., changing to wireless continuous monitoring) or
    • Other interventions including a change in PMS parameters or use of a scoring system, artificial intelligence, or communication cascade
• Evidence Summary3
  • Implementation of an early warning system alone did not improve the relevant outcomes.
  • In primary studies of adults, none of the interventions showed consistent improvement in relevant outcomes (low/insufficient strength of evidence [SOE]). However moderate evidence suggests that changing the monitoring modality may reduce transfers to higher levels of care and length of hospital stay and modifying monitored parameters was associated with decreased hospital mortality (moderate evidence).
  • There was insufficient evidence for pediatric populations.
  • The new evidence supports the previous MHS III report's findings that continuous monitoring and early warning scores are key interventions for addressing failures in the afferent limb of rapid response systems and preventing failure to rescue. Additionally, this review confirms the moderate evidence suggesting continuous monitoring's benefit in reducing hospital mortality, cardiac arrest incidence, and length of stay, while noting variability across studies.
• Barriers, Facilitators, and Resources
  • Barriers included understaffing, time pressure, and nurse burden.
  • Facilitators included clinician engagement, organizational resources, interdisciplinary teamwork, workload reduction, and integrating vital sign monitoring data.
  • None of the studies detailed the resources required, but human resources are essential. Commercial scoring systems and PMS hardware have acquisition and ongoing costs.

1c. Ratings and Comments From MHS IV TEP

• Most of the TEP encouraged adoption of each of the three types of patient monitoring system interventions, with 80 percent encouraging or strongly encouraging a new standard of care, 70 percent encouraging a change in monitoring modality, and 60 percent encouraging or strongly encouraging other interventions. (Figures 1a, 1b, and 1c).
• The TEP’s primary future research recommendations focused on enhanced interventions, and methods of implementation to overcome barriers (Figure 2).

2. Use of Structured Handoff Protocols for Intrahospital Within-Unit Transitions

2a. Findings From Previous MHS Reports

• In MHS III, handoffs were discussed in “cross-cutting PSPs” based on 3 studies of handoff protocols. Only one study reported the effects of handoff protocols on patient outcomes, and it found insignificant differences in outcomes during transfers from the operating room to the surgical intensive care unit.
• Structured tools for within-unit handoffs were not discussed in isolation in prior MHS reports.

2b. MHS IV Rapid Review Findings

• Literature Search
  • Searched PubMed, EMBASE, PSNet, and CINAHL for systematic reviews and primary studies (2013-2024)
  • A search for evidence on 12 different structured handoff protocols (identified from literature and experts) was conducted.
  • 2 systematic reviews and 13 studies (4 randomized controlled trials (RCT) and 9 pre-post) that examined two different structured handoff protocols:
    • SBAR tool (situation, background, assessment, recommendation)
    • I-PASS tool (illness severity, patient summary, action list, situation awareness and contingency plans, and synthesis to receiver)
• Evidence Summary4
  • Almost all studies were conducted in academic teaching hospitals and assessed physician-to-physician handoffs.
  • SBAR tool: 2 systematic reviews and 2 new studies (1 RCT) provided low certainty evidence that use of the SBAR tool can improve patient safety outcomes such as adverse events.
  • I-PASS tool: 10 studies (2 RCTs) provided moderate certainty evidence that the I-PASS tool can improve patient safety outcomes such as medical errors and adverse events.
  • Multiple co-interventions and implementation strategies were used with the I-PASS tool.
• Barriers, Facilitators, and Resources
  • Barriers included resistance to change, and time needed to use the tool.
  • Facilitators included education, monitoring and use of a resident champion.
  • None of the studies provided details about resources required (cost, staffing, or time), but several studies detailed the co-interventions used to implement the tool.

2c. Ratings and Comments From MHS IV TEP

• Discussion during the meeting highlighted that the research on handoff protocols is limited by study design. IPASS results varied, with discussion on whether to recommend specific protocols or broader structured handoff approaches, acknowledging the risk of inconsistent implementation (Figures 3a, 3b, and 3c).
• The TEP’s primary recommendations for future research were studies targeting different populations, studies focused on methods of implementation, and studies to identify facilitators and barriers to implementation (Figure 4).

3. Supply Chain Disruption Monitoring (SCDM) Programs

3a. Findings From Previous MHS Reports

• Previous MHS reports did not cover this topic.

3b. MHS IV Rapid Review Findings

• We defined supply chain monitoring programs as any formal process or system to identify potential disruptions at any node of the supply chain external to the individual organization.
• Literature Search
  • Searched PubMed, Embase and Cochrane Library for systematic reviews and primary studies (2010-2024)
  • 7 studies (5 case reports, 1 survey/qualitative study, and 1 narrative review; reported in 9 publications)
• Evidence Summary
  • No studies addressed the effectiveness review question.
  • Seven studies provided details about implementation facilitators, implementation barriers, and resource requirements for healthcare-related SCDM programs. These studies described various types of SCDM programs and supplies/products.
• Barriers, Facilitators, and Resources
  • Barriers included substantial up-front financial costs and staff time, disjointed communication, rapid development and implementation timelines, limited electronic health record functionality, consolidating and interpreting information from multiple data sources, and data coding errors.
  • Internal facilitators included effective communication, dedicated development time, availability of existing infrastructure, technology, and expertise (e.g., formulary governance structure, electronic health records (EHR), automated dispensing cabinets, health information technology (HIT) expertise). External facilitators included existing relationships with wholesalers and suppliers.
  • Resource requirements included up-front monetary investment, staff time and training, technology, and HIT expertise. Specific costs varied widely depending on the complexity, individual components of SCDM programs, and size of the organization.

3c. Ratings and Comments From MHS IV TEP

• TEP members did not express a consistent viewpoint about this topic. Five of 10 TEP members voted to encourage SCDM programs, and five felt there was not enough evidence to rate (Figure 5).
• The discussion focused on the lack of research on healthcare supply chains, despite their impact on patient safety.
• Participants emphasized that even without robust healthcare-specific evidence, the need for supply chain disruption monitoring is clear. Healthcare professionals consistently report being diverted from patient care due to supply disruptions, and these disruptions can lead to patient safety issues.
• TEP members emphasized the need for proactive planning and learning from other industries to address this gap. The group agreed that simply stating "don't do it" due to lack of evidence is not a good option. The panel acknowledged the necessity for further research and recognized the issue's importance.
• The TEP’s primary future research recommendations focused on methods of implementation to overcome barriers, studies to identify facilitators and barriers to implementation, stronger designs, better outcomes, and enhanced interventions (Figure 6).

4. Examining the Impact of Implementing High Reliability Organization (HRO) Principles on Patient Safety Outcomes

4a. Findings From Previous MHS Reports

• Previous MHS reports did not cover this topic.

4b. MHS IV Rapid Review Findings

• Literature Search
  • Searched PubMed and Cochrane Library for systematic reviews and primary studies (2019-2024)
  • One rapid evidence review and one pre-post study were identified.
    • The review summarized and updated the results of the older 2019 evidence review from the Veterans Affairs (VA) examining HRO principles. The updated search in this review identified two additional effectiveness studies for inclusion, but the authors did not report new insights or conclusions from those described in the original 2019 evidence brief.
    • The pre-post study involved a single VA medical center (Truman). At Truman, the National Center for Patient Safety implemented a comprehensive high-reliability hospital model based on HRO principles during a 3-year period.
• Evidence Summary
  • A 2022 review of 23 studies (plus 2 additional effectiveness studies found in an updated search) suggested that multi-component HRO interventions lasting at least 2 years may improve patient safety, but the evidence was weak.
  • A single-center study of a comprehensive HRO model showed initial improvements in several safety measures, but some worsened after the intervention ended, and the study had significant bias concerns.
  • While data suggests that sustained implementation of HRO principles in organizations may improve patient safety outcomes, the published evidence on multicomponent intervention implementation is limited, making it challenging to draw conclusions about the effectiveness of specific interventions
• Barriers, Facilitators, and Resources
  • Barriers include budget limits, competing priorities, staffing issues, and lack of HRO expertise.
  • Successful implementation requires leadership engagement, shared learning, and dedicated resources.
  • Resource needs are unquantified but often include consultants, training, and culture assessments.

4c. Ratings and Comments From MHS IV TEP

• All TEP members voted that there was not enough evidence to rate this practice. The TEP panel discussed the challenges inherent in the implementation and evaluation of HRO principles within healthcare settings. Specifically, participants questioned the empirical validation of individual HRO components, such as patient safety assessments and leadership walk-arounds. Concerns were raised regarding the difficulties of generalizing findings from vertically integrated health systems, potential publication bias, and the overlap between HRO initiatives and existing patient safety measures (Figure 7).
• The TEP’s future research recommendations focused on stronger study designs, better outcome measures, longer followup, methods of implementation to overcome barriers, appropriate comparison groups, and identify facilitators and barriers to implementation (Figure 8).

5. Programs for Responding to Harms Experienced by Patients During Clinical Care

5a. Findings From Previous MHS Reports

• Previous MHS reports did not cover this topic.

5b. MHS IV Rapid Review Findings

• Literature search
  • Searched PubMed and Cochrane Library for systematic reviews and primary studies (2010-2024)
  • 0 reviews, 8 primary studies
  • Eight studies discussed 5 Communication and Resolution Programs (CRPs) that share several components but vary on specifics and implementation. We categorized CRPs into 2 groups based on design and implementation:
    • Most Robust: Seven Pillars (Lambert 2016, Helmchen 2016), CARe (Mello 2017, Kachalia 2018), University of Michigan Health System claims management program or disclosure-with-offer program (Kachalia 2010, Adams 2014)
    • Least Robust: LeCraw 2018 and Mello 2014
• Evidence Summary
  • No studies discussed the effectiveness of CRPs on patient safety outcomes.
  • The most frequently reported outcomes were rate of claims, time to resolution, and cost to healthcare organizations.
  • The most robust interventions were associated with fewer malpractice claims (4 studies) and reduced defense legal expenses (3 studies) with low SOE. There were conflicting results with insufficient evidence on other outcomes related to malpractice experience and cost to organizations.
  • The least robust interventions had insufficient evidence on the effects on the rate of malpractice claims (1 study) or defense legal fees and expenses (1 study).
  • Despite their patient-centered design, studies of CRPs’ effects have focused on organizational liability and costs rather than patient-oriented outcomes.
• CRPs have positive or neutral effects on measured outcomes, with no significant negative effects. Barriers, Facilitators, and Resources
  • Barriers included a lack of leadership support, insufficient resources, and a lack of engagement and training.
  • Facilitators included leadership support, culture of safety, effective collaboration, robust processes, and clinician engagement and support.
  • Data were limited about specific information on resources.

5c. Ratings and Comments From the MHS IV TEP

• All TEP members voted to encourage or strongly encourage the most robust programs (Figure 9a).
• All but one member of the TEP voted that there was not enough evidence to rate the least robust programs responding to harms experienced by patients during clinical care (Figure 9b).
• The discussion focused on how to measure the effectiveness of CRPs after patient harm. Some panel members shared their impression that CRPs’ primary goals are to prevent secondary harm after events and improve the experiences of the involved patients, families, and clinicians, rather than to improve patient safety by preventing future events. Another panelist noted that patients who have been harmed, and their families, are very interested in ensuring that their experiences help drive learning and improvement that makes care safer, and that because CRPs are embedded in patient safety programs, they are well positioned to promote patient safety improvements. Participants agreed there was a need for more studies on outcomes and experiences of the parties involved, especially from the patient and family perspective.
• The TEP’s primary future research recommendations focused on better outcome measures, stronger designs, and studies targeting different populations (Figure 10).

6. Interventions To Prevent Nonventilator Hospital-Acquired Pneumonia (NV-HAP)

6a. Findings From Previous MHS Reports

• Previous MHS reports did not cover this topic.

6b. MHS IV Rapid Review Findings

• Literature Search
  • Searched PubMed, Embase, and Cochrane Library for systematic reviews (2019-2024) and primary studies (2010-2024)
  • 18 primary studies across 3 types of PSPs:
    • Oral care (6 studies)
    • Dysphagia screening and management (7 studies)
    • Prevention bundles (5 studies; all 5 prevention bundles included oral care as one component of their multicomponent bundles)
• Evidence Summary
  • Prevention bundle studies reporting NV-HAP incidence favored the PSP (low SOE).
  • For all other reported outcomes for each PSP, we rated the evidence as insufficient, mostly due to high study limitations and imprecision. Therefore, we were unable to conclude whether PSPs for oral care or dysphagia screening and management are effective in preventing NV-HAP and other undesired outcomes.
• Barriers, Facilitators, and Resources
  • Barriers included staff perceptions and discomfort providing oral care, resistance to change, patient volume/limited time, unclear protocols, competing organizational priorities/changes, and supply quality/availability.
  • Facilitators included staff training/education, institutional support, consistent material and process monitoring, information technology support, access to analytics, multi-disciplinary teams, standardization of care, and simple interventions.
  • Few studies provided details about implementation resources.

6c. Ratings and Comments From the MHS IV TEP

• A majority of the TEP members voted that there is not enough evidence to make a recommendation (Figures 11a, 11b, 11c, 11d, and 11e).
• Five pre-post studies showed positive results for prevention bundles, but the overall evidence was weak due to high risk of bias. Those voting "not enough evidence" cited concerns over study sizes, weak evidence, and inconsistent individual component effects. Those voting "encourage" pointed to indirect evidence from ventilator-associated pneumonia studies and past positive results with similar bundles.
• The TEP’s primary future research recommendations focused on studies with stronger designs, better outcome measures targeting different populations, enhanced interventions, and methods of implementation to overcome barriers (Figure 12).

Rapid Response Summary

In contrast to rapid reviews, which used streamlined systematic review methods, rapid responses were narrowly focused based on preliminary search results and consisted of a descriptive summary of a small number of recent relevant studies and systematic reviews.

7. Acute Care Nursing Staff Shortages That Compromise Patient-to-Nurse Ratios

7a. Findings From Previous MHS Reports

• This topic was not specifically addressed in previous MHS reports, although the MHS II report found moderate strength of evidence on improving nurse-to-patient staffing ratios to prevent patient deaths.

7b. MHS IV Rapid Response Findings

• Literature search
  • Searched PubMed, CINAHL, and Cochrane for systematic reviews and primary studies (2013-2024)
  • 6 systematic reviews and 16 studies (mostly observational and focused on quality improvement initiatives)
  • Included studies discussed interventions at four different levels: staff initiatives, nursing team scheduling and staffing-level interventions, organizational structure, and government-level policies.
• Evidence Summary
  • Mandated nurse ratios generally improved patient and staff outcomes (lower mortality, fewer safety issues, and better nurse retention were observed).
  • Acuity tools showed mixed results. They improved nurse satisfaction in some cases, but not overall patient outcomes in others.
  • Other interventions were too heterogeneous to support any definitive conclusion about their effectiveness.
• Barriers, Facilitators, and Resources
  • Barriers included workload, staffing issues, lack of resources, funding, and training, implementation challenges, and negative workplace.
  • Facilitators included teamwork, nurse engagement, resources, nurse experiences, and collaboration.
  • The most commonly cited facilitating resource was having adequate staff.

7c. Ratings and Comments From the MHS IV TEP

• The TEP did not express a consistent viewpoint on the interventions addressing nursing staff shortage in acute care settings (Figures 13a, 13b, 13c, 13d, and 13e).
• For nursing team scheduling interventions (Figure 13b) and government-level policy interventions (Figure 13e), the majority of TEP members voted to encourage. For the remaining practices, most TEP members voted that there was not enough evidence to rate.
• Some interventions had merit, but it was unclear what might be effective or impactful.
• TEP’s primary future research recommendations focused on better study designs, better outcome measures, longer followup, and methods of implementation to overcome barriers (Figure 14).

Table 1 presents a summary of the advice from the Technical Expert Panel (TEP) about whether to encourage or discourage widespread use of the patient safety practices (PSPs) covered in our reports. The table includes categorization of the advice in terms of whether the TEP encouraged, discouraged, or was neutral about each PSP. Although the TEP concluded that evidence was insufficient to support a recommendation for some of the PSPs, and had mixed advice about one of the PSPs, the TEP reached consensus about encouraging 9 specific PSPs.

Limitations Across All Topics

Across the topics of this report, PSP interventions face several barriers to implementation that affect study design quality and may limit the evidence for positive impact on patient safety and quality outcomes. Significant limitations were present in each topic, impacting the ability to generalize the findings. Common issues include high risk of bias in included studies, significant heterogeneity in patient populations and interventions, and inconsistent outcome measures. Many topics lacked robust statistical analysis, relied on observational studies, and were limited by the rapid review process. Specific topics were further limited by factors such as a lack of research (supply chain monitoring), a focus on specific settings or populations (e.g., acute care, Veterans Affairs hospitals). Additionally, issues like poorly reported outcomes (interventions to prevent nonventilator hospital-acquired pneumonia), and the confounding impact of the Coronavirus Disease 2019 pandemic (nursing staff shortages) further restricted the conclusions that could be drawn.

Implications for Future Research Across All Topics

TEP ratings of future research needs varied widely across PSPs reviewed in this report. However, several trends are apparent in the TEP’s recommendations. The most commonly cited high priorities for future research included:

• Methods of implementation to overcome barriers;
• Stronger designs;
• Better outcome measures to assess effectiveness;
• Prioritize studies targeting different populations;
• Enhanced interventions for improved outcomes;
• Identify facilitators and barriers to implementation.
  

All PSPs had at least one of these above-mentioned topics in their highest rated research priorities.

Future research should focus on improving study quality and develop effective implementation strategies, including the development of implementation toolkits and training programs. Addressing implementation barriers, such as infrastructure costs and data transparency, is critical for making sure that research findings translate into meaningful patient care improvements.

1. Rosen M, Dy SM, Stewart CM, Shekelle P, Tsou A, Treadwell J, Sharma R, Zhang A, Vass M, Motala A, Bass EB. Final Report on Prioritization of Patient Safety Practices for a New Rapid Review or Rapid Response. Making Healthcare Safer IV. (Prepared by the Johns Hopkins, ECRI, and Southern California Evidence-based Practice Centers under Contract No. 75Q80120D00003). AHRQ Publication No. 23-EHC019-1. Rockville, MD: Agency for Healthcare Research and Quality. July 2023.
2. Bass EB, Shekelle P, Treadwell J, Rosen M, Mull NK, Stewart CM, Motala A, Zhang A, Sharma, R. Making Healthcare Safer IV—Summary of Findings on Year 1 Topics. (Prepared by the Johns Hopkins, ECRI-Penn, and Southern California Evidence-based Practice Centers under Contract No. 75Q80120D00003). AHRQ Publication No. 23(24)-EHC019-15. Rockville, MD: Agency for Healthcare Research and Quality. May 2024. DOI: https://doi.org/10.23970/AHRQEPC_MHS4YEAR1.
3. Winters BD, Rosen M, Sharma R, Zhang A, Bass EB. Patient Monitoring Systems To Prevent Failure To Rescue. (Prepared by the Johns Hopkins Evidence-based Practice Center under Contract No. 75Q80120D00003). AHRQ Publication No. 25-EHC008-1. Rockville, MD: Agency for Healthcare Research and Quality. December 2024.
4. Shekelle PG, Motala A, Lawson E. Use of Structured Handoff Protocols for Intrahospital Within-Unit Transitions. Making Healthcare Safer IV Rapid Review. (Prepared by the Southern California Evidence-based Practice Center under Contract No. 75Q80120D00003). AHRQ Publication No. 25-EHC008-2. Rockville, MD: Agency for Healthcare Research and Quality. January 2025.

This evidence review was funded by the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services, under contract no. 75Q80120D00003.

Bass EB, Shekelle P, Treadwell J, Rosen M, Motala A, Zhang A, Sharma R. Making Healthcare Safer IV: Summary of Findings on Patient Safety Practices and Ratings by a Technical Expert Panel, 2024-2025. (Prepared by the Johns Hopkins, ECRI-Penn, and Southern California Evidence-based Practice Centers under Contract No. 75Q80120D00003). AHRQ Publication No. 25-EHC008-8. Rockville, MD: Agency for Healthcare Research and Quality. May 2025. DOI: https://doi.org/10.23970/AHRQEPC_MHS4YEAR2. Posted final reports are located on the Effective Health Care Program search page.