Pathway map development for medical device event reporting in operating theatres: a human factors approach to improving the existing system

Introduction

Recent evidence from literature has shown that a large number of quality and safety problems in healthcare are a result of operational and system-related problems.1–5 Process mapping is a method often used in industrial engineering as a tool for improving quality and safety. It assesses the processes and systems into which a new intervention is introduced. Process mapping is used to gain a better understanding of the processes and systems under review and aid in introducing system improvements to be taken forward.1

A process is a series of connected steps or actions to achieve an outcome.6 A good understanding of any process is essential to quality improvement. While this process is common in other industries, its application is often lacking in healthcare.7 8 Process mapping assists in the identification of points in a system where improvements would have the biggest impact on patients and staff.6 Hence, it allows the members of the multidisciplinary team (eg, patient, doctors, nurses, managers, medical device industry representatives) to fully understand the problems from each other’s perspective. This way methods for improvement can be developed taking under consideration the effect on all team members.6 9

In recent years, process mapping has been applied to healthcare as part of quality improvement projects.1 9–12 It is found to be very useful for mapping activities in complex environments such as healthcare and it provides an insight into ‘work as done’ as opposed to ‘work as imagined’ and areas where the greatest risks exist.1 9 13–15 The healthcare environment is highly complex with processes which are variable and dynamic as well as dependable on the department or organisation in discussion.12 16 For example, the diagnostic process consists of a number of steps that start with the patient first engaging with the healthcare system and followed by a cycle of cognitive activities then diagnosis and treatment plan.17 When built of actual processes followed, process mapping has the capacity to break down the complexity of the system and provide a shared understanding of the work when it has been constructed using the experiences of those actually undertaking the processes. It does not, of itself, provide a full representation of ‘work as done’.12 ‘Work as done’ describes what actually happens in a particular situation. It takes into consideration the complexity of the environment and constantly changing conditions of work. ‘Work as imagined’ on the other hand assumes that work is completely analysed and prescribed. It is an idealised view of how a task is performed.18

To introduce improvements to patient care (in any aspect) it is essential to fully understand the process involved. The process itself has a starting and an endpoint, a defined group of users, a purpose, usually linked to other processes and it can vary from simple and short to complex and long.12 To fully understand, map and analyse the processes followed, enduser/staff involvement is essential and models for improvement have been produced by the NHS (National Health Service) Institute for Innovation which provides a framework for developing, testing and implementing positive changes.6 This model for improvement tries to answer three main questions. First, what are we trying to achieve, second how will we know that a change is an improvement and third what change can we make that will result in improvement. The plan–do–study–act (PDSA) cycle as part of the method for improvement aims to introduce new ideas in the healthcare setting through introduction in a small scale and learning from their potential impact. The cycle of learning from each structured change increases the chances of success.6 This process could be used in making improvements to the current system of reporting performance of medical devices and data gathering.

The Institute of Medicine has previously introduced the idea of learning health systems to harness the power of data and analytics to learn from patients and endusers and be able to feed this knowledge of ‘what works best’ back to the clinicians, manufacturers, public health officials and other stakeholders. The collection of appropriate data on the performance of medical devices in their context of use may assist in improving patient care. For example, the recording and analysis of patient safety issues arising from their use, complication rates associated with their use and even workarounds that result from common device problems may all affect the performance of a task. Appropriate collection and analysis of such data has the potential to help understand how to improve the underlying healthcare system. While in theory, the idea of systems generating this type of data would allow for a cycle of continuous service improvement, in practice real-life examples of these principles are rare.19 20

Technology has greatly advanced in recent years and could be effectively used to collect data on device performance and safety and help with the design of the pathway mapping. However, while it has the ability to allow data sharing and facilitate the relationship between stakeholders, these data need to be meaningful to the end users. This would allow use in patients as well as for research purposes and service improvement.19

This project aimed to develop the pathway of reporting and information transfer in operating theatres as it happens (ie, work as done). We aimed to gather data on the day-to-day processes followed in reporting malfunctions or failures of medical devices in operating theatres by using stakeholder interviews and survey. This was followed by interviews with Medicines and Healthcare products Regulatory Agency (MHRA) and manufacturers of medical devices to gain a better understanding of the process outside healthcare. We took this human-factors approach to identify methods for understanding the existing system and recognising methods for improvement.

Methods

We decided to concentrate on devices used in operating theatres because of the high volume of high-risk devices in this setting. These devices are used in a complex but controlled environment by a number of experienced professionals. A large proportion of surgical devices/equipment belongs to class 3 and 4 (intermediate to high risk) as per the MHRA’s and European Commission’s classification system,21 for example, cardiovascular catheters, biological adhesives, vascular prosthesis and stents, etc.21 In addition, surgical teams have regular contact with technology and representatives from medical device companies for complex devices including training in use.

This study consists of three main steps as follows:

1. A thorough literature review using Medline, Embase and PubMed carried out on medical device reporting and methods of developing pathway mapping.

2. Semistructured interviews and surveys carried out with end users (surgeons, senior nurses, manufacturers) and medical device safety officer (MDSO) to accurately map the pathway. Invites were sent to UK clinicians and manufacturers supplying surgical devices to the UK market.

3. Meetings were held with representatives from the MHRA to better understand the process of reporting and information transfer between healthcare, MedTech and regulatory bodies. Once qualitative data were analysed and coded, the developed pathway was reviewed with MHRA to ensure it is representative of the current steps followed by them. Methods of communication between MHRA, manufacturers and the trust were also clarified.

Pathway development occurred in a number of steps as shown in figure 1. Following the literature review, the pathway to reporting and information transfer was developed. The design and derivation of a process pathway mapping follows five distinct steps, which include (1) organisation and process identification, (2) information gathering, (3) map generation, (4) process analysis and (5) taking improvement forward (figure 1).12 22

Request permission

Different methods of pathway design are presented in the literature including hierarchical task analysis and sequential flow diagrams. A hierarchical task analysis approach to pathway mapping was followed as it allows for a greater granularity to be incorporated within the diagram.9 The developed pathway was compared with the one published by NHS Improvement (figure 2).

Request permission

The developed pathway was presented to and discussed with members of the National Institute for Health and Care Research- London In-Vitro Diagnostics Co-operative for comments and suggestions for improvement. Lucidchart software (Lucid Software) was used to design the diagrammatic representation of the pathway. The PPI group at Imperial College was consulted throughout this project.

The stakeholder interviews were also used to discuss methods for improvement.

Key principles of Lean Six Sigma were used as a comprehensive set of principles and tools that allow improvements in efficiency and effectiveness for organisation following pathway mapping.9 23 These principles were adapted to healthcare as shown in table 1.

A mixed-methods approach was used for participant recruitment. A mixture of convenience and snowball sampling methods was employed. All participants were consented prior to the start of the interviews, which were voice recorded. The survey had the consent incorporated at the start. An invitation email was sent to participants ahead of the interview containing the participant information sheet, what was required of them and the consent form for the study.

Thematic analysis of semistructured interview transcripts was carried out as guided by published literature24–26 to ensure a rigorous process. Interviews were transcribed verbatim by the first author (AT), to increase familiarity with the data. The same methods were used for the analysis of the surveys. The interviews were reviewed by a second reviewer to minimise bias in data analysis (GH). The survey data consisted of free-text entry and multiple-choice answers. The questions on the survey were derived from the interview themes to gain further stakeholder feedback. The data gathered was thematically analysed and used together with the interview data to finalise the study results.

The interviews were used to gather data on (i) steps followed by clinical teams in reporting malfunctions or failures of medical devices in operating theatres, (ii) steps followed by manufacturers following a report to them and (iii) steps followed by the MDSO and MHRA following an event. The data gathered was used to develop the pathway to reporting and information transfer. The developed pathway was discussed with MHRA including identified points for improvement and updated accordingly. A list of points where improvements could be applied and suggestions for these improvement were collected from the stakeholders.

The five qualities for good interpretation as presented by Yin26 were used at the data interpretation stage. Standards for Reporting Qualitative Research27 was used to structure and report the results of this study.

Results

The number of participants in the semistructured interviews and surveys is shown on table 2. The M:F ratio was 2.3:1 for the clinical teams and 8:1 for manufacturing teams. Collected data demonstrated that the existing pathway (‘work as imagined’) does not represent the actual pathway followed in the process of reporting (‘work as done’).

Pathway to reporting and information transfer

The Datix system of reporting is used to report all events that have or can lead to patient harm. Its data is nationally collected into NRLS (National Reporting and Learning System). Figure 2 represents the set pathway to reporting laid out by NHS England for reporting any patient related event. The pathway in figure 2 was reviewed and compared with the pathway developed in this study presented in figure 3. As clearly seen, the ‘ideal’ pathway presented in figure 2 differs greatly from the ‘actual’ pathways followed on a day-to-day basis by clinical staff. The ‘ideal’ pathway only includes the formal reports made through Datix for patient related incidents while the ‘actual’ pathway demonstrates all other routes of reporting and communication taken by clinical staff in operating theatres.

Request permission

Figure 3

Existing pathway to reporting and information flow in operating theatres (*by incident here we mean an event that has lead to patient harm). CAS, central alerting system; MDSO, medical device safety officer; MHRA, Medicines and Healthcare products Regulatory Agency; NRLS, National Reporting and LearningSystem.(**EBME - Medical, Biomedical and Clinical Engineering)

While all clinicians taking part in the study were aware of the Datix system, it was not used in all events related to medical devices. In operating theatres, the most common method of reporting a problem, is through direct contact with the manufacturer or their sales representative. This occurs mostly through a phone call or email (individual). While this method is effective in ensuring continuation of workflow, it lacks the process of data collection for future review.

Some devices are replaced and not reported by senior nursing staff in theatre. In these cases, reporting was not considered necessary. When a Datix form is submitted, this is reviewed by the MDSO in the trust. The cases which the MDSO considers appropriate are reported to the MHRA via the yellow card system. The remainder are assessed by the clinical engineering team. The MHRA communicates with the trust via the CAS (central alerting system) system. It informs the MDSO of safety incidents raised by other trusts or manufacturers. A log of the Datix forms, yellow cards and CAS reports received is kept with the MDSO and trust safety committee.

The analysis of the developed pathway (figure 3) identified a number of problem sources. In the event a device malfunction or failure, five possible pathways were possible (1) Datix form completion, (2) direct reporting to manufacturer (with or without a Datix completion), (iii) reported to clinical engineering, (4) reported to MHRA or (5) not reported. These pathways occur in parallel and hence are not inclusive of each other. This often requires more than one pathway to be simultaneously followed. The choice of the pathway(s) followed depends on the event and/or device being reported with generally no consistency.

The most common example of this is pathways 1 and 2, 2 and 3 or 1, 2 and 3 occurring together. Apart from the official Datix form submissions, no evidence was found of a reliable database or log of other events maintained in the trust. The unreported cases were difficult to estimate. Very often single use or single-patient use devices were thrown away unless expensive with no maintained log.

The system was found to be too complex with biases in decision making introduced at multiple points. These biases relate to the individual decision-making process which is variable and not based on a set protocol. They occur on all pathways developed in this study.

The proportion of devices reported via each pathway is not known to allow for further analysis. The knowledge on the different steps of this pathway is often poor among healthcare professionals. This relates to the clarity of information presented and lack of training. The complexity of the ‘work as done’ leads to a number of barriers to reporting which in turn affects the knowledge we have on device performances.

Discussion

This study developed a pathway map to reporting and information transfer for medical devices in operating theatres. It took a bottoms-up approach to identify differences with the existing pathway. Identification of these differences broke down the complexity of the system and provided a shared understanding of the ‘work as reported’. This led to identification of possible system improvement methods. Furthermore, this study led to a deeper understanding of the reasons behind poor reporting levels and lack of data available on medical device performance.

Quality improvement of medical device reporting and subsequent patient safety in relation to technology relies on the ability of the clinical teams to recognise the problem, have appropriate knowledge on the limitations of the device as well as knowledge of overcoming it or reporting the problem to the right person. This leads to a continuous learning process.

Hence, continuous quality improvements in healthcare rely on the organisation’s ability to learn from itself and the experiences of the people working within it.5

For organisational learning to occur, two inter-related processes need to occur, exploration and exploitation. Exploration involves the utilisation of new knowledge and technology. Exploitation involves taking advantage of existing knowledge from current staff. Both are important for organisational learning in healthcare, however, a good understanding of the balance to be achieved between the two is not yet accomplished.28 A different method being studied as an alternative to process mapping is that of process mining as a quantitative method that may allow a greater granularity of information gathered.29

Quality improvement methodologies in healthcare such as the PDSA, Healthcare Failure Modes and Effect Analysis30 Lean31 and Six Sigma32 require an understanding of the existing system and its examination from a new perspective to find where the greatest risks are before attempting to design improvement strategies.9 The development of this pathway map could be the first step in this process.

An understanding of the ‘actual pathways’ versus ‘set pathway’ are essential to understand the existing problems leading to the current situation. This difference is considered in the concept of ‘work done’ versus ‘work imagined’. In this study, we achieved ‘work as reported’ which is the closest point to ‘work as done’. A clear discrepancy was found between failures reported into the current system (through reporting) and those emerging from the interviews and surveys in this study. Hence, the use of process mapping is the closest point that allowed a more detailed understanding of the current system of reporting.

An efficient system needs to be resilient to daily changes in activities. For this to occur, a good understanding of the work processes and accurate description of work completed rather than work imagined needs to occur.33 This would lead to an understanding of the day-to-day variability of how work is carried out and how people adapt to the existing system.34 35

Each person has their own interpretation as to how their work is carried out as well as how this affects the existing regulations and operating procedures.36 37 This was observed during the stakeholder interviews. From this study, we could derive that in a complex environment such as the operating theatre, understanding ‘work as done’ is essential in ensuring safe management of all events. Furthermore, it would identify positive workarounds developed that could be incorporated within future improvements.

Application of improvement methods related to medical technology is complex. Currently, collaboration between healthcare and MedTech industries is limited. We require a system that works well for these complex industries. This system should allow for adequate ‘intelligent’ information transfer from the end users that affects learning and positive developments in both industries. Hence, we need to develop an effective information transfer ‘bridge’ between these two complex industries. An appropriate system that allows for this level of effective communication and data for regular interrogation would be an essential requirement to achieve improved levels of reporting and reduce subsequent patient-related risks.

The NHS Learn From Patient Safety Events (LFPSE) service (previously called Patient safety incident management system) which is replacing the NRLS system will continue to play an important role in recording all patient safety-related events.38 LFPSE is expected to play an important role in recording safety events related to medical technology in healthcare. The questions whether it will fully address all the complex issues related to technology reporting remains a valid one and remains to be seen once the system is live.

As errors cannot be fully eliminated,39 a reduction in patient risk can be achieved by the development, maintenance and continuous improvement of the capacity to detect and recover from these errors as soon as possible.40 An efficient reporting system for medical devices needs to constantly interrogate the data in order to pick up any evolving issues before they cause a patient related incident. This constant interrogation of data is not possible with the current level and quality of data available.

In order for improvements to be made, the main concerns regarding the existing system require addressing with increased training available, simplification of the current pathways and involvement of end users in all stages being the first priority. Second, an improvement on the level and quality of communication between healthcare and MedTech industry should be addressed as a crucial factor in ensuring safety and improvements of future medical devices.

Conclusions

By developing the actual pathway to reporting and information transfer, this study has clearly identified where the problems with the existing system lie. By doing so, we can now try to address these issues by identifying specific points for improvements. The developed pathway demonstrated a great complexity to the existing system not reflected by existing pathway. The identification of these differences and analysis of each step was possible due to utilisation of end user experience which is essential in quality improvement. The human factors approach and process mapping give opportunities not only for identification of problems but also addressing them in a systematic manner.

Study limitations

The pathway to reporting developed in this study is representative of general surgery operating theatres. Although these findings could be applied to other high-risk environments in healthcare such as endoscopy units, cardiac catheter laboratories and intensive care units, further research is required to study them in more detail and identify differences between these environments. Another limitation of this study is the group of clinicians involved. We appreciate that healthcare is variable and further issues and workarounds are present in other groups of healthcare workers. In this study, we could achieve ‘work as reported’. Although not the same as ‘work as done’ it still allowed a better understanding of the system and points for improvement. Further work is required to fully understand ‘work as done’ in healthcare. The quality improvement methods used in this study could be further applied to other areas of healthcare. Further work is also required to more efficient methods of communication between healthcare and MedTech.