Error and harm in healthcare are common, costly and often preventable.1 2 The scale of the problem is a major policy concern. In the UK, adverse events may occur in 10% of all hospital admissions, with many thought to be preventable. They inflict substantial human suffering and have a significant financial cost of at least £1 billion per year.3^–5 In primary care, around 300 million consultations take place annually, but evidence of error and harm is comparatively limited because of insufficient research. However, one study suggests that errors may occur in 1 per 120 consultations, but this is likely to be an underestimate.6

A multi-method approach is now accepted as the most effective strategy for improving the quality and safety of healthcare.7 In the UK a variety of methods is used in learning from error and preventing harm, including significant event analysis, criterion-based audit, case note review, voluntary incident reporting and the review of litigation claims. A relatively new improvement method, which originated in the USA, involves the use of “trigger tools” to highlight errors and adverse events through the focused review of patient records.8

The trigger method is endorsed by the Institute of Healthcare Improvement (IHI) because it has been shown to be a reliable technique for uncovering harm and measuring change over time. It also encourages a move away from the over-riding policy focus on quantifying healthcare error to concentrating energies on measuring and addressing patient harm. Hospital-based studies have provided evidence of how efficient the trigger method can be in detecting medication-related harm and other adverse events in surgical, paediatric, neonatal and intensive care units and in measuring adverse drug reactions in hospitalised patients.9^–12

“Triggers” are defined as easily identifiable flags, occurrences or prompts in patient records that alert reviewers to potential errors and adverse events that were previously undetected. The development of appropriate triggers allows rapid, systematic and standardised review by trained reviewers. For example, an international normalised ratio (INR) of >5 would be a “trigger” for the reviewer to undertake a more focused examination of the record for evidence of bleeding and whether this caused harm. The error rate and levels of patient harm can be quantified as, for example, error per 1000 hospital days or harm per 1000 consultations. In addition, the trigger method may provide the healthcare team with a better understanding of where harm could occur or is occurring to enable them to develop targeted improvement strategies. Error and harm rates can then be monitored and improved over time.

A lack of information about the nature and extent of adverse events in primary care has been identified as a key barrier to achieving sustainable improvements in quality and safety. The trigger method may have the potential to close this evidential gap by providing greater insight and understanding of what can go wrong. However, evidence of the use of trigger tools in non-acute settings is limited, and there is some doubt over their transferability between different healthcare environments. Given the evidence for their effectiveness and the desire to improve patient safety in the UK, it is clear that this approach should be tested as a potential strategy for identifying error and harm in primary care.

The aim of this preliminary study therefore was threefold:

1. To develop and test a validated global trigger tool for the primary-care setting.

2. To determine the rate of previously undetected error and quantify levels of harm identified in a sample of patient records.

3. To describe the types and severity of errors and patient harm uncovered.

METHODS

Definitions

In this study “error” was defined as the failure of a planned action to be completed as intended or the use of the wrong plan to achieve an aim. “Adverse event” was defined as an injury (harm) caused by medical management rather than the underlying condition of the patient. An adverse event attributable to error is a “preventable adverse event.”1

Global trigger tool development

Literature review

A comprehensive literature review was performed to identify published studies involving trigger-based methods. Electronic databases (Medline, Embase and PsycINFO) were searched using the following terms: “adverse events”, “error”, “harm”, “trigger tool”, “patient records”, “general practice”, “family practice” and “primary care”. Grey literature and patient safety-related websites were also searched. Eleven relevant research papers were identified, and their references were cross-checked to identify a further 16 articles of interest. After review of the evidence, the IHI outpatient adverse event trigger tool was identified as the most relevant as a starting point for developing a primary-care equivalent.

Instrument development

The IHI outpatient trigger tool consists of 11 triggers, most of which were adapted for primary care by the lead author (CdW) to ensure their relevance. The west of Scotland General Practitioner (GP) Audit Development Group was recruited to participate in further instrument development, refinement and content validity using a modified Delphi technique. The group consists of 20 GPs who are highly experienced in the educational assessment of criterion audit projects and significant event analysis reports undertaken by their peers or doctors-in-training. The group was accorded “expert” status because of this experience. In total, 17 members contributed to the emerging instrument by suggesting additional, different or modified triggers in an iterative manner over a 4-week period. Triggers were chosen because their presence or absence in patient records is consistently documented and, when found, they potentially signal an error and/or adverse event. Other triggers could be used, but they may not be consistently found in patient records—for example, complaints or patient satisfaction survey findings. A 10-item global trigger tool was agreed (table 1).

Preliminary testing of global trigger tool

Setting, sample and subjects

A convenience sample of five urban GP practices was recruited through GP educationalists with close links to NHS Education for Scotland (NES), a special health authority with responsibility for the education and training of the healthcare workforce. Two practices were based in Edinburgh, two in Glasgow and one in Lanarkshire. All used the General Practice Administration System for Scotland (Gpass), an electronic medical record database used by over 80% of Scottish general practices. For pragmatic time and resource reasons, a random sample of 100 clinical records was selected per practice from Gpass using a random number generator.13 The only criterion that excluded a patient’s record from the study was being registered with that practice for less than 12 months.

Data collection and study timeframe

Five GPs and two practice nurses received a short training session in the use of trigger tools from an IHI-trained patient safety expert to ensure standardised record review and interpretations. Two auditors (data collectors) were assigned to each practice, and they initially reviewed records independently to improve reliability.14 The 12-month period from 1 January 2007 to 31 December 2007 was reviewed for each record. Auditors answered the following four questions whenever they found a trigger while examining each patient record:

1. Did the patient suffer actual harm or was there potential for harm?

2. Was harm caused by clinical management (ie, an adverse event)?

3. Was it preventable?

4. What is the severity rating of the adverse event?

Auditors were trained to consistently apply more conservative ratings whenever they had any doubt in grading severity or judging preventability. If the same harmful event was identified by more than one trigger, the researchers chose the single trigger with most bearing on that event or the trigger that occurred first.

The following data were collected on a standard pro-forma: presence of triggers, errors and adverse events, event severity, preventability judgement, age, sex, consultation types, and time taken for record review. A total of 174 h of protected time was allocated, with 14 h training time and 160 h for data collection.

Error and adverse event severity was graded using the National Coordination Council for Medication Error Reporting and Prevention index for categorising errors (table 2). In secondary-care studies, only categories E–I have been used, as they correlate with actual patient harm occurring. We included all categories because we assumed that relatively minor events may be more common in primary care.

Data validity

After the two auditors had independently applied the trigger tool in each practice, they met to compare and discuss their individual findings. Where differences in any aspect of the data collected occurred between them, they re-accessed the patient’s record in question and jointly reviewed it until a consensus agreement was reached.

Data analysis

All data were coded and entered into Microsoft Excel. SPSS V14 was used for descriptive analysis. The numbers of positive triggers and adverse events together with severity level and whether events were preventable were calculated.

RESULTS

Basic data and patient consultations

Of the 500 records reviewed, 252 were for female patients (51%), and the mean age of all patients was 38.3 years (range 1–99). The records indicated that 2251 consultations took place in the study period, which equated to a mean of 4.5 consultations per record reviewed. These consisted of 1293 face-to-face consultations with GPs (in practice or at the patient’s home), 486 with the general practice nurse, 410 telephone consultations with GPs, 42 with the health visitor, and 29 with other members of the primary-care team (eg, community-based psychiatric nurses, midwifes and clinical psychologists). A higher rate of consultation was associated with increased incidences of harm and also with the need for more time to review the record (table 3).

Incidence of triggers, errors and patient harm severity

A total of 730 positive triggers was identified in the 500 patient records (table 4). Further review of these triggers in 47 records led to the identification of patient harm (9.4%). In a further 17 records, the triggers led to the discovery of errors, but harm to the patient either did not occur or was prevented by intervention (3.4%). Of these 64 adverse events or potential adverse events, 27 were judged to be preventable (42%).

The combined error and adverse event rate was one per 35 patient consultations, and the harm rate was one incident per 48 patient consultations. The grading severity of harm is also outlined in table 2. Most cases of harm were graded as category A–E (temporary harm to the patient requiring an intervention). The two events graded as G (permanent patient harm) were both related to procedures in secondary care. The majority of harm was detected in the older age groups despite this patient subgroup representing only a small percentage of all the records that were reviewed (table 5). The number of identified adverse events increased with age: 38 of 64 events (59%) of detected harm occurred in patients older than 60 years, and 23 of the 64 events (36%) occurred in patients older than 75 years.

Medication and related activities such as prescribing directly or indirectly accounted for most adverse events (table 6). Administrative issues included coding errors and errors resulting from correspondence with secondary care. The procedures that resulted in harm took place in the community or secondary care. Delayed diagnosis and referral accounted for two of the incidents classed as “other”.

Feasibility: review time

The mean time for auditors to review a single electronic medical record was just over 3 min, which is an approximation of the actual time spent on a “first-pass” review of each record and excludes the other data-collection tasks. The time required for review increased with the number of consultations, but no record took longer than 20 min to review, as shown in table 3. Feedback from the auditors suggested that they found the trigger tool straightforward to use, and that there was a high level of agreement in identifying incidents of harm and assigning their severity grading. However, there was occasionally some difficulty in deciding whether an event was preventable or not.

DISCUSSION

Main findings

A generic global trigger tool was developed and tested on a random selection of electronic primary-care records. The trigger method facilitated the standardised and focused review of patient records by trained people, which led to the identification of previously undetected errors and harm. Around one in ten patients (9%) was found to have suffered some form of harm. Most harm was temporary and of low-to-moderate severity, but a small number of patients required hospitalisation as a result of their care. A substantial minority of adverse events were judged to be preventable. Most patients who suffered harm were older, and medication issues were implicated in most of these events. Ironically, the most severe cases uncovered in this study originated in secondary care, which may imply that the harm levels suffered by hospital patients may be higher than previously estimated.

The lack of progress in effectively and reliably measuring harm in UK primary care is arguably a key obstacle to developing targeted strategies for improvement to minimise future occurrences. The trigger method shows early promise in potentially measuring error rates and levels of harm in primary care, which could be benchmarked and monitored over time. The information gathered may provide a greater understanding of the threats to patient safety, enabling the primary-care team—and policymakers—to gain greater insights into the frequency and nature of errors and adverse events.

Study limitations

The small sample size limited the generalisability of the findings, and no statistical extrapolation could be made about the possible levels of error and harm in primary care at the practice or national level. The auditors indicated whether a trigger was present in each record but not whether it occurred more than once. They also coded each event of harm only with what they judged the most likely or “strongest” trigger even where more than one trigger may have led to detection of the event. The predictive value, sensitivity and specificity of the instrument and each of the triggers could therefore not be calculated and the trigger tool could not be refined further.

The trigger tool is more time efficient than traditional medical record review. The structured approach and pre-defined time period for each record restricts the review process to those parts of the record that are “triggered”. In one secondary-care trigger tool study,15 auditors took ∼6 min per patient record, which is comparable to our findings. However, many primary-care records selected for our review would have had few or no consultations. If these records were excluded, the mean review time would increase to >5 min. Additional data-collection tasks are also required, which results in a considerable time investment for the practice.

Any clinical member of the primary-care team can be trained to apply the trigger tool. But the method is dependent on the quality and availability of medical records and on the reliability of the reviewers. We did not examine inter-rater reliability, but informal feedback suggested that there were very low levels of disagreement between reviewers requiring resolution. All reviewers were team members of the participating practices, which may introduce the potential for bias, subjectivity and possible collusion.

Few adverse events were in categories A–D despite the expectation of high error incidence. It is possible that these events are minor and/or transient and therefore become undetectable using a trigger tool. It may be reasonable to use only categories E–I in future to target patient harm rather than just error.

Policy context and implications

Trigger methods are strongly promoted by the IHI, which is advising the Scottish Patient Safety Programme as a technical partner. Although trigger methods may appear to have great potential, a number of feasibility obstacles still require to be considered. As shown in this study, the process is comparatively time and labour intensive and will clearly come at an opportunity cost to other practice activities.

Additional challenges need also to be considered in transferring the trigger approach from secondary care to primary care. For example, if the results generated are to be reliable, trustworthy and able to inform trend analysis, statistical guidance will be required to determine the minimum number of records to be sampled, how often and over what time period.

Knowledge and understanding of the epidemiology of error and harm in primary care is severely limited compared with that in secondary care. The trigger method may have greater potential as a research tool rather than as a sophisticated approach to the audit of patient records. If we accept that it may arguably be the most reliable method of highlighting harm rates in primary care, the application of the trigger tool in a large, generalisable study may provide an estimate of error and harm in UK primary care. However, specific interventions will then be needed to ensure that measurements actually translate into patient safety improvement. Practices may require extra resources and support to deal with the harm that the trigger tool detects.

Further research

There is some evidence to suggest that the trigger method may be a more sophisticated and reliable approach to error and harm detection than other surveillance systems such as voluntary incident reporting, significant event analysis and unfocused record reviews.7 11 In comparing the study findings with related published research in primary care, it would seem that different types of error and harm may be uncovered by the trigger method as well as a far greater number. There is a need to test assumptions by comparing the trigger tool against the “gold standard” of conventional case review and also against other quality-improvement methods. It also needs to be refined further and its overall utility subjected to psychometric evaluation to ensure that it is reliable, feasible and has educational impact. In this way, evidence can be provided on its potential purpose and value (or not) as a quality-improvement method for practices.

CONCLUSIONS

The global trigger tool developed may have the potential to detect and measure previously undetected harm in primary care more reliably than other quality-improvement methods such as incident reporting and significant event analysis. However, its general application would come at an opportunity cost for other improvement activities. Research is required to explore the feasibility and acceptability of trigger methods among the primary-care team and to further test the potential of this approach for identifying and measuring previously undetected error and harm in a larger, generalisable study.