Clinical assessment of a new anaesthetic drug administration system: a prospective, controlled, longitudinal incident monitoring study*
Presented in part at the Annual Meeting of the American Society of Anesthesiologists, Chicago, USA, October 2006.
Summary
A safety-orientated system of delivering parenteral anaesthetic drugs was assessed in a prospective incident monitoring study at two hospitals. Anaesthetists completed an incident form for every anaesthetic, indicating if an incident occurred. Case mix data were collected and the number of drug administrations made during procedures estimated. From February 1998 at Hospital A and from June 1999 at Hospital B, until November 2003, 74 478 anaesthetics were included, for which 59 273 incident forms were returned (a 79.6% response rate). Fewer parenteral drug errors occurred with the new system than with conventional methods (58 errors in an estimated 183 852 drug administrations (0.032%, 95% CI 0.024–0.041%) vs 268 in 550 105 (0.049%, 95% CI 0.043–0.055%) respectively, p = 0.002), a relative reduction of 35% (difference 0.017%, 95% CI 0.006–0.028%). No major adverse outcomes from these errors were reported with the new system while 11 (0.002%) were reported with conventional methods (p = 0.055). We conclude that targeted system re-design can reduce medical error.
In the year 2000, the Quality of Healthcare in America Project, initiated by the US Institute of Medicine [1], stated as one of its goals the reduction of error in medicine by 50% within 5 years. A decade later, little has been achieved in demonstrably reducing medical error. Public and professional concern about error in medicine remains widespread, and drug administration error has been highlighted as a leading cause of iatrogenic harm in major national reports from the UK and the US [1–3]. The problem of drug administration error in anaesthesia is of particular concern because of the large number of potent agents administered in relatively rapid succession during the short period of a general anaesthetic. In a large-scale prospective study, we have previously shown that drug administration error affects 0.75% of anaesthetics, and a similar rate (0.68%) has been found during anaesthesia at a university hospital in the US [4, 5]. Some of these errors harm patients and the human and financial costs of error in healthcare are thought to be substantial [1, 2].
For more than 10 years our group has been involved in the development of a system that targets this important source of iatrogenic harm in medicine. The result is a new system that reorganises conventional methods of anaesthesia according to the modern safety principles advocated by the Institute of Medicine and others in the human factors field [6, 7]. The new system was designed to reduce error and facilitate safe practice. It is compliant with the international colour-code standard for anaesthetic labels and consistent with the US Food and Drug Administration’s directive that drug containers should incorporate barcodes [8–10]. We have conducted a large, controlled, multi-centre, incident monitoring study to evaluate the hypothesis that use of the new system would be associated with a reduction in reported rates of parenteral drug administration error.
Methods
With regional ethics committee approval we collected prospective, facilitated, anonymous incident reports at two tertiary teaching hospitals (Hospitals A and B) in New Zealand. All anaesthetics at both hospitals during the study period were included. A new system for administering anaesthesia, based on human factors principles and primarily designed to increase the safety of parenteral drug administration, was progressively introduced to the operating rooms of Hospital A from August 1998, and is described in detail elsewhere [6, 7]. Principal features of the new system include: labels with the class and generic name of each drug in large, clear lettering and incorporating a barcode; use of the international colour-code standard for anaesthetic drugs on labels and other aspects of the new system [9, 10]; alerts for known allergies and expired drugs; custom software and a barcode scanner to allow a drug-identity cross-check before each administration, by redisplay of the drug name and its international colour code on a computer screen and an auditory announcement of the drug name; reorganisation of the workspace using novel syringe trays and a standardised, colour-coded drug trolley layout; prefilled syringes for calcium chloride, ephredrine, fentanyl, lidocaine, magnesium sulphate, metaraminol, midazolam, neostigmine, and pancuronium, labelled with the colour-coded, highly legible labels; a complete, automated anaesthetic record; and operational rules designed to decrease human error and facilitate safe practice. The two principal operating rules are: to scan each drug before administration and to use the auditory and visual prompts to check its identity; and to retain used ampoules and syringes in designated zones on the new trays as a physical ‘record’ of what has been given. The new system also includes an additional labelling system for the accurate preparation of infusion drugs [11].
Conventional methods of the presentation and preparation of drugs for parenteral administration in anaesthesia used in this study were those normally used by the participating anaesthetists; overall, these have changed little in recent decades [12, 13], except that, in 1997, user-applied coloured syringe labels, consistent with the international colour-code standard, were introduced into conventional anaesthetic methods in New Zealand [14].
Incident forms were placed in all patient notes and anaesthetists were asked to return a form for every anaesthetic, indicating which method of anaesthesia provision had been used (new or conventional), and whether or not a drug error or near miss had occurred, with details of the incident if the response was affirmative. The incident forms contained definitions of terms, tick boxes, and space for written comments. This facilitated incident monitoring technique is described in detail elsewhere [4]. Education on the empirical and theoretical reasons for the new system, and training in its use, were provided at Hospital A, as far as practicable, to all staff involved in the study at its outset, and to new staff as they became involved. In addition, numerous presentations were given at both participating hospitals with the aim of motivating anaesthetists both to participate in the study and to report high quality incident data.
Reliable denominators of the total number of patients included in the study at each hospital were drawn from computerised databases. We then corrected for case mix and case complexity by calculating rates of error per drug administration in the following way. As we knew the proportion of procedures conducted with the new system would increase during its introduction, case mix data were sampled at Hospital A by counting every procedure during the study period. At Hospital B, there was no reason to expect case mix to change over time and there were greater logistical difficulties in obtaining older records, so case mix data were sampled by counting all cases in one complete year (2002) (Table 1). Using computer-generated numbers we randomly sampled ten records from the year 2002 for each hospital for each category of case and used these to calculate the mean number of parenteral drug administrations per case for each category (Table 2). Drugs administered by infusion were counted as one administration at initiation and one administration for each of any subsequent bolus administrations. The mean numbers of parenteral drug administrations per procedure and the casemix data were then used to estimate the total number of parenteral drug administrations made in each study group, and hence the rates of error per administration for new and conventional methods.
Hospital A – new system | |
Coronary artery bypass graft | 4591 (42.4%) |
Dental surgery | 327 (3.0%) |
Otorhinolaryngology | 4335 (40.1%) |
Thoracic surgery | 1562 (14.4%) |
General surgery | 1 (0.01%) |
Total | 10 816 (100%) |
Hospital A – conventional system | |
Main operating rooms | |
Cardiology | 2518 (11.4%) |
Coronary artery bypass graft | 2994 (13.6%) |
Dental surgery | 359 (1.6%) |
Otorhinolaryngology | 3195 (14.5%) |
Respiratory | 176 (0.8%) |
Thoracic surgery | 1690 (7.7%) |
Vascular surgery | 148 (0.7%) |
Vascular radiology | 8 (0.04%) |
Day surgery unit | |
Cardiology | 53 (0.2%) |
Dental surgery | 7174 (32.5%) |
Dermatology | 5 (0.02%) |
General surgery | 1 224 (5.6%) |
Gynaecology | 238 (1.1%) |
Otorhinolaryngology | 1733 (7.9%) |
Urology | 535 (2.4%) |
Total | 22 050 (100%) |
Hospital B | |
Acute pain management | 8 (0.1%) |
Line insertion | 104 (0.7%) |
Cardiac surgery | 544 (3.4%) |
Cardiology | 168 (1.1%) |
Dental surgery | 684 (4.3%) |
Gastroenterology | 46 (0.3%) |
General medicine | 17 (0.1%) |
General surgery | 2043 (12.8%) |
Gynaecology | 1678 (10.5%) |
Haematology | 7 (0.04%) |
Mental health (ECT%) | 180 (1.1%) |
Neurology | 7 (0.04%) |
Neurosurgery | 506 (3.2%) |
Obstetrics | 1956 (12.2%) |
Oncology – adult | 95 (0.6%) |
Oncology – paediatric | 113 (0.7%) |
Ophthalmology | 1269 (7.9%) |
Orthopaedic surgery | 2737 (17.1%) |
Otorhinolaryngology | 1282 (8.0%) |
Paediatric surgery – neonatal | 106 (0.7%) |
Paediatric surgery | 581 (3.6%) |
Radiology | 202 (1.3%) |
Renal surgery | 106 (0.7%) |
Thoracic surgery | 169 (1.1%) |
Urology | 835 (5.2%) |
Vascular surgery | 528 (3.3%) |
Total | 15 971 (100%) |
- ECT, electroconvulsive therapy.
n | Parenteral administrations per procedure | |
---|---|---|
Hospital A | ||
Main operating rooms | ||
Cardiology | 10 | 4.3 (1–7) |
Coronary artery bypass graft | 10 | 23.5 (16–39) |
Dental surgery | 10 | 9.9 (0–17) |
Otorhinolaryngology | 10 | 13.1 (2–39) |
Respiratory | 10 | 6.0 (3–10) |
Thoracic surgery | 10 | 10.2 (4–20) |
Vascular surgery | 10 | 10.3 (7–16) |
Vascular radiology | 8 | 5.9 (1–10) |
Day surgery unit | ||
Cardiology | 4 | 2.8 (1–5) |
Dental surgery | 10 | 3.8 (0–8) |
Dermatology | 3 | 7.7 (5–12) |
General surgery | 10 | 5.7 (3–8) |
Gynaecology | 10 | 3.8 (2–9) |
Otorhinolaryngology | 10 | 3.9 (2–7) |
Urology | 10 | 4.3 (3–7) |
Hospital B | ||
Acute pain management | 4 | 7.5 (0–19) |
Line insertion | 5 | 2.4 (0–8) ) |
Cardiac surgery | 10 | 21 (10–35) |
Cardiology | 10 | 4.3 (1–11) |
Dental surgery | 10 | 4.8 (1–13) |
Gastroenterology | 10 | 4.6 (3–9) |
General medicine | 6 | 7.5 (1–16) |
General surgery | 10 | 13.9 (6–21) |
Gynaecology | 10 | 9.9 (6–18) |
Haematology | 6 | 5.0 (0–11) |
Mental health (ECT) | 10 | 2.8 (2–3) |
Neurology | 6 | 9.8 (0–15) |
Neurosurgery | 10 | 12.2 (5–28) |
Obstetrics | 10 | 4.8 (2–7) |
Oncology – adult | 10 | 6.1 (0–11) |
Oncology – paediatric | 10 | 1.6 (1–3) |
Ophthalmology | 10 | 7.2 (0–13) |
Orthopaedic surgery | 10 | 7.9 (2–18) |
Otorhinolaryngology | 10 | 6.4 (0–14) |
Paediatric surgery – neonatal | 10 | 6.9 (2–14) |
Paediatric surgery | 10 | 5.1 (1–16) |
Radiology | 10 | 3.8 (0–9) |
Renal surgery | 10 | 13.4 (5–32) |
Thoracic surgery | 10 | 10.4 (7–16) |
Urology | 10 | 10.0 (2–22) |
Vascular surgery | 10 | 9.3 (3–19) |
- ECT, electroconvulsive therapy.
- In eight uncommon procedures types, < 10 records could be found. In two of these (dermatology and vascular radiology), random sampling was performed from all identified cases during the entire study period because no records could be found during 2002.
The Poisson approximation was used to estimate rates and their 95% CI and to compare these. All statistical tests performed were two-tailed. Sub-group analyses were performed after the finding of a significant primary effect and without correction for multiple comparisons. To show a 33% reduction in error rate per anaesthetic from 0.75% with a two-tailed α = 0.05 and 80% power would require slightly over 10 000 records with the new system, assuming that the ratio of conventional to new system anaesthetics was five to one [4].
Results
Incident reports were collected from February 1998 at Hospital A and from June 1999 at Hospital B, until November 2003 (including previously reported data up to August 1999 [4]). The number of anaesthetics conducted with the new system increased from 161 in the first twelve-month period of its use, to approximately half the number of anaesthetics carried out at Hospital A from the mid-point of the study. A total of 74 478 anaesthetics were included in the study, for which 59 273 incident forms were returned, giving a 79.6% response rate overall, with a high response rate maintained throughout (Tables 3 and 4). These represented 10 816 anaesthetics conducted with the new system (all in Hospital A) and 63 662 anaesthetics conducted with conventional methods (22 050 in Hospital A and 41 612 in Hospital B).
Start date of six-month period | Anaesthetics conducted with new system | Anaesthetics conducted with conventional methods | Total number of completed incident forms returned | Response rate of forms returned |
---|---|---|---|---|
February 1998 | 0 | 2359 | 1707 | 72% |
August 1998 | 161 | 2266 | 1723 | 71% |
February 1999 | 191 | 2500 | 2417 | 90% |
August 1999 | 146 | 2449 | 2387 | 92% |
February 2000 | 416 | 2279 | 2330 | 86% |
August 2000 | 1274 | 1306 | 2290 | 89% |
February 2001 | 1456 | 1681 | 2860 | 91% |
August 2001 | 1404 | 1530 | 2774 | 95% |
February 2002 | 1622 | 1646 | 2844 | 87% |
August 2002 | 1482 | 1394 | 2424 | 84% |
February 2003 | 1638 | 1557 | 2698 | 84% |
August 2003* | 1026 | 1083 | 1639 | 78% |
Total | 10 816 | 22 050 | 28 093 | 85% |
- *Final period encompasses 4 months only to end of study.
Anaestheticsadministered | Forms completed | Drug administrations | Errors reported | Errors rates per drug administration | |
---|---|---|---|---|---|
New system | 10 816 | 8 666 | 183 852 | 58 | 0.032% (0.024–0.041%)* |
Conventional methods | |||||
Hospital A | 22 050 | 19 427 | 190 853 | 88 | 0.046% (0.037–0.057%)† |
Hospital B | 41 612 | 31 180 | 359 252 | 180 | 0.050% (0.043–0.058%)† |
Conventional total | 63 662 | 50 607 | 550 105 | 268 | 0.049% (0.043–0.055%)* |
- *The error rate per drug administration was lower with the new system than with conventional methods, p = 0.002.
- †Error rates for conventional methods were not different between participating hospitals, p > 0.1.
The mean number of parenteral drug administrations per procedure varied from 1.6 to 23.5 (Table 2). The number of drug administrations given over the study period was estimated at 550 105 with conventional methods and 183 852 with the new system (Table 4). The mean (range) number of administrations per anaesthetic was 17.0 (0–39) with the new system and 8.6 (0–39) with conventional methods (the former reflecting a more complex case mix; Tables 1 and 2). Combining these results produced an overall mean of 9.9 (0–39) drug administrations per procedure.
Fifty-eight drug administration errors were reported with the new system vs 268 with conventional methods (no incident form indicated more than one error per anaesthetic). The rates of drug error per parenteral administration were 0.032% with the new system vs 0.049% for conventional methods, a relative reduction of 35% (difference 0.017%, 95% CI 0.006–0.028%), p = 0.002. There was no significant difference in error rates per administration between Hospitals A and B for conventional methods (Table 4). Descriptions of these errors are shown in Tables 5 and 6. By error type, there were significantly fewer dose and omission errors reported per administration with the use of the new system than with conventional methods (Table 7). Errors in the ‘other’ category were greater with the new system – a result attributable to the administration of seven pre-filled syringes that had passed their expiration dates, a type of error not reported with conventional methods. The international colour-code for user applied labels in anaesthesia, emphasised in the new system, groups drugs by colour into broad pharmacological classes [9, 10]. Of substitution errors, a significantly lower proportion of wrong drugs belonged to a different colour-code class than the drug intended with the new system than with conventional methods (Table 7).
Error type (type total) | Bolus injections | Total | Infusion pump | Total |
---|---|---|---|---|
Incorrect dose (14) | Cephalosporin, heparin (4), insulin (2), ketamine, metaraminol, nitrogylcerine (2) | 11 | Insulin, nitroglycerin, remifentanil | 3 |
Substitution* (22) | Inter-class substitutions (3): magnesium for ephedrine, metaraminol for phenoxybenzamine, rocuronium for cephazolin | 14 | Inter-class substitutions (2): insulin for nitroglycerine, insulin for propofol | 8 |
Intra-class substitutions (11): adrenaline for ephedrine, cefoxitin for cephazolin, fentanyl for unspecified, magnesium for calcium (3), magnesium for heparin, metaraminol for ephedrine (2), pancuronium for rocuronium, tranexamic acid for protamine | Intra-class substitutions (6): dopamine for nitroglyercine (4), fluids, magnesium | |||
Repetition/insertion (6) | Atropine, cephazolin, heparin, insulin, unspecified | 5 | Vancomycin | 1 |
Omission (4) | Cephazolin (2) | 2 | Noradrenaline, protamine | 2 |
Incorrect route (0) | Nil | Nil | ||
Other (12) | Cyclizine labeled as antibiotic, expired pre-filled drugs given (magnesium, pancuronium (5), ephedrine), drugs administered to patients with (possible) allergies (ephedrine, co-amoxiclav), rocuronium labeled as suxamethonium | 11 | Aprotonin spilt on floor | 1 |
Route totals | 43 | 15 |
- *Substitution errors between (inter-class), and within (intra-class) pharmacological classes, as defined by the international colour-code for anaesthetic drugs, are shown above.
Error type (type total) | Bolus injections | Total | Infusion pump | Total |
---|---|---|---|---|
Incorrect dose (91) | Atracurium (2), atropine (2), amoxicillin, cephazolin, droperidol, droperidol with tramadol, ephedrine (3), esmolol, fentanyl (3), gentamycin, heparin (4), insulin, ketamine, metaraminol (2), metronidazole, midazolam (3), morphine (7), neuromuscular blocking drug (2), neostigmine, propofol (2), remifentanil (2), rocuronium (2), suxamethonium, unspecified (3) | 48 | Bupivacaine, dextrose, dopamine, nitrogylcerine (3), nitroprusside (6), propofol (6), ropivacaine (2), remifentanil (20), vecuronium, vancomycin, unspecified | 43 |
Substitution* (75) | Inter-class substitutions (44): bicarbonate for local anaesthetic, cefamandole for thiopental, clonidine for morphine, clonidine for pethidine, ephredrine for morphine, ephedrine for pethidine, fentanyl for etomidate, flumazenil for midazolam (2), heparinised saline for lidocaine (2), ketamine for dexamethasone, ketamine for remifentanil, lidocaine for fentanyl, lidocaine for midazolam, magnesium for atracurium, metaraminol for atropine, metaraminol for bupivacaine, metaraminol for local anaesthetic, metaraminol for saline flush (2), midazolam for lidocaine, midazolam for rocuronium, mivacurium for lidocaine, morphine for metaraminol, naloxone for ephedrine, neostigmine for rocuronium, pethidine for ephedrine, pethidine for cephazolin, remifentanil for metaraminol, remifentanil for saline, rocuronium for fentanyl, rocuronium for midazolam, rocuronium for morphine, rocuronium for neostigmine (2), rocuronium for saline flush, saline for morphine, suxamethonium for fentanyl, tramadol for dexamethasone, thiopental for cefuroxime, thiopental for metaraminol (2), water for tramadol | 68 | Inter-class substitutions (3): antibiotic for bupivacaine, calcium for nitrogylcerine, propofol for dopamine | 7 |
Intra-class substitutions (24): amoxicillin, atracurium for rocuronium, adrenaline for metaraminol, bupivacaine for lidocaine, etomidate for propofol, fentanyl for morphine (2), flucloxacillin for amoxicillin, metaraminol for ephedrine, mivacurium, mivacurium for atracurium, morphine, morphine for fentanyl (5), morphine for pethidine, neostigmine, nitroglycerine for ephedrine, propofol for thiopental, remifentanil for morphine, suxamethonium for atracurium, tranexamic acid for amiodarone | Intra-class substitutions (4): adrenaline for dopamine,nitroprusside for nitroglycerine, unspecified (2) | |||
Repetition/insertion (24) | Atracurium, adrenaline, atropine (2), cefamandole (2), cefamandole to patient before taking cultures, cephazolin, cephazolin to patient before taking cultures, amoxicillin, dexamethasone, enoxaparin, fentanyl, metoclopramide, nitroglycerine, propofol, rocuronium (2), tenoxicam, vecuronium (2), unspecified (2) | 23 | Dopamine | 1 |
Omission (57) | Alfentanil, amoxicillin (2), antiemetic, atracurium (2), atropine, cefuroxime, cephazolin, dexamethasone, dopamine, fentanyl (2), fluids, glycopyrronium, heparin (2), local anaesthetic, metaraminol, mivacurium, morphine, neuromuscular blocking drug, ondansetron, rocuronium (4), vecuronium (2) | 29 | Dopamine, insulin (2), nitroprusside, propofol (9), propofol with remifentanil (2), remifentanil (12), rocuronium | 28 |
Incorrect route (11) | Bupivicaine – iv instead of epidural (3), lidocaine with adrenaline – iv instead of epidural, prilocaine syringe used for cuff deflation, propofol – subcutaneous instead of iv, ropivacaine, unspecified – iv instead of epidural, unspecified – intramuscular instead of iv | 9 | Bupivacaine – spinal instead of epidural, pethidine – iv instead of epidural | 2 |
Other (10) | Amoxicillin to patient with known allergy to drug (2), diclofenac to patient with ulcer history, expired cephazolin given, metaraminol labeled as adrenaline and given, morphine to patient with known allergy to drug, tenoxicam to patient on ACE inhibitor, unknown drug in unlabeled syringe given, vancomycin given too quickly | 9 | iv set connected to subsequent patient | 1 |
Route total | 186 | 82 |
- *Substitution errors between (inter-class), and within (intra-class) pharmacological classes, as defined by the international colour-code for anaesthetic drugs, are shown above.
Error type | New system | Conventional methods | p value | ||
---|---|---|---|---|---|
Incorrect dose | 14 | 0.008% (0.004–0.013%) | 91 | 0.017% (0.013–0.020%) | 0.006 |
Substitution | 22 | 0.012% (0.007–0.018%) | 75 | 0.014% (0.011–0.017%) | 0.59 |
Inter-class substitutions | 5 | 0.003% (0.001–0.006%) | 47 | 0.009% (0.006–0.011%) | 0.01 |
Repetition/insertion | 6 | 0.003% (0.001–0.007%) | 24 | 0.004% (0.003–0.006%) | 0.523 |
Omission | 4 | 0.002% (0.001–0.006%) | 57 | 0.010% (0.008–0.013%) | 0.001 |
Incorrect route | 0 | 0% (upper CI: 0.002%) | 11 | 0.002% (0.001–0.004%) | 0.055 |
Other | 12 | 0.007% (0.003–0.011%) | 10 | 0.002% (0.001–0.003%) | 0.001 |
Past expiration date | 7 | 0.004% (0.001–0.008%) | 0 | 0% (upper CI: 0.0007%) | < 0.001 |
Contributing factors for errors fell into 15 categories (Table 8). In 19 reports of errors with the new system, sufficient detail was recorded to allow the event to be identified as a clear violation of the system’s operating rules: either the voice announcement was switched off, or the drug was not scanned before administration, or both (Table 9). One of these 19 reports stated that if the new system had been used correctly, the error could have been prevented.
New system | Conventional method | |
---|---|---|
Failure to check | 32 | 88 |
Inattention | 13 | 51 |
Haste or pressure to proceed | 9 | 40 |
Distraction | 6 | 63 |
Communication problem | 6 | 25 |
Inexperience or inadequate knowledge | 4 | 13 |
Unfamiliar workplace or equipment | 1 | 19 |
Drug label problem | 2 | 21 |
Similar ampoules | 2 | 2 |
Fatigue | 1 | 17 |
Relief anaesthetist or change of staff | 1 | 13 |
Immediate intervention situation | 1 | 5 |
Equipment problem | 0 | 16 |
Fault of technique | 0 | 6 |
Inadequate assistance | 0 | 4 |
Total | 78 | 383 |
Type of error (group total) | Scanned before administration? | Voice on? | |
---|---|---|---|
Substitution errors (9) | |||
Cefoxitin given for cephazolin | 1 | No | |
Fentanyl given for unidentified agent | 1 | No | No |
Magnesium given for calcium | 3 | No | No |
Magnesium given for ephedrine | 1 | No | |
Magnesium given for heparin | 1 | No | |
Metaraminol given for phenoxybenzamine | 1 | No | No |
Tranexamic acid given for protamine | 1 | No | |
Labelling errors (2) | |||
Cyclazine labelled as antibiotic | 1 | No | |
Rocuronium labelled as suxamethonium | 1 | No | |
Expired pre-filled drugs given (7) | |||
Ephedrine | 1 | Yes | No |
Magnesium | 1 | Yes | No |
Pancuronium | 1 | No | No |
Pancuronium | 2 | No | |
Pancuronium | 1 | No | |
Pancuronium | 1 | No | Yes |
Repetition error (1) | |||
Second dose of cephazolin given when first anaesthetist absent, no empty ampoule retained and no record made | 1 | No | No |
There were no significant differences between the new system and conventional methods in rates (95% CI) of errors per drug administration that resulted in major physiological changes (0.0033% (0.0012–0.0071%) vs (0.0042% (0.0027–0.0063%)), prolonged physiological changes (0.0060% (0.0030–0.011%) vs 0.010% (0.0077–0.013)), or minor morbidities (0.0011% (0.0001–0.0039%) vs 0.0005% (0.0001–0.0016%)) (all p > 0.1) No major adverse outcomes occurred with the use of the new system vs 11 with conventional methods (0% (upper 95% CI: 0.002%) vs 0.002% (0.0010–0.0036%), respectively, p = 0.055, Table 10).
Error event | Major physiological change? | Prolonged effect (> 5 min)? | Final outcome | |
---|---|---|---|---|
Overdose of neuromuscular blocking drug | 3 | Yes | Prolonged patient stay in PACU | |
Substitution error with neostigmine | 1 | Yes | Prolonged patient stay in PACU | |
Neuromuscular blocking drug before induction | 1 | Yes | Awareness | |
Propofol infusion turned off | 1 | Awareness | ||
Patient given amoxicillin for which they had a known allergy | 1 | Yes | Anaphylactoid reaction | |
Incorrect dose of dextrose by infusion | 1 | Yes | Yes | Unplanned stay in ICU |
Unspecified insertion error | 1 | Yes | Major morbidity | |
Dopamine omission | 1 | Yes | Yes | Major morbidity |
Repetition error with cephazolin | 1 | Yes | Major morbidity |
- PACU, post-anaesthetic care unit.
Discussion
We have shown a reduction by approximately one third in the reported rate of drug administration error, and a non-significant reduction (p = 0.055) in the harm attributable to drug administration error, associated with the introduction of the new system. These findings are an important step towards the threshold improvement in healthcare safety called for over recent years by many throughout the world [1, 2, 15, 16].
This study has several limitations. Our data were collected over 6 years ago, but there have been no changes in practice in New Zealand or internationally that would invalidate our findings or reduce their relevance. The study deals with self-reported rather than objectively identified errors, it is not a randomised controlled trial (RCT), and it was not blinded. We have reported significance levels without correcting for the testing of multiple secondary outcomes. However, it would not be possible for any study of the new system to be blinded, and the cost of carrying out an RCT of the necessary size would be substantial. Furthermore, compliance with the new systems’ safety features was voluntary and (on the basis of anecdotal evidence) variable. The strengths of this study lie in its size and the facilitated nature of the incident reporting; we think that obtaining negative as well as positive responses enhances the likelihood that errors will in fact be reported, and also allows a response rate to be calculated (in this case, almost 80% overall, which is very high) [17]. In addition, few previous studies have been able to show clear benefits from specific safety interventions in healthcare. The fact that there was no difference in the error rate seen with conventional methods between the two hospitals adds weight to our finding of a reduction with the system – it also suggests that the underlying rate of self-reported errors using our facilitated incident reporting approach may be relatively consistent. These base rates are of comparable magnitude to those reported in other studies using facilitated incident reporting [4, 5, 18], but ours is the only study in which case mix has been taken into account. In previous work we have simply reported drug errors per anaesthetic, but obviously more errors are likely with a complex case involving many administrations (in some cases as many as 100) than with a simple case involving perhaps only two or three [4]. There is no particular reason for the number of anaesthetics to be used as a denominator; in this study we have used the estimated number of drug administrations instead, as this seemed more logical.
Even facilitated reporting is likely to result in under-reporting of errors, if only because a number go undetected by anaesthetists. Normally one would assume that the extent of under-reporting would be similar in each group, but in the present study the multiple checking techniques of the new system may make undetected events less likely to occur than with conventional methods. This can be seen from the identification of nine errors by the barcode check after the drugs had been administered, in violation of the principle of scanning first (Table 9). Given that these errors actually were identified when the drugs were finally scanned, it seems reasonable to assume that compliance with the system’s requirements for scanning before administration might have averted all or most of them. It was disappointing, therefore, that there were 19 incidents associated with clearly documented violations of the requirement for audio and visual cross-checks before drug administration. Anecdotally, these violations are not uncommon; some participants in the study appear to be highly conscientious in their use of the system, and others less so. Our results suggest that better compliance with the new system’s operating rules would be likely to reduce error further. Improved compliance could be achieved by progressive change in the safety culture of participating departments, but improvements in the design of the system may also be helpful. Seven of the procedural violations committed with the new system were of the administration of prefilled syringes that had passed their expiration date (Table 9). A licensed pharmaceutical manufacturer prepared all prefilled syringes for the new system under quality assured conditions [6]. However, testing of the prefilled drugs for sterility and stability continued during the first half of the study to extend the safe shelf-lives of the syringes (typically from 1–2 weeks early in the study to 3–6 months), so the expiration dates associated with the first four of these errors were very conservative. This problem was addressed in the latter part of the study by the combination of longer shelf-lives and better stock management. Frustratingly, all these cases could have been detected by the new system had the drugs been scanned before administration, again emphasizing the importance of compliance.
Colour-coding is thought to promote safety in many complex, high-technology fields [19–21]. Although labels coloured according to the international colour-code standard [14] were in use in both conventional and new-system groups, colour cues were used more extensively in the new system: colour-coding was incorporated into the computer display and drug-trolley drawer compartments, in addition to the prefilled syringe and user-applied labels, and these labels were approximately four times larger than those in use in the conventional group. With the new system, syringes must be labelled if the barcode checking is to be used, whereas there is no equivalent pressure actually to label all syringes with conventional methods. Colour-coding used in this way is most likely to reduce substitution errors between colour classes (rather than dosing or omissions errors, for example). In the context of drug administration, substituting a drug from the same class is likely to be less dangerous than substituting one from a different class. This is consistent with the concept that it may be more effective to focus on managing errors (and their consequences) rather than attempting the more difficult objective of eliminating them completely. We were, therefore, disappointed that the rate of substitution errors overall was not reduced, but interested to see that the use of the new system was associated with fewer potentially more dangerous errors in which the substituted drug was from a different colour class. This result provides some support for the value of colour-coding in the improvement of patient safety, and contributes to the ongoing debate on this topic [19, 22].
Preventing error in medicine requires more than simply telling clinicians not to make mistakes [23]. Despite the fact that we have known for many years that exhortation is inadequate to improve patient safety it remains the predominant paradigm throughout healthcare, and appears to be deeply ingrained in the mindsets of clinicians [12]. By contrast, the new system takes a systems approach, by viewing errors and failures as evidence of faulty work systems that need to be re-designed, rather than due to weaknesses of character. Ultimately, the aim of re-designing work systems is to remove sources of potential error or failure from the environment, permanently and completely, thereby making the workplace error-proof. In reality this is difficult, and we have adopted the well-established method of alternating iterative improvement with assessment, in a cycle of continuous quality improvement [24]. Evidence of effectiveness is commonly requested by funders of any new initiative, and there are increased costs in the use of the new system, which have been discussed elsewhere [7]. However, given evidence of effectiveness it is important to include the savings side of the cost equation when assessing the liabilities facing an organisation [12]. Engagement by clinicians in this process of quality improvement seems a reasonable expectation on the grounds of patient safety alone, and even more so once hospitals have agreed to invest in a safety initiative of this type.
The new system is based on empirical data as well as theoretical considerations [6, 16, 19, 25–27]. We have previously evaluated it in a simulator and in a small randomised clinical trial [7, 28]. A further evaluation in a larger randomised clinical trial is currently nearing completion. Collectively, all the data so far point to the value of the new system and the present study adds considerably to this body of evidence. This study demonstrates that targeted system re-design can be effective in reducing error, and ultimately iatrogenic harm in medicine, but suggests that a greater commitment to safety on the part of individual practitioners is also important.
Acknowledgements
We thank the staff of the Department of Anaesthesia and Pain Management, Wellington Hospital for assistance with data retrieval, and Diana Grieve, Department of Anaesthesiology, University of Auckland for critical review of the manuscript. This project was funded by: the Health Research Council of New Zealand; the Lottery Grants Board of New Zealand; the Australian and New Zealand College of Anaesthetists; and Safer Sleep LLC, Auckland, New Zealand. CSW contributed to this work partly during the tenure of a 3-year Fellowship award from the Health Research Council of New Zealand. The above funding bodies had no direct role in the conduct of this study or the preparation of this manuscript. This study was granted ethics approval number 97/032 by Regional Ethics Committee X, Auckland, New Zealand. Incident reports were voluntary and anonymous, and as such individual participants were not required to give written consent.
Competing interests
AFM and CSW own shares in Safer Sleep LLC, Auckland, New Zealand, a company that aims to improve safety in medicine and which manufactures the new system mentioned in this paper. AFM founded and is a Director of Safer Sleep LLC. All remaining authors have no competing interests to declare.