Article Text
PDF
Abstract
Background Research evidence is insufficient to change physicians' behaviour. In 1996, Pathman developed a four step model: that physicians need to be aware of, agree with, adopt, and adhere to guidelines.
Objective To review evidence in different settings on the patterns of ‘leakage’ in the utilisation of clinical guidelines using Pathman's awareness-to-adherence model.
Methods A systematic review was conducted in June 2010. Primary studies were included if they reported on rates of awareness and agreement and adoption and/or adherence.
Results 11 primary studies were identified, reporting on 29 recommendations. Descriptive analyses of patterns and causes of leakage were tabulated and graphed. Leakage was progressive across all four steps. Median adherence from all recommendations was 34%, suggesting that potential benefits for patients from health research may be lost. There was considerable variation across different types of guidelines. Recommendations for drug interventions, vaccination and health promotion activities showed high rates of awareness. Leakage was most pronounced between adoption and adherence for drug recommendations and between awareness and agreement for medical management recommendations. Barriers were reported differentially for all steps of the model.
Conclusion Leakage from research publication to guideline utilisation occurs in a wide variety of clinical settings and at all steps of the awareness-to-adherence pathway. This review confirms that clinical guidelines are insufficient to implement research and suggests there may be different factors influencing clinicians at each step of this pathway. Recommendations to improve guideline adherence need to be tailored to each step.
- Practice guideline
- evidence-based practice
- guideline adherence
- protocols & guidelines
- medical education & training
- statistics & research methods
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
Statistics from Altmetric.com
- Practice guideline
- evidence-based practice
- guideline adherence
- protocols & guidelines
- medical education & training
- statistics & research methods
Introduction
For healthcare professionals to avoid actions that are harmful or fail to do actions that are beneficial, they should ideally always make decisions informed by high quality research evidence. However, the transfer of research evidence into practice is often complex and incomplete.1 The availability of research evidence alone is usually insufficient to change physicians' practice.2 While a wealth of literature exists to assist physicians to understand and evaluate research, comparatively little is known about the processes of change.3 Evidence based clinical practice guidelines are one important tool for helping physicians access the best research evidence efficiently. Well developed guidelines can help physicians and patients make better decisions, and reduce the risk of negligent care. Theoretically, they promote consistent and quality care that is cost effective for the healthcare system.4 However, for a variety of reasons, known and unknown, publication of clinical practice guidelines has had a limited effect in changing physician behaviour.2 Cabana et al's work showed that simply knowing about guidelines is rarely sufficient to change physicians' behaviour.2
In 1996, Pathman et al developed a four step model (1. awareness→ 2. agreement→ 3. adoption→ 4. adherence) to look at the utilisation of clinical practice guideline recommendations on paediatric vaccine usage. The underlying conceptual framework suggested that to conform to guidelines, physicians must be aware of them, agree with them, decide to follow (adopt) them for some patients, and then actually follow (adhere to) them at appropriate times for all patients. The authors showed progressive drop off with a proportion dropping off at each step. This tendency has also been described as a ‘pipeline’, in which research evidence ‘leaks’ at various steps and reduces the extent to which research findings are implemented in practice.1
We wanted to know whether the patterns observed by Pathman for paediatric vaccination recommendations are similar in other professions and for different types of clinical recommendation. We wanted to explore the patterns of ‘leakage’ and summarise any identified barriers to guideline implementation. Given the uncertainty about how this model had been applied in practice, we did not begin with any preconceived notions about how patterns might be similar or different from each other. We decided, therefore, to systematically review all studies that had explicitly or implicitly applied the Pathman awareness-to-adherence model to the utilisation of a clinical guideline.
Question addressed
What are the patterns of ‘leakage’ in the utilisation of clinical guidelines in practice?
In particular, can the rates of leakage be quantified and compared between steps, and are there identified causes for this leakage?
Methods
Design
A systematic review of all studies that look at the utilisation of clinical practice guidelines using the awareness-to-adherence model.
Included studies
Studies that look at the utilisation of one or more clinical practice guideline recommendation/s, that measure awareness and agreement and either adoption or adherence (or both)
Design: any primary survey or cross-sectional study
Response rate: not specified as we wished to include internet surveys, and determining the denominator is not always possible
Outcome measures: both objective and self reported
Specialty or area: any area of healthcare (eg, surgery, physiotherapy, public health)
Healthcare objective: any (eg, diagnosis, treatment, prevention, screening).
Excluded studies
Studies that failed to measure awareness AND agreement
Studies that failed to measure at least one measure of practice (ie, adoption or adherence).
Search strategy
Four electronic databases were searched (AMED 1985–2010, Embase 1996–2010, Medline 1996–2010, and PsychInfo 1987–2010), at 30 June 2010, using the text words aware* AND agree* AND (adopt* OR adhere*) combined with a MESH and free-text search to identify clinical practice or health planning guidelines (full strategy available on request). A forward citation review of the original 1996 article ‘The Awareness-to-Adherence Model of the Steps to Clinical Guideline Compliance’3 by Pathman was performed using Web of Science, Scopus and PubMed's related articles.
Selection of papers
Two authors independently reviewed identified citations for eligibility. Disagreements were resolved by consensus.
Assessment of quality
Quality was assessed using a pro forma against the following criteria:
Study design
Pilot or validation of measurement method
Appropriateness of sampling frame and method
Number of participants and completeness of data
Response rates (where measurable)
Representativeness of responders
Method of measurement of outcome (subjective or objective)
Consistency of reporting across abstract, tables and full text.
The awareness-to-adherence pathway considers whether target users agree with the guidelines. It does not consider whether the guidelines were in fact appropriate or evidence based. Therefore we did not assess the quality of the guidelines and the evidence supporting recommendations.
Data extraction
Two authors independently extracted data. Disagreements were resolved by consensus. The following data were extracted: study design, date of guideline, date of study, the quality criteria given above, target condition of guideline and recommendations, target population, sampling frame, rates of awareness, agreement, adoption and adherence, measures of uncertainty if reported (eg, p values, confidence intervals). We also tabulated separately any identified barriers to and recommendations for guideline implementation reported by the authors.
Analysis
We calculated the response rate for each study. From those responding we calculated the proportion who were aware, agreed, adopted, and adhered to each recommendation (‘absolute rates’). We also calculated ‘conditional rates’—that is, the proportion of those who were aware and who agreed, who agreed and adopted, and who adopted and adhered to the recommendation.
We grouped recommendations by different intervention types (eg, drug interventions, medical management, health promotion, screening, vaccination). These categories were not pre-defined as we did not know in advance what guideline topics we would find but were chosen post-hoc, to reflect clinically related and relevant themes.
Although our protocol suggested we would perform a meta-analysis, the heterogeneity of the studies precluded this.
Protocol
A protocol was agreed in advance and can be obtained from the authors. No amendments to protocol were made.
Results
Studies identified
We identified 427 citations. From reading abstracts alone, 410 articles were excluded for not measuring rates of awareness and agreement and one or both of adoption and adherence (figure 1). For 17 studies, the abstracts were unclear and they had to be read in full before they were excluded for the same reason. Eleven studies met the inclusion criteria. The participants, intervention, sample frame and size, measurement method, guideline publication, and timing of each study are summarised in table 1.
Study selection diagram.
Study descriptions
Study characteristics
All studies were surveys, using a self-reported questionnaire. Eight surveys were distributed by mail, two by internet,8 11 and one was given to participants after a personal interview. Only two studies explicitly acknowledged using Pathman's original questionnaire.5 11 Only one survey compared physician reports with other data: namely patients' reports of their influenza vaccination within one calendar year.14
Sampling strategies were sometimes not well described. Most commonly, random samples were obtained from specialist or general practitioners within specific geographical regions or specialty organisations. Sample sizes of eligible participants ranged from 153–2146 with an average of 920 participants. Reported response rates varied from 33–97%.
While all studies reported the date of the publication of the guideline, only nine reported the timing of their study. Three studies surveyed physicians <1 year after guideline publication.5 9 14 Most surveyed practice between 1–2 years after publication.
The quality of studies was moderate to poor, though it was often difficult to distinguish whether studies were poorly conducted or simply poorly reported. A number of studies presented inconsistencies between tabulated and text figures, abstract and content, and absolute and calculated values. To maintain consistency of reporting, absolute response rates for awareness, agreement, adoption, and adherence were recalculated from raw data, where possible. Any inconsistencies within the reporting of studies did not substantively alter our conclusions. Conditional response rates were also calculated to measure the change per step: to capture the percentage of agreement, given awareness; adoption, given awareness and agreement; and adherence, given awareness, agreement and adoption. All conditional rates are included in table 2.
Absolute responder rates and conditional leakage rates for each recommendation
Participants and recommendations
Across the 11 studies there were 29 recommendations that reported at least three of the four steps of the model. Only 12 recommendations reported data at all four steps, 12 did not report adoption, and five did not report adherence. The topic of recommendations included advice regarding: drug interventions (six topics—stable angina,5 chronic heart failure (CHF)8 and hypertension (HT),11 and prescriptions for children with asthma.6), medical management (nine topics—three on hypertension,11 six on anaesthetic practice guidelines in Thailand12), vaccination schedules (two studies of vaccinations given by American paediatricians and family physicians for infants,9 children3 and elderly adults14), screening tests (American gastroenterologists for surveillance at different stages of Barrett's oesophagus7; and Canadian staff physicians and residents for screening for chlamydia.13) and health promotion (two studies reviewed five health promotion recommendations: three for American paediatricians for advising parents on media use10; and two for general practitioners in England for adults with hypertension11).
Comparison of rates at each step
There was considerable variation in the pattern at different steps and for different recommendations (table 2). The leakage between steps is not simply a function of difference in specialties or sampling frames, but varied between recommendations from the same guideline within the same population of responders. This variation prompted a deeper level of comparison of patterns across different purposes for using clinical guidelines. We made visual comparisons across a range of different purposes. From these comparisons some interesting observations emerged.
Different patterns of leakage were apparent when recommendations were grouped according to their purpose of drug prescription, medical management, vaccination, screening, and health promotion activities. For the six drug interventions there are consistently high rates of awareness and agreement across recommendations (figure 2). However, adherence rates vary almost fourfold: from just above 20% to over 80%. Adoption and adherence rates are generally progressively lower, except for the recommendation of statin treatment for hypertension, where surprisingly physicians adopt the guideline slightly more often than they agree with it (the authors note that there are specific financial incentives for complying with this particular recommendation11).
Absolute responder rates for drug recommendations. LDL, low density lipoprotein.
For the nine medical management recommendations, awareness rates are lower and more variable than for drugs. Figure 3 documents six key recommendations; three similar patterns within the same study have been removed for ease of reading. The leakage between aware and agree is also more pronounced. There is less variation in rates of adoption and adherence which seem to reflect agreement. The recommendation for annual review of mild hypertension increases between agreement and adoption and again the authors note that there are specific financial incentives for compliance with this recommendation.11
Absolute responder rates for medical management recommendations.
The six vaccination recommendations demonstrate a different pattern (figure 4); there is high awareness of all recommendations, but agreement rates vary almost twofold. The step to adoption is highly variable. The authors of the study in which adoption was higher than agreement reported that adoption and adherence were more likely when physicians believed that parents know about and requested the vaccine and when physicians belong to specialist societies where peer expectations were likely to be greater.3 All three studies that report adherence show that there is leakage after the adoption stage.
Absolute responder rates for vaccination recommendations.
For the five health promotion recommendations awareness is high (figure 5). Agreement was also relatively high, with the lowest rate being 70%. There is steady leakage across all four steps. The screening recommendations demonstrate most variability across the pathway (figure 6), with awareness of the chlamydia screening guidelines being the lowest across the 29 recommendations. It was noted that reasons for agreement and disagreement with screening guidelines for Barrett's oesophagus changed according to the severity of disease, perceived risk of cancer, and medico-legal liability. This could be in part explained by differences in interpretation of the literature, which does not include definitive evidence comparing surveillance intervals across different grades of dysplasia.7
Absolute responder rates for health promotion recommendations.
Absolute responder rates for screening recommendations.
Comparison of rates across all studies
Given the heterogeneity of the guidelines and their recommendations, it is difficult to combine results into a single summary statistic; therefore, we reviewed the median and IQRs. The median awareness of guidelines was high at 90%. The absolute rates (figure 7) show a steady fall in the median from aware to agree and from adopt to adhere, with a plateau between agree and adopt. The median proportion of respondents who report adhering to guideline recommendations is 36% (IQR 30–56%). In practice, the adherence to recommendations is probably even lower given that most adherence was self-reported. We also examined the median percentage reduction in the conditional rates as a proxy for the rate of leakage. The median rates of leakage—16% agree given that responders were aware, 13% adopt given agreement, and 17% adhere given adoption—are of all the same order, and suggest similar proportional leakage at each step.
Box and whisker plot for all absolute rates.
Identification of barriers for guideline implementation
From the 11 included studies, different rates of awareness, agreement, adoption, and adherence have been reported within the same clinical population and group of clinicians, suggesting that differences depended on individual clinicians' training and experience and the clinical action required.11 For example, specialists are more likely to adopt and adhere than their generalist peers, whereas clinicians working in solo or two physician practices were less likely to adopt guidelines than those in large practices and hospitals.3 6 9 12 13
The credibility of a guideline's authors, its scientific rationale, and the method of dissemination impacted agreement and adherence. Physicians had more confidence in guidelines developed by their own specialty organisation.6 European cardiologists preferred European and national guidelines over American ones.8 Physician agreement was enhanced when information was shared among colleagues and adherence was enhanced when the Centres for Disease Control and Prevention (CDC) provided guideline information.3 Even when global agreement for a guideline was high, practical considerations and the perception of relative advantage for groups of patients5 impacted on adherence. Guidelines were seen as impractical in hospitals where there was limited equipment and staff.12
Methods of dissemination varied and influenced uptake. In Thailand, announcements of guidelines in the annual general meeting of the Royal College of Anaesthesiologists and in the monthly newsletter were considered more effective than formal documents sent to hospital directors, and announcements on websites and in journals.12 In contrast, European cardiologists reported gaining knowledge of guidelines from congresses and medical journals.8 Most family physicians and paediatricians preferred receiving guideline information from their own specialty society.9
Finally, patients could influence clinicians' behaviour; those who believed that parents knew about and requested a vaccine were more likely to adopt the guideline.3 Adoption of influenza vaccination was also higher when patients planned to be vaccinated.14
Discussion
This systematic review—which included 29 guideline recommendations from 11 identified studies—shows an average 15% ‘leakage’ between each step in the Pathman awareness-to-adherence model. The heterogeneity of setting in these studies suggests that the Pathman model can be usefully applied to guidelines addressing very different healthcare objectives (eg, screening, anaesthetic management, drug prescribing), across a range of different clinician groups, and in different countries and healthcare systems. This pattern supports the conceptual model of a ‘pipeline’ from guideline publication to utilisation, with progressive leakage across stages which leads to a failure to get research findings into practice.1 The cumulative leakage is substantial and suggests that guidelines may not be being adhered to about two-thirds of the time—a significant loss in potential health gain to patients.
In most cases awareness precedes agreement, as the model suggests. However, sometimes there is agreement with guideline recommendations even though the responders are not aware of the guidelines. This could occur with poor dissemination of guidelines, combined with either the existence of widespread consensus within the profession about appropriate practice, or limited research evidence supporting the recommendation, or both.7 For some recommendations there is a large drop between awareness and agreement, suggesting that failure to implement may due to active disagreement.
Following on with the pipeline metaphor, agreement usually precedes adoption. However, some recommendations have higher adoption than agreement rates, suggesting that physicians may adopt recommendations with which they do not agree. This pattern was noted by Pathman et al, who reported that 11% of physicians adopted a recommendation to provide hepatitis B vaccination without agreeing with it. They suggested this may be due to peer pressure, malpractice fears, patient demand, community norms, and practice organisation policies.3 Other external factors that may play a part include target setting, policy directives, and financial incentives. This suggests that winning hearts and minds of clinicians through the dissemination of evidence is not the only strategy to increase the adoption of research evidence into practice. Financial incentives and penalties may be levers for change, but may also invoke distrust and professional demoralisation.
Despite the small overall rate of adherence, rates of awareness and agreement are high for many recommendations, especially those for drugs and vaccinations. The higher agreement rates for drug treatments and vaccinations compared to medical management or health promotion may be a function of the strength of the evidence underlying the recommendations. Clearly, moving from awareness through agreement to adopt and ultimately adhere to guidelines is a complex process.
To understand better the causes for leakage along this pipeline of guideline implementation, we summarised factors that were reported as barriers, for each included study. We used the comprehensive systematic review conducted by Cabana et al which identified 293 potential barriers to guideline adherence to conceptually organise these factors.2 We identified those barriers which influenced clinicians' beliefs of outcome expectancy and their perception of the inertia of current practice. External barriers were investigated as to whether they were related to the guidelines themselves, patient preferences, or environmental obstacles. We then reviewed all identified obstacles and attributed their contribution to one of the four steps of the awareness-to-adherence model. Table 3 provides a more generic summary of the identified barriers and recommendations to promote awareness, agreement, adoption, and adherence.
Barriers and recommendations influencing awareness, agreement, adoption, and adherence to clinical practice guidelines
From this analysis, recommendations can be inferred. By ensuring that guidelines are consistently and clearly understood across appropriate clinical groups, clinician awareness and agreement may be improved. Useful strategies could include: liaison across specialties regarding the content and timing of recommendations; promoting local development, pilot testing, implementation and monitoring of guidelines; targeting dissemination to clinicians who have less experience and who are less connected with their peers; and using specialist clinicians to influence the guideline compliance of their peers.
Through better recognition of patient needs and values, and strategies to manage environmental obstacles, clinician adoption and adherence to guidelines may be improved. It is important to consider providing patients with concise information about their condition and its recommended treatment. Clinicians also need short and relevant summaries within guidelines for possible side effects, contraindications and risks, to ensure they and their patients are appropriately informed. Environmental barriers can be identified and reduced with sufficient planning, organisation and piloting. Some practical strategies may include: developing special purpose clinics; using patient reminders or prompts; having accessible and relevant patient education posters; and using publicly visible monitoring tools such as the calculated practice wide immunisation rates. Therefore, it is apparent that to enhance guideline implementation, a better understanding of the leakage and its potential cause is required at each step of the awareness-to-adherence model.
This systematic review is limited by the validity of the included primary studies, which have a number of weaknesses in their design and reporting. All studies except one used self reporting of awareness, agreement, adoption, and adherence. We know that for patients, self reporting tends to overestimate compliance.15 Physicians are likely to be prone to a similar bias and are likely to overestimate their own adherence.6 13 Inconsistencies were noted between physicians' self reports and their performance on specific case studies, and confirms that self-reports overestimate.13 The likely overestimation of adherence rates is compounded when one considers that non-responders to these surveys probably have lower rates than responders.
This systematic review suggests that the awareness-to-adherence model is a useful theoretical model to inform clinical guideline development and evaluation. In the first instance it would be useful to ascertain if the patterns and causes of leakage identified across 11 published studies are meaningful beyond these studies. Simple audit and evaluations of guideline implementation could be made more systematic if this model was applied. For future research, it is important to design quality studies that measure and report all steps of awareness, agreement, adoption, and adherence using self-report and other objective measures. Further, it should not be assumed that barriers and facilitators influence all four steps equally and deeper investigation of change at each step may be enlightening.
Finally, it is valuable to briefly reflect on how this pipeline of leakage may impact on healthcare policy and practice. In the UK, the recent white paper Equity and Excellence: Liberating the NHS,16 proposes a move from targets to measurable evidence based outcomes. Relevant recommendations embedded in evidence based clinical guidelines may be able to identify appropriate outcomes that are meaningful and motivating to both clinicians and their patients. The Quality and Outcomes Framework contract for general practices in the UK provides a strategy for measuring and reporting clinical indicators in general practice. If these indicators are derived from research evidence and included in appropriate guidelines and clinical practice, they may serve to inform patients and enhance clinicians' rates of adoption and adherence. While financial incentives can be effective in increasing the rate of adoption beyond that expected from awareness and agreement alone, they have substantial costs and other downsides. There is a need to validate and evaluate suggested strategies to improve the implementation of guidelines in clinical practice.
Conclusion
Leakage from guideline publication to utilisation occurs among different specialties, across a range of recommendations, in different countries and healthcare systems, and at all steps of the awareness-to-adherence pathway. Leakage generally increases proportionally along the pipeline from awareness, through agreement, adoption, and adherence and we conclude that guideline recommendations may not be adhered to, up to two-thirds of the time. We identified different patterns of leakage across varying purposes of recommendations. Drug interventions showed high rates of awareness and agreement, but had varied and pronounced leakage to adoption and adherence. Recommendations for vaccination and health promotion activities also showed high rates of awareness, but leakage was more progressive between agreement, adoption, and adherence. For medical management recommendations, awareness rates were lower and more variable, and leakage was greatest between awareness and agreement. We investigated barriers for guideline implementation at each step of the awareness-to-adherence pathway and made differential recommendations for clinical practice and future research. This systematic review confirms that the production and dissemination of evidence based clinical guidelines is not sufficient to ensure that research evidence gets into practice. This is a complex process and requires deeper investigation at each step of clinician awareness, agreement, adoption, and adherence to guidelines.
Main messages
Research evidence progressively leaks out at each step of clinician awareness, agreement, adoption, and adherence to guidelines.
It is suggested that only one third of the research evidence informing guidelines is being routinely adhered to.
Rates and patterns of leakage vary across different types of clinical guidelines.
Barriers to guideline implementation vary for each step of the awareness-to-adherence pathway.
Recommendations to improve guideline adherence also need to consider strategies for improving clinicians' awareness, agreement, and adoption of guidelines.
Current research questions and suggested strategies
What causes the leakage of research evidence in implementing clinical guidelines?
conduct a case study, audit of a service's pattern of leakage of research evidence across a specific clinical guideline/s using the four steps of the awareness-to-adherence pathway
How is research utilised in clinical practice?
use questionnaires/focus groups/interviews to evaluate barriers for guideline implementation between the steps of clinician awareness, agreement, adoption, and adherence
How do self-report measures of awareness, agreement, adoption, and adherence compare with objective measures?
compare questionnaire responses with actual prescribing or vaccination rates
Acknowledgments
We thank the NIHR for the funds to make this publication open access.
References
Footnotes
Competing interests All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that (1) SM, AB and PG have support from the Department of Primary HealthCare, University of Oxford, and Bond University for the submitted work; (2) have no relationships with any companies that might have an interest in the submitted work in the previous 3 years; (3) their spouses, partners, or children have no financial relationships that may be relevant to the submitted work; and (4) SM, AB and PG have no non-financial interests that may be relevant to the submitted work.
Ethics approval This study did not require ethical approval. It was funded from the working budget of the Department of Primary HealthCare and there were no additional sponsors. All authors had full access to all of the data, on a shared internal drive, and take responsibility for the integrity of the data and its accuracy.
Provenance and peer review Not commissioned; externally peer reviewed.