Background To comply with an action plan patients need to recall information accurately. Little is known about how well patients recall consultations, particularly telephone consultations increasingly used to triage acute problems.
Purpose of study This was an exploratory study to measure how accurately patients recall the content of face-to-face and telephone consultations and what factors may be associated with accurate recall.
Study design In Scotland in 2008, the advice (diagnoses; management plan(s); and safety-netting arrangements) given in audio recorded face-to-face and telephone consultations was compared with the advice recalled by patients at interview approximately 13 days later. Patients also performed a memory test. Interactions were sought between accurate recall, consultation type, and factors postulated to influence recall.
Results Ten general practitioners (GPs) and 175 patients participated; 144 (82%) patients were interviewed. Patients recalled most important components of telephone and face-to-face consultations equally accurately or with only minor errors. Overall, patients presenting multiple problems (p<0.001), with brain injury (p<0.01) or low memory score (p<0.01) had reduced recall. GPs rarely used strategies to improve recall; however, these were not associated with improved recall.
Conclusions Contrary to previous hospital based research, patients tended to remember important components of both face-to-face and telephone consultations—perhaps reflecting the familiar, less anxiety provoking environment of primary care. The unsuccessful use of strategies to improve recall may reflect selective use in cognitively impaired patients. Clinicians should compensate for situations where recall is poorer such as patients presenting multiple problems or with brain injury. Patients might be advised to restrict the number of problems they present in any one consultation.
- information technology
- health services administration &
- quality in health care
Statistics from Altmetric.com
- information technology
- health services administration &
- quality in health care
In order for patients to make decisions about medical treatments and to carry out an agreed plan of action, they need to recall information accurately. Impaired recall may have serious consequences. For example, errors in medication adherence, particularly among the elderly (who are most at risk of memory problems) are a frequent cause of hospital admission and other adverse events.1
Although a variety of strategies have been shown to influence recall in non-clinical contexts (box 1), there has been relatively little high quality research on the effectiveness of such techniques in clinical settings.11 What research there has been on recall of information from clinical encounters has been predominantly carried out in hospital settings,6 12 in the specific contexts of cancer care, or in obtaining informed consent for surgical procedures, most of which have explored interventions using written or recorded materials.11
Factors associated with increased accuracy of recall
Reducing the volume of information.2
Reducing delay from presentation to recall.3
Ordering information according to priority (the first and last pieces of information in a list are best remembered).4
Checking patients' understanding.5
Repeating or summarising information.6
Rehearsing (asking patients to repeat advice).5
Using specific rather than generalised instructions.7
Avoiding use of technical language.8
Providing written instructions.9
Providing recordings of consultations.10
The vast majority of clinical encounters, however, take place in primary care where consultations address diverse, and often multiple, problems. There have been no high quality studies carried out in this setting,11 and none have occurred in the context of the growing field of telephone consulting where it is more difficult to provide supportive written instructions. Researching this issue is becoming particularly important because telephone consulting has now become the main medium for unscheduled care in many countries and an increasingly important part of in-hours work.13
Very few studies have tested cognitive methods (box 1) of improving patient recall, although there is a body of research from other non-medical disciplines to suggest that such techniques may be effective in facilitating subsequent recall of information.14 15 Telephone consultations are shorter, deal with fewer problems, and typically contain less information,16 and so might be more readily remembered, though hitherto this has not been tested. In this exploratory study we wanted to discover how well patients recalled the content of face-to-face and telephone consultations and what strategies might be associated with accuracy of recall by comparing patients' recollection of the advice they were given with audio recordings of their consultation, with a view to possibly carrying out an intervention study in the future. Measuring the accuracy of recall is problematic and relies on the judgement of an external observer; this issue, however, is complicated by the fact that achieving agreement between observers when judging consultations is notoriously difficult.17 We therefore sought to determine intra-rater agreement in this task.
In 2008 we approached 11 general practitioners (GPs) working in four Lothian practices of differing socioeconomic backgrounds (table 1), whom we knew from previous work regularly undertook telephone consulting. Reading a standardised initiation, receptionists asked sequential adult patients (including those consulting on behalf of a child) who were either attending or telephoning these doctors to allow recording of their face-to-face or telephone consultations. Consent to recording was documented on the general practice appointment system. Those who agreed to recording were told they would be approached subsequently by letter asking them to participate in a study which explored how well people remembered the content of their consultation. They would then have time to consider if they wished to participate.
Patients who were considered incapable of consenting, those who could not provide a telephone number for subsequent interview, and those who did not have an adequate command of English were excluded. We also excluded telephone consultations that resulted in the doctor suggesting a face-to-face consultation for the same problem.
Audio recording, consultation type and self assessed memory score
We recognised that face-to-face consultations were more likely than telephone to present multiple problems,18 so we recorded relatively more face-to-face consultations in order to ensure sufficient single-problem consultations for comparative purposes. As we excluded telephone consultations which ended in a request to attend the surgery, replacement recordings were made for these; we also recorded replacements for technical failures. Each clinician aimed to record 15 sequential face-to-face and 10 telephone consultations using a hand-held digital audio recorder which they could carry with them so that conversations could continue to be recorded even during a physical examination. Patient agreement to recording was highlighted on the computer appointment screen by the receptionist, reducing the need for explanation by the GP. We asked the doctor to classify the consultation type (ie, new/follow-up and clinician/patient initiated), indicate if they knew the patient, if the patient was accompanied, and if the patient was given written information. Following the consultation, patients were posted information about the project and asked to participate. In order to assess the impact of memory problems, such as dementia, which might have been more common in one mode of consultation than the other, participants were asked to complete the Prospective-Retrospective Memory Questionnaire (PRM-Q).19 This instrument explores patients' perception of their memory and their educational level. They were also asked about any history of head injury or stroke.
We aimed to interview patients around 9 days after their consultation. This time period was chosen to allow patients to make a decision on participation and for invitations and replies to be posted. The choice of short time interval was also influenced by our concerns that consultations for the common, acute, short lived conditions that characterise much of general practice might otherwise be forgotten. The researcher (who had not heard the recording) telephoned consenting patients and asked: how many problems they presented to the doctor; what diagnosis/explanation was provided for each of them; what the management plan was; what they were told in terms of prognosis; and any safety-netting arrangements. If they presented more than one problem, they were asked to indicate which was the most important, but were asked the same standardised questions for each of the problems. At the end of the interview participants were invited to take part in a validated test of recall (the Telephone Memory Impairment Screen (MIS-T).20 This consisted of asking patients to recall four standard words (for which they were given prompts if necessary) after a period of 4 min. The researcher filled the 4 min gap with questions about how they found the research process and what they thought about participating in research generally. Participants scored two points for every unprompted recall and one point if prompting was required. Responses were recorded onto a standard data collection form.
Using a standard data extraction sheet, the duration (in seconds) of the consultation and the number of problems were counted and recorded along with specific advice for each problem. The consultations were coded for the presence of techniques hypothesised to enhance recall (box 1), namely: summarising and repetition of advice; rehearsal (ie, asking patients to repeat advice and checking understanding); and practices that might reduce recall (use of technical language and general rather than specific instructions). Any mentioned use of written information was recorded.
The accuracy of patient recall was compared with the audio recorded consultation in each of the domains of diagnosis, treatment plan, prognosis, and safety-netting instructions. In each case, recall was rated as ‘not recalled at all or wrong’, ‘partly correct’, or ‘correct’. If a domain was not covered (eg, diagnosis in follow-up consultations) this was recorded as ‘not applicable’. In addition, specific advice in the domains of diagnosis, treatment plan, prognosis, and safety-netting were extracted from the recording and compared with the written record from the patient interview. Intra- and inter-rater reliability of this coding were measured using κ scores.21 The patient's response was marked as ‘wrong’ if they gave false recollection—for example, where the patient remembered a diagnosis or a piece of safety-netting information which had not been mentioned by the doctor (although we recognised that in some follow-up consultations advice may have been remembered from previous discussions). To test the intra- and inter-rater reliability of the coding system, 40 consultations were coded again 2 weeks later by the same coder and 40 consultations were also independently coded by two other researchers.
We explored differences between the consultations conducted by telephone and face-to-face, aggregated to doctor level (comparing single and multiple problem consultations separately) with regard to the presence of the parameters in tables 2 and 3 and also in accuracy of patient recall. Additionally, we sought interactions between accurate recall and: the presence of the strategies in table 3; the time delay between consultation and interview; score on PRMQ and MIS-T; age; sex; education; history of brain injury; whether new or follow-up consultation; whether the consultation was GP or patient initiated; and familiarity of the patient to the GP. We compared recall of first and second problems in the same consultation. Analyses were undertaken using t test, χ2 tests, Pearson correlation or Wilcoxon signed ranks tests, and ordinal logistic regression as appropriate. Allowing for multiple testing, we decided that only those at p<0.01 were likely to represent real effects. Confidence limits for differences in percentage were calculated by the Wilson method using CIA software.22
Sample size calculations
In the absence of prior relevant data, we conducted an exploratory study, which sought to establish the frequency of recall error generally in consultations and specifically to compare the accuracy of patient recall in telephone and face-to-face consultations. While our planned number of consultations (around 40 each of: multiple face-to-face; single face-to-face; and telephone consultations) was unlikely to provide us with definite conclusions in relation to the influence of cognitive techniques to improve recall, we hoped that this would provide clear pointers to the likely magnitude of any difference and inform sample size calculations for possible future intervention studies.
Practice and GP recruitment
All four practices and 10 out of 11 of the doctors whom we approached agreed to take part (table 1).
Almost all patients agreed to initial recording of the consultation. All but one doctor reached the target of recording 15 eligible face-to-face consultations and 10 telephone consultations. One hundred and seventy-five patients agreed to participate; however, we were unable to interview 31/175 (18%). Overall, we obtained complete data for 144 patients: 94 face-to-face and 50 telephone consultations. Delays in returning forms and arranging interviews at suitable times meant that interviews took place at a mean of 13.0 days (SD 6.1) after the consultation.
Participation rates varied by practice with the highest rate in the least socioeconomically deprived practice and the lowest in the most deprived (57/75 (76%) vs 39/75 (52%); p<0.01). We were refused ethical permission to explore the demography of those who did not participate. However, those who gave consent, but who could not be interviewed, were younger (mean 45 years vs 54 years; p=0.01) than those who participated.
Table 2 shows the characteristics of those patients who were interviewed and their consultations. As in previous research,16 telephone consultations were shorter than face-to-face consultations, involved discussion of fewer problems, and were less likely to be follow-up consultations. Telephone consulters were less likely to be well known to the GP.
Approaches to enhancing recall
Doctors only infrequently used methods that might enhance recall in both telephone and face-to-face consulting. Repetition was the most widely used technique. Table 3 shows a comparison of these features in different parts (diagnosis, advice, prognosis and safety-netting) of the telephone and face-to-face consultations. We recognised that usage of these techniques was not always appropriate. There was only one significant difference between the two modes of consultation—repetition of advice being more common in face-to-face consultations.
Accuracy of recall
Table 4 shows the assessment of the accuracy of the patients' recall on a three-point scale (wrong/partially correct/correct); partially correct largely indicated minor omissions or errors. Telephone and face-to-case consultations showed little difference, and agreement was generally good. This remained so even when single problem consultations were compared. Of all the parameters measured only one showed a trend towards a difference in accuracy of recall between telephone and face-to-face. More patients consulting by telephone accurately recalled whether or not they had been given safety-netting instructions (37/45 (82%) vs 58/85 (68%), difference 14%, 95% CI −2% to 27%). Patients were not prompted in any way although this may have improved recall. For example, one patient was marked ‘partially correct’ because, in addition to her analgesia and advice on mobilisation, she did not mention that she had been referred to physiotherapy for her back pain, but may well have recalled this had she been prompted.
Intra- and inter-rater reliability judgements of patient accuracy of recall
Most rater differences were in distinguishing between partially and wholly correct recall. Reliability was fair to moderate for all parameters (tables 5 and 6). However, even seemingly clear-cut measurements such as the number of problems presented could be challenging. For example, a patient suffering from insomnia and anxiety might be recorded as presenting one problem by one rater and two by another.
Factors associated with accuracy of recall
To determine which factors were associated with accuracy of recall, correlations were tested on all consultations between the measures of agreement in table 4 and features of the consultation including those in table 3 as well as other recorded factors (GP or patient initiated and new or follow-up consultation, GP familiarity with patient, time delay to interview, MIS-T, PRM-Q, age, sex, education, and history of brain injury such as stroke or head injury). This gave a total of 290 tests, of which six were significant at p<0.05, five at p<0.01, and three at p<0.001. Accurate recall was less likely for patients who presented multiple problems (p<0.001), and with worse scores on the MIS-T or history of brain injury (both p<0.01) However, contrary to what was expected, the use of checking understanding of advice and repeating prognosis was associated with poorer recall (both p<0.01). All of these explanatory variables remained independently predictive in ordinal logistic regression. To investigate the possibility that doctors were reserving these techniques for patients whom they anticipated would have memory problems, we sought correlations between patient demography and MIS-T and the use of these techniques; however, we found no significant association, but numbers were small and it is possible with a larger sample such an association could be found. Other demographic factors, consultation features, or time between consultation and interview, bore little relation to how well the patients recalled their consultations with no evidence of trends.
Recall of multiple problems
We found that the greater the number of problems presented in a consultation the less likely patients were to recall the number of problems accurately (p<0.001). Second problems were remembered equally accurately as first problems. There were insufficient numbers of consultations with more than two problems to allow meaningful analysis.
This investigation has found that patients generally had good memory over periods of approximately 13 days of the salient content of their consultations with GPs, irrespective of whether these were conducted face-to-face or by telephone. While many made errors on their recall of advice, most of these errors or omissions were relatively minor. This is in contrast to widely quoted research previously carried out in hospitals and experimental non-clinical settings.6 12 23 Anxiety is known to reduce recall24 and it may be that patients are less stressed in the familiar setting of general practice compared with, for example, a hospital oncology clinic or before surgery. It is particularly interesting that telephone consultations seemed to be as well recalled as face-to-face consultations even when only single problem consultations were compared.
Interestingly, doctors infrequently used cognitive techniques to try and improve recall and, even when they did, these attempts were associated with either no discernable effect or a reduced recall. However, from subsequent informal discussions with participating doctors, we speculate that this latter observation may be due to selective use of these cognitive techniques in more complex clinical situations and with patients anticipated to have problems with their memory (although our data did not confirm an association between their use and features such as old age or history of brain injury). A formal controlled trial of the application of such techniques would be required to determine if they were effective in clinical practice. As expected, patients who showed signs of memory impairment on the MIS-T, and those with a history of brain injury, recalled consultations less well. Given these findings and our overall finding of generally good recall, future research might be best targeted at groups more likely to have cognitive impairment.
Volume of information is known to affect recall2 and we found that patients were less able to recall accurately the content of multi-problem consultations. This is consistent with findings from our previous research, which suggests that the time spent on individual problems in the consultation is greatly reduced when multiple problems are presented.16 This suggests that patients presenting with long lists of problems should possibly be advised to re-consult rather than attempt to address all their problems in one consultation.
Audio recordings of consultations and written information in the context of cancer care have been shown to be effective in improving recall11 and it may be appropriate for some patients (possibly those with brain injury or known cognitive impairment) in primary care to be given recordings or written information. Research exploring the utility and effectiveness of this approach in primary care should be considered.
Strengths and limitations of this work
Our use of sequential consultations increased the likelihood of a representative sample. The practices represented a broad range of socioeconomic backgrounds. However, all GPs who took part used telephone consulting regularly, were from training practices, and may therefore not be typical of GPs generally.
Study limitations also need to be considered and these include our exclusion of patients considered to be unable to consent, some of whom will have had important memory problems. Despite efforts to keep information leaflets and consent forms as simple as possible, it is possible that those who were less literate (a feature known to be associated with poorer recall25) may have found it more challenging to participate. The fact that there was a significant difference between the most and least affluent practices in terms of participation may support this. However, advice from our local ethics committee was that opt-out consent, even after initial telephone consent, was not acceptable. Insistence on written consent in low risk studies such as this—specifically targeted and aiming to benefit those who may have reading or cognitive problems—is, we believe, a major issue which needs to be addressed. Also of note is that the inter-rater agreement was fair to modest as has been found in other studies attempting to rate the content of GP consultations16 17 26; the findings from this exploratory study should therefore be interpreted with caution as should apparently significant associations when multiple parameters are being considered. Finally, in order to reduce the burden of paper work and partly because of the delay there would have been between their consultation and completion, we decided not to ask patients to complete anxiety or depression questionnaires. However, the impact of such conditions on recall is already well established.24
Patient recall of consultations in primary care is generally better than the findings from research in other contexts would suggest; of note is that telephone consultations appear to be as well recalled as face-to-face consultations. Patients and doctors should be aware that consultations involving multiple problems are less likely to be well recalled and that patients with a history of brain injuries such as stroke and early onset dementia may need additional reinforcement. Given the generally good recall of important content of consultations, future interventional research should focus on patients who are likely to be cognitively impaired.
Generally patients recall the content of primary care consultations well with only minor errors.
Telephone and face-to-face consultations seem to be equally well remembered.
The more problems a patient presents in a consultation the less likely the content of the consultation will be recalled accurately.
Brain injury (such as stroke) is associated with impaired recollection of the content of consultations.
Cognitive techniques to improve patient recall are infrequently used, and were not associated with better recall, although this may be because of selective use in patients expected to have poor recall.
Current research questions
Do cognitive techniques thought to improve recall help people with mild/moderate cognitive impairment to remember the important content of their consultation? How do these techniques compare with written instructions or recordings?
How acceptable would it be to people to restrict the number of problems they present in a consultation? What strategies can doctors use to enable this?
We are very grateful to the patients, the general practitioners and the practice staff who took part and to the Scottish Primary Care Research Network for their support.
Funding Chief Scientist Office, Scottish Executive Health Department, St Andrew's House, Regent Road, Edinburgh, EH1 3DG Other Funders: Chief Scientist Office, Scottish Executive Health Department.
Competing interests None.
Ethics approval This study was conducted with the approval of the NHS Lothian REC 07/S1103/32.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.