Article Text

Download PDFPDF

Postgraduate students’ perceptions of what makes for effective assessment feedback: a case study of a clinical masters course
Free
  1. Sofia Hadjieconomou1,
  2. Michal Tombs2
  1. 1Prince of Wales Hospital, Bridgend Local Health Board, Port Talbot, UK
  2. 2Centre for Medical Education, Cardiff University, Cardiff, UK
  1. Correspondence to Michal Tombs Neuadd Meirionnydd, Heath Park, C4ME, School of Medicine, Cardiff University, Cardiff CF14 4Y, UK; TombsM2{at}cardiff.ac.uk

Abstract

The purpose of this study was to examine postgraduate students’ perceptions of assessment feedback. Using the Critical Incident Technique, students enrolled on a taught clinical course were asked for their perceptions of effective and ineffective examples of assessment feedback. The data were analysed using thematic analysis and nine themes emerged that capture perceptions associated with feedback content and feedback process. Students perceived effective feedback if it was specific and clear, using positive tone of language. They expressed a preference for feedback that is delivered in a standardised format, reflecting the grades given, individualised, and when the marking criteria is explicit and enables dialogue with the marker. Students perceived feedback to be ineffective when it focused on grammatical errors rather than content, when it was provided by anonymous graders and if it was too personal. Timeliness of feedback was also important to participants. Practical implications and suggestions for future research are highlighted in this paper.

  • Medical education & training
  • education & training (see medical education & training)
View Full Text

Statistics from Altmetric.com

BACKGROUND

Researchers have long been occupied with examining students’ perceptions of assessment feedback.1–4 This is because an understanding of what makes for effective and ineffective feedback can provide insight into how to maximise its benefits.5 A review of the literature on the topic revealed that the majority of research to date have been conducted with undergraduate students,6–13 and while some included postgraduate participants,6 8–11 the focus was on non-taught postgraduate students.14 Given the additional time and financial investment (in comparison to undergraduate students), postgraduates’ learning tends to be more self-driven, with specific purpose.15 Also, the structure of taught postgraduate courses often includes smaller numbers of students, making the feedback process different from that of undergraduate courses.

Murdoch-Eaton and Sargeant16 argued that there may be some maturational differences among junior and senior students moving from student passivity to activity in relation to their feedback. In their study, they found that junior students preferred positive written feedback, whereas senior students valued verbal feedback that indicated ways to improve in the future. Additionally, senior students appreciated feedback from all sources and valued peer feedback in contrast to junior students who questioned the credibility of feedback coming from sources other than an experienced tutor. Ferguson,8 on the other hand, did not find support for such differences, showing that regardless of level of seniority, students preferred clear and personalised feedback that was communicated in a positive tone and was given as quickly as possible. Students were willing to wait to receive such feedback. The present study therefore focused on postgraduate taught students’ perceptions, the aim of which was to understand potential solutions for addressing students’ dissatisfaction with assessment feedback.

METHOD

Design

A cross-sectional design employing semi-structured interviews was used in line with the aims of the study and its exploratory nature. The Critical Incident Technique (CIT) was utilised as a framework for data collection as part of the semi-structured interviews.17 The study received ethical approval from Cardiff University Ethics Committee.

Participants

A purposive sampling method was adapted whereby all 18 students who were enrolled on the full-time postgraduate taught course were invited to take part in the study (17 females and 1 male). The intention was to continue interviewing until we had reached saturation, however, the number of individuals volunteering to participate dictated when recruitment into the study ceased. Ten students were interviewed, five participants face-to-face and five were online via Skype. The use of Skype reduced the limitation of geographical boundaries, especially as the majority were international students.18 By the last two interviews, significant repetition of concepts was occurring, suggesting ample sampling for a case study. One of the participants dropped out and their data were removed, resulting in a final sample of nine participants (8 females and 1 male). Although generalisability of findings to other courses may be questioned due to gender bias, this may not be an issue for the local context where anecdotal evidence suggests that the ratio of females to males is typically high.

Materials and procedure

The written CIT record forms (can be made available on request from the corresponding author) used during the interviews were developed by the research team.17 19 Students were asked to: describe what happened, why it was effective/ineffective feedback and explain the outcome. Participants were encouraged to provide as many examples as possible, but two examples of incidents were required as a minimum from each participant, one describing a feedback event perceived as effective and one perceived as ineffective. Two pilot interviews provided opportunity for practice, reflection and development,20 in light of which a more open-ended questioning technique was employed, with some re-phrasing and probing in subsequent interviews.21 The semi-structured interview schedule consisted of three sections. The first part aimed at establishing rapport between interviewer and participant, the second part focused on completion of the CIT form, and finally, the completed record forms were discussed to analyse and clarify participants’ experiences.

RESULTS AND DISCUSSION

Thematic analysis revealed two higher-order themes. The first theme emerged from data related to feedback content and reflects five subthemes. The second theme derived from participants’ responses that reflect four sub-themes of perceptions associated with the process and strategies used by tutors to provide feedback.

Content

Clarity and specificity

Similarly to previous research,1 22 23 participants appreciated feedback indicating faulty practices and information that assisted them to correct these in future. Specific comments within the feedback facilitated their learning, helped improve grades, and were useful for other assignments. P10 said ‘…because I would say…a study showed that this treatment is more effective than that treatment and then the examiner inserted a feedback note saying okay..which study? And how many patients did that involve? And how was it conducted? And how were the results compared to that study? So these comments triggered me to think. Oh, this is what I should have done…’. The same participant noted ‘I found it quite useful and I went back to those comments again before writing my dissertation literature review as well’. Clear and specific feedback also improved students confidence, as noted by P3: ‘The assignment was reviewed by the marker and mistakes were tagged throughout the documentthis feedback made me more confident about my writing ability, and helped make me be more confident about how to do my dissertation’. In comparison, basic and generic feedback was described as ineffective. P2 commented ‘I got the feedback, it was very basic, it was literally like one line. Good point here, bad point here… I didn’t really know what they were expecting from me’. Ambiguous feedback led to negative emotions, as reflected in P1’s comment ‘I got even more confused and it gave me anxiety about my performance’.

Grammar

Feedback that focused on grammatical and spelling errors rather than content was perceived as ineffective. P2 noted ‘I learnt nothing…it was just about English and grammar stuff’. P9 commented ‘…after reading that particular feedback I was like, they’re more concerned with my language, with my sentence formation, with my use of different terminology that I used them wrong not in a medical way so it wasn’t good for me because I thought…they would concentrate on the content of the information’. This confirms previous findings, suggesting that the subject of their assignments are of higher importance to students than the ability to formulate a grammatically correct essay.8 14 If done constructively, however, feedback related to grammar can lead to learning and improvement. For example, P1 said ‘…the feedback that I liked from my assignments is that they would comment on the essays and they would say…this particular grammar should be used or you could have used a different word. So, basically…it’s not necessarily a positive feedback but its constructive, so it means I could improve that’.

Tone and use of language

Comments made by participants suggest that when tutors used positive language and tone, it resulted in the acceptance of negative comments more easily. P3 said ‘None of the things said were mean or derogatory and were said in a way that you could not misconstrue as something negative’. Another participant commented that ‘The style or the language of the feedback was always professional, it was written in a positive way rather than a judgemental or a negative way’ (P10). An encouraging tone was perceived to have a positive impact on confidence. P1 commented ‘He was encouraging me, so ok this is the thesis you want…you can aim high and probably if its really good, you can have it published. So that gave me a lot of confidence, like, oh I have that potential, so I’ll actually definitely work on it’. Feedback that was perceived to be given with a negative tone had a negative emotional impact on students. P5 commented ‘…after the marks came out I received a very harsh feedback that says I know nothing about this certain disease. I didn’t find the feedback professional enough, I would love to know where exactly I did wrong. To improve my work and do extra effort to write a better one. And it is really harsh and inappropriate that it hurt my feelings’. The findings thus highlight the importance of providing positive and professional tone of feedback.14 24 25

Feedback reflecting grades

This study confirms the importance of making sure that students understand how they achieved a specific grade and why marks were lost.26 27 Dissatisfaction was identified when the grade and feedback comments were perceived as conflicting. P2 noted ‘it was one written assignment where I had this issue …I went to one of the tutors and I asked her…I did fairly well, I got an 86%. I just want to know where the 14% I lost…. P3 expressed frustration with the conflicting positive comments and what they perceived as a low grade saying: ‘They finished by saying the essay was good and they seemed quite impressed. But the grade was in the mid-sixties’. This was further reinforced by P6 who said ‘I don’t challenge the mark because this will not change but I challenge the feedback’.

Individualised feedback

Comments made by participants indicate the importance of tailoring the feedback to individual students without it being too personal.2 8 22 28 In relation to a practical assignment, P2 commented that ‘…at the first feedback it actually had stuff like I spoke too fast or that I rushed…which is important for me because as a doctor I should know how to deal with things’. This was echoed by P3 who said ‘…that was very personal for me because…for the next practical exam, the first thing that I did was that’. Both participants found that receiving individualised feedback for their performance was helpful not only for that specific incident but for changing their future practice. A potential caveat to providing individualised feedback relates to the perception of comments that are too personal. One participant described such an experience: ‘They should tell me that for example, the knowledge is not enough you should write this more, you should have corrected your spelling, you should have done this and that. Instead, I just got a general opinion about me personally and not my work, that’s how I felt’ (P5).

Process

Anonymous graders

Previous research suggests that anonymous marking can weaken the tutor and student relationship.29 P10 commented: ‘I am not sure who wrote it… maybe the examiner, maybe another examiner who reported the score or it could have been the coordinator, the secretary in the course’. Participants felt that not knowing who provided their feedback sometimes jeopardised the perception of credible comments. For example, P2 said ‘if that tutor hadn’t checked my paper, they didn’t know how the other tutors marked my paper, so they couldn’t say much’. Comments highlighted that for some, not knowing who marked their feedback resulted in them making speculations about the markers. P3 hoped that it wasn’t marked by a particular tutor ‘if the feedback was from [Tutor], I would have been crushed because that would mean she saw many things wrong and was just trying to be polite’. Consequently, this student did not ask any questions: ‘I left it alone because I couldn’t bothered to find out who it was, why they said this and why exactly did they mark’.

Marking criteria and standardised feedback

Comments made by participants provide additional support to previous findings on students’ preference for a clear marking criteria.8 P3 commented on being unaware of something being a marking criterion saying ‘…if something is not part of the marking criteria, if you are not told to do this, then don’t expect me to’. Feedback was perceived as effective when markers alerted and reminded them of the marking criteria, so they were able to use it in future assignments. For example, ‘they said that you should improve your grammar while writing it next time, and …before you wrote it you should have looked at the guidelines in the sense that they provided a certain font style’ (P4). A preference for receiving feedback in a unified manner from tutors was expressed: ‘I felt they should have standardised form or a standardised method of giving feedback to students…the report should come out looking the same for every tutor. So, you know they should have like maybe certain headings to work with or certain areas that they should be focusing on or something like that’ (P2).

Timing

This study adds to an extensive body of literature on the importance of feedback timing.1 2 22 Participants commented that immediate feedback to a practical assignment was particularly useful: ‘After I presented my case, she immediately commented on my performance…that I could have asked some other things in the history. It was effective because she immediately pointed out what I did correctly and also gave suggestions to improve my performance’ (P1). Participants accepted the delay in receiving feedback to written assignments saying: ‘I think that it’s fine…they were quite prompt with their feedback and it was fine. It wasn’t too delayed and it wasn’t like 2 days, it was appropriate’ (P2). If feedback was not provided in time for it to be useful for the following assignment, students perceived it to be ineffective. P7 highlighted that if time was provided for receiving and processing feedback between the two assignments, mistakes would not have been repeated: ‘I got the feedback after submitting both of my papers. The mistakes I made in the first paper were same as second and I didn’t know what mistakes I did. As the feedback was given after the submission’.

Dialogue and discussion of feedback

Findings support previous research on the use of dialogue in the feedback process.30 P2 said that ‘talking in person definitely makes it, it’s just better than writing on a piece of paper’. All participants attending such meetings found them helpful, ‘…I had a meeting with my tutor…he saw all my record of all my results and then he said ok what I did wrong and what should I do…this part of the programme…was also helpful for me’ (P9). P6 provided a similar example, saying ‘…its quite important to have a face to face meeting…written feedback is so impersonal and somebody to know you and adjust the things to you personally and to your style is more helpful feedback’. Students that did seek help in the form of dialogue found that it enabled them to clarify and understand their written feedback‘…after talking to her, I came to know that every time you make the same mistake, the grade goes down, on every line and every mistake. So that was something, which I would not have known by just looking at it’. For this participant, the opportunity to further discuss the written feedback with the tutor resulted in a change in their initial negative emotional response: ‘Initially considering my hard-work and the efforts I had put in, I wasn’t satisfied with my percentage but after the feedback, it made more sense to me…’ (P7).

Summary and conclusions

The purpose of this study was to examine the perceptions of taught postgraduate students, with the overall aim of informing local stakeholders of potential solutions for addressing students’ dissatisfaction with assessment feedback. It is important to recognise that this study is not without some limitations. Due to the length of the course and the timing of interviews, critical incidents could date back approximately 8 months, presenting a limitation to participants’ memory of the incident.31 Moreover, the generalisability of findings to other postgraduate taught courses may be unclear (small sample size, gender bias, restricting the sample to students’ perceptions). Nevertheless, this study addresses a local need and adds to the body of knowledge that can direct educators in their feedback provisions to students on taught—postgraduate clinical courses. By employing practices that adopt a holistic approach and consider students as adult learners, students will be put in control of their learning. Existing behavioural feedback approaches (eg, anonymous graders) could be replaced by constructivist paradigms (increased opportunities for dialogue) to satisfy participants’ preferences.

Main messages

  • Research to date suggests that postgraduate taught students value verbal feedback and perceive it to be a helpful component for their learning and development. However, there are instances where the feedback that is provided is not effective and the present study demonstrates that students perceive feedback as effective when it is:

  • Clear and specific, with a focus on content rather than grammar; when the tone and language used are positive and encouraging development; when the feedback is reflected in the grade provided; and when it is individualised but not too personal.

  • Consistent and standardised among tutors with a clear marking criterion, provided in a timely manner.

  • Transparent so they know who marked their work. This is linked to a preference for dialogue with the tutor to discuss the feedback.

Current research questions

  • Similar studies should be conducted with other postgraduate taught courses to expand the body of knowledge and generalisability of findings.

  • Researchers should aim to collect a larger number of CITs to strengthen the reliability and validity of findings.

  • A triangulated approach that includes the opinions of tutors could also provide invaluable insights and increase the validity of findings.32

REFERENCES

View Abstract

Footnotes

  • Twitter Michal Tombs @MichalTombskatz.

  • Acknowledgements The authors would like to acknowledge the contribution made by Dr Hamish Cox, University of South Wales, as part of the data analysis.

  • Contributors SH—substantially contributed to the research concept and design, acquired the data and analysed and interpreted it, critically reviewed the manuscript for important intellectual content, and has agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. MT—substantially contributed to the research concept and design, drafted the manuscript and approved its final publication, and has agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Data are available upon reasonable request. All data relevant to the study are included in the article or uploaded as supplementary information.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.