Article Text

PDF

Formative student-authored question bank: perceptions, question quality and association with summative performance

Abstract

Purpose of the study There are few studies on the value of authoring questions as a study method, the quality of the questions produced by students and student perceptions of student-authored question banks. Here we evaluate PeerWise, a widely used and free online resource that allows students to author, answer and discuss multiple-choice questions.

Study design We introduced two undergraduate medical student cohorts to PeerWise (n=603). We looked at their patterns of PeerWise usage; identified associations between student engagement and summative exam performance; and used focus groups to assess student perceptions of the value of PeerWise for learning. We undertook item analysis to assess question difficulty and quality.

Results Over two academic years, the two cohorts wrote 4671 questions, answered questions 606 658 times and posted 7735 comments. Question writing frequency correlated most strongly with summative performance (Spearman’s rank: 0.24, p=<0.001). Student focus groups found that: (1) students valued curriculum specificity; and (2) students were concerned about student-authored question quality. Only two questions of the 300 ’most-answered' questions analysed had an unacceptable discriminatory value (point-biserial correlation <0.2).

Conclusions Item analysis suggested acceptable question quality despite student concerns. Quantitative and qualitative methods indicated that PeerWise is a valuable study tool.

  • peerwise
  • SBA
  • single-best answer
  • question bank
  • MCQ
  • formative
  • summative
  • student question writing
  • student authored
  • student contributing pedagogy
  • trolling
  • faculty review
  • gamification
  • student-authored question quality.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Footnotes

  • Contributors All named authors have contributed the following: JLW: contributed substantially to the conception and design of the study, acquisition, analysis and interpretation of the data; wrote the first draft and revised and critically reviewed all subsequent drafts; gives final approval for the submitted version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. BHLH: contributed to the conception and design of the study, acquisition, analysis and interpretation of the data; cowrote the second draft and revised and critically reviewed all subsequent drafts; gives final approval for the submitted version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. PD: contributed to the conception and design of the study, analysis and interpretation of the data; cowrote the third draft and revised and critically reviewed all subsequent drafts; gives final approval for the submitted version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. PS: contributed to the conception and design of the study, analysis and interpretation of the data; critically reviewed all drafts; gives final approval for the submitted version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

  • Competing interests None declared.

  • Ethics approval Cardiff School of Medicine Research Ethics Committee.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.