TABLE 2

Commonly used methods of assessment

MethodDomainType of useLimitationsStrengths
Written exercises
Multiple-choice questions in either single-best-answer or extended matching formatKnowledge, ability to solve problemsSummative assessments within courses or clerkships; national in-service, licensing, and certification examinationsDifficult to write, especially in certain content areas; can result in cueing; can seem artificial and removed from real situationsCan assess many content areas in relatively little time, have high reliability, can be graded by computer
Key-feature and script-concordance questionsClinical reasoning, problem-solving ability, ability to apply knowledgeNational licensing and certification examinationsNot yet proven to transfer to real-life situations that require clinical reasoningAssess clinical problem-solving ability, avoid cueing, can be graded by computer
Short-answer questionsAbility to interpret diagnostic tests, problem-solving ability, clinical reasoning skillsSummative and formative assessments in courses and clerkshipsReliability dependent on training of gradersAvoid cueing, assess interpretation and problem-solving ability
Structured essaysSynthesis of information, interpretation of medical literaturePreclinical courses, limited use in clerkshipsTime-consuming to grade, must work to establish interrater reliability, long testing time required to encompass a variety of domainsAvoid cueing, use higher order cognitive processes
Assessments by supervising clinicians
Global ratings with comments at end of rotationClinical skills, communication, teamwork, presentation skills, organisation, work habitsGlobal summative and sometimes formative assessments in clinical rotationsOften based on second-hand reports and case presentations rather than on direct observation, subjectiveUse of multiple independent raters can overcome some variability due to subjectivity
Structured direct observation with checklists for ratings (e.g. mini-clinical-evaluation exercise or video review)Communication skills, clinical skillsLimited use in clerkships and residencies, a few board-certification examinationsSelective rather than habitual behaviors observed, relatively time-consumingFeedback provided by credible experts
Oral examinationsKnowledge, clinical reasoningLimited use in clerkships and comprehensive medical school assessments, some board-certification examinationsSubjective, sex and race bias has been reported, time-consuming, require training of examiners, summative assessments need two or more examinersFeedback provided by credible experts
Clinical simulations
Standardised patients and objective structured clinical examinationsSome clinical skills, interpersonal behaviour, communication skillsFormative and summative assessments in courses, clerkships. Medical schools, national licensure examinations, board certification in CanadaTiming and setting may seem artificial, require suspension of disbelief, checklists may penalise examinees who use shortcuts, expensiveTailored to educational goals; reliable, consistent case presentation and ratings; can be observed by faculty or standardised patients; realistic
Incognito standardised patientsActual practice habitsPrimarily used in research; some courses, clerkships, and residencies use for formative feedbackRequires prior consent, logistically challenging, expensiveVery realistic, most accurate way of assessing clinician's behavior
High-technology simulationsProcedural skills, teamwork, simulated clinical dilemmasFormative and some summative assessmentTiming and setting may seem artificial, require suspension of disbelief, checklists may penalise examinees who use shortcuts, expensiveTailored to educational goals, can be observed by faculty, often realistic and credible
Multisource (“360-degree”) assessments
Peer assessmentsProfessional demeanour, work habits, interpersonal behaviour, teamworkFormative feedback in courses and comprehensive medical school assessments, formative assessment for board recertificationConfidentiality, anonymity, and trainee buy-in essentialRatings encompass habitual behaviours, credible source, correlates with future academic and clinical performance
Patient assessmentsAbility to gain patients’ trust; patient satisfaction, communication skillsFormative and summative, board recertification, use by insurers to determine bonusesProvide global impressions rather than analysis of specific behaviours, ratings generally high with little variabilityCredible source of assessment
Self-assessmentsKnowledge, skills, attitudes, beliefs, behavioursFormativeDo not accurately describe actual behaviour unless training and feedback providedFoster reflection and development of learning plans
PortfoliosAll aspects of competence, especially appropriate for practice-based learning and improvement and systems-based practiceFormative and summative uses across curriculum and with-in clerkships and residency programmes, used by some U.K. medical schools and specialty boardsLearner selects best case material, time-consuming to prepare and reviewDisplay projects for review, foster reflection and development of learning plans