9 - Assessing the Cognitive Domain through MCQs: Critical to Quality Assurance in Higher Education
Corresponding Author(s) : Zeenatkhanu Kanji
Journal of Higher Education in Africa,
Vol. 15 No. 1 (2017): Journal of Higher Education in Africa: Special Issue on Trends in Higher Education and Quality Assurance in East Africa
Abstract
While establishing assessment methods as a critical element of quality assurance, one must not lose sight of the intent to measure both expected learning outcomes and intended programme objectives. The achievement of quality assessment is rooted in ensuring that the content and approaches to delivery produce graduates and programmes which are high quality. One of the most common strategies to assessing higher order cognitive domain is via Multiple Choice Questions (MCQs). In this paper we will reflect on a quality assurance effort to ensure that this strategy was appropriate (doing the right thing), necessary (for the right reasons) and sufficient (achieving the desired results) in meeting the requirements. An exemplar of a continuing professional development approach to the constructing, measuring and evaluation of MCQs is provided, which emphasised the imperative of both faculty (peer) and organisational commitment to the process and achieving the desired product of competent and successful graduates.
Keywords
Download Citation
Endnote/Zotero/Mendeley (RIS)BibTeX
- Bandaranayake, R., 2008, ‘Setting and maintaining standards in multiple choice examinations: AMEE guide No.37’, Medical Teacher, No. 30, pp. 836–845. Available online at https://doi:10.1080/01421590802402247.
- Bourke, M. P., and Ihrke, B. A., 2012, ‘The evaluation process: an overview’, in D.M. Billings, and J. A. Halstead, Teaching in nursing: a guide for faculty, 5th ed.,pp. 422–440, St. Louis, MO: Elsevier Saunders.
- Considine, J., Botti, M. and Thomas, S., 2005, ‘Design, format, validity, and reliability of multiple choice questions for use in nursing research and education’, Collegian, No. 12, pp. 19–24. Available online at https://doi:10.1016/S1322-7696(08)60478-3.
- Downing, S., 2005, ‘The effects of violating standard item writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education’, Advances in Health Sciences Education, No. 10,pp. 133–143. Available online at https://doi.org/10.1007/s10459-004-4019-5.
- Er, H. M., Ramamurthy, S., and Pook, P. C. K., 2014, ‘Can learning outcomes in cognitive domain be assessed effectively using multiple choice questions? A study in an undergraduate pharmacy curriculum’. International E-Journal of Science Medical and Education (IeJSME), Vol. 8, No. 3, pp. 9–18. Available online at https://doi: 10.1007/s10459004-4019-5.
- Haladyna, T., Downing, S., and Rodriquez, M., 2002, ‘A review of multiple-choice item writing guidelines for classroom assessment’, Applied Measurement in Education, Vol. 15, No. 3, pp. 309-334. Available online at https://doi: 10.1207/S15324818AME1503_5.
- Inter-University Council for East Africa (IUCEA), 2010, A Road Map to Quality: Handbook for Quality Assurance in Higher Education, Volume 1: Guidelines for Self-assessment at Program Level, The Inter-University Council for East Africa/ DAAD. Available online at
- http://iucea1.org/qawp/wp-content/uploads/2013/12/ Handbook-Volume-11.pdf.
- McDonald, M. E., 2007, The Nurse Educator’s Guide to Assessing Learning Outcomes, Sudbury, MA: Jones & Bartlett.
- Morrison, S., and Walsh Free, K. (2001). ‘Writing multiple choice test items that promote and measure critical thinking’, Journal of Nursing Education, 40(1), 17–24.
- National League for Nursing, 2005, ‘Core competencies of nurse educators’. (http:// www.nln.org/profdev/corecompetencies.pdf ).
- Ramsden, P., 2003, Learning to Teach in Higher Education, Routledge New York: Falmer. Rodriguez, M. C., 2005, ‘Three options are optimal for multiple choice items: a meta- analysis of 80 years of research’, Educational Measures Issues and Practices, No. 24,
- pp. 3–13. Available online at https://doi.org/10.1111/j.1745-3992.2005.00006.x. Sadaf, S., Khan, S., and Ali, S. K., 2012, ‘Tips for developing a valid and reliable bank of multiple choice questions (MCQs)’, Education for Health, Vol. 25, No. 3, pp.
- –197. Available online at https://doi.org/10.4103/1357-6283.109786.
- Talebi, G. A., Ghaffari, R., Eskandarzadeh, E., and Oskouei, O. E., 2013, ‘Item analysis an effective tool for assessing exam quality, designing appropriate exam and determining weakness in teaching’, Research and Development in Medical Education, Vol. 2, No. 2, pp. 69–72. Available online at https://doi:10.5681/ rdme.2013.016.
- Tarrant, M., Knierim, A., Hayes, S.K., and Ware, J., 2006, ‘The frequency of item writing flaws in multiple choice questions used in high stakes nursing assessments’, Nurse Education Today, Vol. 26, No. 8, pp. 662–671. Available online at https://doi:10.1016/j.nedt.2006.07.006.
- Tarrant, M., and Ware, J., 2008, ‘Impact of item-writing flaws in multiple choice questions on student achievement in high-stakes nursing assessments’, Medical Education, No. 42, pp. 198– 206. Available online at https://doi: 10.1111/j.1365- 2923.2007.02957.x.
- Tarrant, M., and Ware, J., 2010, ‘A comparison of the psychometric properties of three – and four-option multiple-choice questions in nursing assessments’, Nurse Education Today, No. 30, pp. 539–543. Available online at https://doi. org/10.1016/j.nedt.2009.11.002.
- Vegada, B., Shukla, A., Khilnani, A., Charan, J., and Desai, C., 2016, ‘Comparison between three option, four option and five option multiple choice question tests for quality parameters: A randomized study’, Indian Journal of Pharmacology, Vol. 48,No. 5, pp. 571–575. Available at https://doi.org/10.4103/0253-7613.190757.
References
Bandaranayake, R., 2008, ‘Setting and maintaining standards in multiple choice examinations: AMEE guide No.37’, Medical Teacher, No. 30, pp. 836–845. Available online at https://doi:10.1080/01421590802402247.
Bourke, M. P., and Ihrke, B. A., 2012, ‘The evaluation process: an overview’, in D.M. Billings, and J. A. Halstead, Teaching in nursing: a guide for faculty, 5th ed.,pp. 422–440, St. Louis, MO: Elsevier Saunders.
Considine, J., Botti, M. and Thomas, S., 2005, ‘Design, format, validity, and reliability of multiple choice questions for use in nursing research and education’, Collegian, No. 12, pp. 19–24. Available online at https://doi:10.1016/S1322-7696(08)60478-3.
Downing, S., 2005, ‘The effects of violating standard item writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education’, Advances in Health Sciences Education, No. 10,pp. 133–143. Available online at https://doi.org/10.1007/s10459-004-4019-5.
Er, H. M., Ramamurthy, S., and Pook, P. C. K., 2014, ‘Can learning outcomes in cognitive domain be assessed effectively using multiple choice questions? A study in an undergraduate pharmacy curriculum’. International E-Journal of Science Medical and Education (IeJSME), Vol. 8, No. 3, pp. 9–18. Available online at https://doi: 10.1007/s10459004-4019-5.
Haladyna, T., Downing, S., and Rodriquez, M., 2002, ‘A review of multiple-choice item writing guidelines for classroom assessment’, Applied Measurement in Education, Vol. 15, No. 3, pp. 309-334. Available online at https://doi: 10.1207/S15324818AME1503_5.
Inter-University Council for East Africa (IUCEA), 2010, A Road Map to Quality: Handbook for Quality Assurance in Higher Education, Volume 1: Guidelines for Self-assessment at Program Level, The Inter-University Council for East Africa/ DAAD. Available online at
http://iucea1.org/qawp/wp-content/uploads/2013/12/ Handbook-Volume-11.pdf.
McDonald, M. E., 2007, The Nurse Educator’s Guide to Assessing Learning Outcomes, Sudbury, MA: Jones & Bartlett.
Morrison, S., and Walsh Free, K. (2001). ‘Writing multiple choice test items that promote and measure critical thinking’, Journal of Nursing Education, 40(1), 17–24.
National League for Nursing, 2005, ‘Core competencies of nurse educators’. (http:// www.nln.org/profdev/corecompetencies.pdf ).
Ramsden, P., 2003, Learning to Teach in Higher Education, Routledge New York: Falmer. Rodriguez, M. C., 2005, ‘Three options are optimal for multiple choice items: a meta- analysis of 80 years of research’, Educational Measures Issues and Practices, No. 24,
pp. 3–13. Available online at https://doi.org/10.1111/j.1745-3992.2005.00006.x. Sadaf, S., Khan, S., and Ali, S. K., 2012, ‘Tips for developing a valid and reliable bank of multiple choice questions (MCQs)’, Education for Health, Vol. 25, No. 3, pp.
–197. Available online at https://doi.org/10.4103/1357-6283.109786.
Talebi, G. A., Ghaffari, R., Eskandarzadeh, E., and Oskouei, O. E., 2013, ‘Item analysis an effective tool for assessing exam quality, designing appropriate exam and determining weakness in teaching’, Research and Development in Medical Education, Vol. 2, No. 2, pp. 69–72. Available online at https://doi:10.5681/ rdme.2013.016.
Tarrant, M., Knierim, A., Hayes, S.K., and Ware, J., 2006, ‘The frequency of item writing flaws in multiple choice questions used in high stakes nursing assessments’, Nurse Education Today, Vol. 26, No. 8, pp. 662–671. Available online at https://doi:10.1016/j.nedt.2006.07.006.
Tarrant, M., and Ware, J., 2008, ‘Impact of item-writing flaws in multiple choice questions on student achievement in high-stakes nursing assessments’, Medical Education, No. 42, pp. 198– 206. Available online at https://doi: 10.1111/j.1365- 2923.2007.02957.x.
Tarrant, M., and Ware, J., 2010, ‘A comparison of the psychometric properties of three – and four-option multiple-choice questions in nursing assessments’, Nurse Education Today, No. 30, pp. 539–543. Available online at https://doi. org/10.1016/j.nedt.2009.11.002.
Vegada, B., Shukla, A., Khilnani, A., Charan, J., and Desai, C., 2016, ‘Comparison between three option, four option and five option multiple choice question tests for quality parameters: A randomized study’, Indian Journal of Pharmacology, Vol. 48,No. 5, pp. 571–575. Available at https://doi.org/10.4103/0253-7613.190757.