How to drive digital assessment beyond the MCQ test

Assessment in the Digital Age has been slow to advance much beyond assessment in the Industrial Age – but why?

The phonograph changed music forever. Recitals and live performances were no longer the only way to listen to compositions. Those who could afford it could sit in the comfort of their homes and listen as a needle traced the grooves engraved on a spinning disc.

It was revolutionary at the time. And even today – more than a hundred years later – there’s still plenty of love for records. Yet it’s hard to imagine a world in which music failed to move beyond the original phonograph. That easy access to new music that we’ve come to expect would be impossible. So too would portability, leaving millions of commuters, travelers, and flaneurs without a soundtrack to accompany their journeys.

In short, music would almost certainly never have become embedded in our lives in the way it has without technological progress.

Yet the change in music didn’t happen over night – in fact, it took decades. This is far from unusual. It often takes years before any major discovery makes a discernible impact.

There are numerous precedents showing this to be the case. When electricity was first introduced as a source of power, factory owners actually kept things running pretty much as they had done for a further two decades (though at least conditions for workers were safer, not to mention brighter). Once society had adapted to the presence of electricity in their lives, people began to envision its true potential. Factories were completely remodeled to better leverage the new power, and business was transformed forever.

It’s a similar story with other game-changing discoveries – from optical lenses to the double helix and the Internet. Major breakthroughs are quick to create possibilities, but making them a reality tends to take significantly longer. Is there any reason to expect that things be any different in education?

The multiple choice question – attached to the past

If the multiple choice question, or MCQ, is a barometer for education’s broader tendencies, then the simple answer is “no”.

Devised over a century ago in 1914 by Frederick Kelly, the MCQ’s dominance in the field of testing has far outlived the record player’s dominance in music. The fact that the multiple choice format could go on to become the standard means of evaluating learning for generations is remarkable; that it managed to do so without significantly evolving during that time is sobering.

The MCQ has dominated assessment for 100 years without really evolving. Share on X

Yet the multiple choice question was very much a product of its era. Kelly, who at the time was a young man still working on his doctoral thesis, believed that different teachers tended to offer subjective judgments on a student’s work. This often made proceedings slow, inconsistent, and messy. Kelly believed that if one could somehow simplify the testing process, it would make it much easier to eliminate subjective marking and speed up the entire process.

His idea, however, was a response to a growing need brought on by a national crisis. Schools were seeing unprecedented growth in student numbers while the turmoil of the First World War was causing a serious shortage of teachers. The multiple choice question was used to help streamline schooling during this upheaval. It offered a simple, quick, and inexpensive way of testing a learner’s declarative knowledge. Multiple choice tests were formulaic, objective, and easy to mark.

But that was then. The MCQ was employed as a stop-gap solution to the exceptional needs of a particular point in history. It has since become a problem in itself.

Students have changed, teachers have changed, and society has changed. Yet the multiple choice question has, broadly speaking, remained the same. Even Kelly, the father of the MCQ, would later advocate for change in learning once the War had concluded. He felt that learning needed to take a more integrated, problem-based approach. Regardless, the MCQ is the legacy to education by which he will be remembered.

Familiarity breeds contempt?

As digital technology continues to develop and popularize, the standard MCQ seems ever more out of both time and place. Yet somehow it persists.

“The format for an MCQ is something familiar to both teachers and students,” says Professor Geoffrey Crisp of the University of New South Wales.

Crisp, who has published research on evolving the form and functions of MCQs, tells me that such habituation has largely worked to strengthen the MCQ’s position in learning and assessment. “In this sense, the MCQ has some advantages as the format doesn’t confuse users or get in the way of responding to the question.” (Read the full interview with Professor Crisp here.)

“The main issue with MCQs is that they have traditionally been used for recall rather than analyzing or evaluating. So the trick is to use the simple format of MCQs but have more sophisticated questions.”

That familiarity, which has proven so powerful in cementing the MCQ’s place in education, has also put the creaking format under considerable pressure as mounting commentators come to question its utility in the digital age.

Though still the dominant tool for testing convergent responses (i.e. where students are expected to give the same answer), the multiple choice question fails to reflect the reality that technology makes it possible to create evaluations that can go deeper than ever before.

E-assessments should offer authentic problems that require students to use authentic tools. Share on X

“There have been enough studies done to show that a well-written multiple-choice question actually measures understanding fairly well,” wrote the author and former teacher Terry Heick. “But in the 21st century, change is happening at an incredible pace.

With multiple points of access to multiple types of information, students find themselves in the midst of “an elegant kind of chaos” – and one for which the standard MCQ is ill-equipped to provide order.

“More than anything else”, Heick continues, “when a multiple-choice question is given to a student in hopes of measuring how well he or she understands something, it manufactures the illusion of right and wrong, a binary condition that ignores the endlessly fluid nature of information … Rather than attempt to mimic paper-based assessments, e-assessments should offer authentic problems that require students to manipulate authentic tools in order to respond to a question.”

Paper, predictability, and progress

This still seems some way off.

Many schools have worked hard to integrate digital into their teaching practice but still switch to paper to administer item-response tests. Others have managed to transition their paper-based multiple choice assessments to digital, yet fail to explore its latent potential for anything other than alacrity and convenience.

In assessment, teachers need to realize that traditional formats are out of date. Share on X

Whether offline or online, MCQ tests that follow the tried-and-tested path of a century ago are neglecting the needs – and overlooking the realities – of today’s learners. But what will it take to bring testing in line with technology?

“The major impediment to change in assessment practices is not the need for better software, but the need for teachers to realize that their traditional assessment formats are out of date,” affirms Professor Crisp. “There is an urgent need for a resetting of the conceptual understanding of the purpose of assessment. If we continue to assess in the ways we have done for the last 30 years then we are doing a great disservice to our students and we are not preparing them for the world in which they operate.”

This is true for a number of reasons. The old-fashioned MCQ fails to test a learner’s problem-solving or critical thinking skills. More often than not, multiple choice questions are set up to reward a sound memory more than they are to gauge depth of understanding.

Multiple choice questions are set up to reward memory more than understanding. Share on X

Aside from failing to properly evaluate a student’s true intellectual abilities, poorly written MCQ tests are also formulaic by nature. This not only makes them less engaging for learners; it makes them predictable.

Because of this, the MCQ can also be “gamed” to some degree. Learners may spot common patterns or simply search online for tips on how to beat an MCQ. Others yet may decide to throw caution to the wind and take their chances with the laws of probability. None of these options, it must be said, are of benefit to the learner, the teacher, or to the pursuit of knowledge generally.

Remodeling the MCQ test

It takes some degree of creativity to properly evaluate creativity. Similarly, it takes complex solutions to assess complex critical thinking processes. The problem for educators is that these can be both expensive and time-consuming to produce. Such hurdles offer another explanation on why assessment types have been re-used so widely, and for so long.

“Multiple choice – four or five choices – has become hard-coded into our educational technologies and our educational practices and our educational policies. We could assess differently,” wrote edtech blogger Audrey Watters. “But even new technologies, developed one hundred years after Frederick Kelly’s work, tend to re-inscribe the multiple choice test.”

But this need no longer be the case. Assessments can do more to burrow deeper into the thought processes of learners and truly engage them. The technology to create the likes of essay-type questions; cloze; drag-and-drop, audio questions, visual, and even VR – can add greater interactivity and flexibility to evaluating a learner’s knowledge and higher-order thinking skills.

Such flexibility must be viewed as a crucial part of revitalizing the form, function, and suitability of the MCQ in the digital age.

Flexibility is crucial to revitalizing the form and function of MCQs in the digital age. Share on X

“A contribution vendors could make is to provide scaffolding in their software for the different types of assessment,” says Professor Crisp. “This would prompt teachers to offer a range of assessments, not only task types, but purpose types.”

If the means are made available then it should follow that there will be those who will use them to create richer, more authentic, and more meaningful assessment experiences to students.

And that’s exactly what we’ve seen in the wake of the COVID-19 pandemic. 

The global shift to remote learning has forced educators to devise more immersive digital assessments. In October 2020 alone, Learnosity was used to deliver 2.3 million video questions, 17.3 million audio questions, and 142 million essay questions to learners. 

Now that there are more sophisticated alternatives to standard MCQs readily available, shouldn’t learners have access to more engaging assessments?

The century-old multiple choice question has its benefits, but the time has surely come to look ahead and see beyond it.

Read our full interview with Professor Geoffrey Crisp on digital assessment and the future of multiple choice questions.