What 39.3 million learners taught us about the future of assessment

In October 2020, Learnosity delivered 2.15 billion questions to millions of students. Here’s what we learned.

By the end of 2020, most of us just wanted to put the year that kept on taking in our rear view mirrors in a hurry.

But if there’s one thing that 2020 made clear, it was that education – both as a field and an industry – has been pushed to do something new: evolve at pace.

As CEO of a SaaS company that deals in assessment infrastructure, I’ve seen the sea change firsthand.

Here are some things I’ve observed from our data.

The rise of authentic assessment

When Mark and I started Learnosity, one of our key drivers was to enable more authentic assessments. Simply put, we wanted to help a whole industry to go beyond standard multiple choice questions, which are more geared for recall than for deeper, more nuanced evaluation.  

In previous years, the vast majority of questions authored or delivered in Learnosity (and indeed across the assessment industry) were indeed multiple choice. But we’ve had some time to make a difference and have seen clear signs that education is starting to free itself from its codependent relationship with MCQs. 

Pre-COVID we expect a user to do an assessment once or twice a week. Post-COVID, we’re seeing students come back every single day to do hours of work.  Share on X

So what do the numbers tell us? 

Our customers have authored over 67 million questions and stored them with us, so we’re well-placed to analyze trends from the number of questions authored. We can see that just over half of all questions authored were MCQs – 56% to be precise. The rest are more innovative question types, better suited to support more authentic assessment. 

Of course the proof is in the pudding. 

Out of the 2.15 billion questions delivered in October, 947 million of them were rich questions (i.e. not MCQs). Without digging into the details too much, here are some notable numbers relating to questions authored: 

• 223 million were math questions
• 2.3 million were video questions,  which are used for demonstrating presentation skills, scientific experiments, and creative work
• There were 13.1 million file uploads, which are suited to demonstrating offline learning (i.e. paper-based upload via photos or handwritten work)
• 17.3 million were audio questions, often used for demonstrating spoken languages skills
• 142 million were essay questions, indicating a switch toward a more long-form format

COVID’s impact on digital assessment

COVID has disrupted life as we know it, but seeing how people adapt to it is some kind of silver lining.

It’s been astonishing to see how quickly our clients have onboarded new users in response to COVID, with the first peak we observed being around week 12 (mid-March, coinciding with earlier global “lockdowns”). 

Each year we expect to see a nice ramp of new users coming onboard over the ‘back to school’ period (around the end of August). This year was different.

The graph below shows new users every week for the last few years. In 2020, we onboarded more than 3.5 million students every week for the first six weeks of back to school.

In addition, what we’re seeing – at least in formative assessment – is that the usage pattern has changed dramatically. Pre-COVID, we might expect a learner to take an assessment once or twice a week (for example, as part of their homework). Post-COVID, we’re seeing students come back every single day to do hours of work. 

In other words, we’re seeing more students each doing more work.

When COVID hit, a bunch of major assessments were cancelled. The tide went out, and a lot of folks were swimming naked. It’s totally understandable. But in 2021, there’s no excuse. The industry needs to adapt, and quickly. Share on X

When all is said and done, compared to last year, we’ve observed a 2.5x increase in the number of students using our system, and a 3.6x increase in the number of digital assessments they’re taking. 

In times of crisis, innovators lead the way

There’s a quote from Warren Buffet I’ve been using a lot lately.

“Only when the tide goes out do you discover who’s been swimming naked.”

When COVID hit earlier this year, a bunch of major assessments were cancelled. The tide went out, and a lot of folks were swimming naked. It’s totally understandable. But in 2021, there’s no excuse. The industry needs to adapt, and quickly. Cancelling two years in a row is not an option. 

The pivot to online as a necessary fail-safe actually paves the way for real innovation in the sector. We’ve seen massive gains in traffic for clients who’ve really leveraged new technology into their own learning products to better serve learners in core areas like accessibility, engagement, and reliability. 

Their success reminds me of something I read in a Uri Friedman piece for The Atlantic on the role of resilience in geopolitical power.

“International power dynamics will be rooted in resilient power amid the types of mass traumas that look set to dominate this century.” 

The surest way for education to fortify its resilience to such traumas is by investing in critical infrastructure now. Let’s not wait for the tide to go out again.

Get more insights with our Re:Vision annual report. Download now for free to get an comprehensive overview of the learning industry’s evolution in 2020 and beyond. 👇

A version of this article originally appeared on e-assessment.com.