5 questions you need to ask before building an assessment tool

Building an assessment tool from scratch is a mammoth task. To feel confident in your decisions, there are a number of important things to consider first.

There’s a barrier to making a learning product as successful as it can be: people don’t always make decisions based on rationality.

Many of the life choices we make every day are guided by impulses, instincts, or emotions – but we convince ourselves after the fact that our decisions have been logical. That explains why we eat too much junk food when it’s harmful to our health, why we procrastinate when we know there’s stuff we need to do, and why we refuse to ask for help even when we’re badly in need of it.

When building an assessment tool, the ability to objectively recognize the best available solutions for overcoming its specific challenges is critical.

And when it comes to the decisions we make in our professional lives, this irrational behavior doesn’t just disappear. When building an assessment tool, the ability to objectively recognize the best available solutions for overcoming its specific challenges is critical.

The API-ification of education is revolutionizing how learning products are made by offering faster, lower-cost, and easier paths to completion. This shift in the paradigm means that a pivotal decision made in the early stages of development can fundamentally affect the quality of your product: whether to build every component of your tool in-house or to buy solutions from elsewhere.

Amazon CTO Werner Vogels described the generic but labor-intensive problems that add no extra value to a company as “undifferentiated heavy lifting” – and these are exactly the sort of problems APIs are designed to solve.

Amazon CTO Werner Vogels described the generic but labor-intensive problems that add no extra value to a company as “undifferentiated heavy lifting” – and these are exactly the sort of problems APIs are designed to solve. Share on X

When it comes to educational technology, Learnosity’s suite of APIs takes care of the spadework for customers by providing the infrastructure for assessment, authoring, and analytics. Companies can build upon these to make superior learning products of their own.

However, even when an API-based solution is the most logical choice, it may be dismissed out of hand due to stakeholders being affected by Not Invented Here Syndrome: the social phenomenon where solutions conceived outside of your project group are considered somehow inferior to those conceived internally.

If you’re having trouble deciding between the routes of buying solutions or developing them in-house, there is a way to restore rationality to the decision-making process: simply answer these five questions.

1. How much will it cost?

When you’ve invested heavily in putting together a team of highly skilled developers, you don’t want to waste their time and abilities on things like maintenance and bug-smashing when they could be focusing on the finer details that set your product apart.

When you factor in the less obvious costs of building everything in-house, the numbers can rise quickly. How much will hosting cost? What about maintenance? And innovations? Will you have to bring on more developers to help you stick to your timeline? Can you be sure you won’t run over budget? According to research by McKinsey, large IT projects run over budget by an average of 45%.

Once you’ve calculated a realistic build estimate, you may conclude that partnering with a specialist vendor is the safest option as you’ll receive a fixed cost upfront for the functionality you need.

If you’re hoping that your learning product will be the next big thing, then you’ll need to make it robust enough to handle that kind of success.

2. Is it reliable?

Is your infrastructure capable of performing at a greater scale than you’re used to? Are you confident you can deliver hundreds of thousands of assessments concurrently? In the event of a crisis, have you the resources to respond quickly and effectively?

You can’t afford to overlook the performance of your product. Your customers certainly won’t.

3. Is it secure?

Have you got adequate defenses to protect your users’ private information from unauthorized disclosure? Data is a valuable resource, so edtech developers need to consider how data privacy, confidentiality, and security practices affect learners. Privacy laws must be carefully adhered to when designing your safeguards. 

If you decide to adopt any solution from a vendor, be sure to determine whether it enhances learners’ experience while also protecting their personal information. At Learnosity, security is paramount – we use TLS encryption in transit and AES 256 encryption at rest; we do not share our customers’ data; and we store data in an orphaned manner (meaning our customers control the mapping and context for the data).

If you decide to adopt any solution from a vendor, be sure to determine whether it enhances learners’ experience while also protecting their personal information. Share on X

Before deciding whether to build or to buy, be sure you’ve laid to rest any concerns you may have about security. 

4. How long will it take?

When it comes to building a learning product, it inevitably becomes a race against the clock – in fact, McKinsey’s findings show that the average large IT project runs 7% over time.

The longer you have to spend reinventing features that already exist elsewhere, the longer your product is going to take to get to market. With all the moving parts involved, it’s difficult to ensure your project rigidly sticks to a schedule. Can you afford to postpone your product’s launch?

When you need to fast-track the whole process, support from a vendor can be invaluable. (Shameless plug alert: Learnosity’s tools helped one customer deliver an assessment system on a national scale in just seven months.)

5. How difficult will it be?

The process of developing a learning product from the ground up is an arduous one. Developers may spend months testing and refining things as they try to reinvent a feature that already exists. You also have to factor in ongoing maintenance and technical debt. All that rigmarole just isn’t necessary.

You need to know if creating every component of your product in-house is worth the level of effort required or if it’s simply a waste of your resources. As you weigh up cost versus value, consider this: the average large IT project delivers 56% less value than initially predicted.

As you weigh up cost versus value, consider this: the average large IT project delivers 56% less value than initially predicted.

“We knew that interactive assessment was going to be a critical tool in our platform,” says Chris Contini, CEO of OnCourse Systems for Education, about the obstacles to development his company had faced. “As we scoped the project, it was clear that our biggest challenge would be tackling Technology Enhanced Items [TEIs]. We knew there was going to be so much functionality behind even supporting basic multiple choice or true/false question types.”

“When we figured out how many developers it would take and how much time, and then the hidden costs like maintaining code, fixing bugs, and constantly trying to innovate, we knew we weren’t going to be able to do it.” 

Ultimately, OnCourse opted for Learnosity’s ready-made solutions and in the process saved “well over a year’s work for multiple developers”.

Next steps

If the best solution is that a vendor should handle your heavy lifting, then let them, so your developers can focus their efforts where they can provide the most value. Whatever decision you ultimately make, be confident that it’s the right one for your product.

Want to make your assessment tool even better? Discover more about our learning product solutions today.