39 standards to help build a learning or assessment product

“Standing on the shoulders of giants” is a well-known metaphor for advancing on the foundations of the past. These standards and guidelines provide a framework for quality in edtech.

We are fortunate in the learning and assessment space that there are a lot of smart people who have laid down useful standards and guidelines. These provide a navigational compass for product builders when developing products, supporting interoperable systems, providing technical safeguards for future clients, and generally improving the quality and usability of the work being done.

We thought it would be useful to give edtech builders to a repository of the key standards in the space.

Accessibility standards

The main document to look at for accessibility is the general WCAG standard, but the two IMS/1EdTech documents are worth noting.

	Formal name	URL	Available	Description
APIP	Accessible Portable Item Protocol	Here	Free	APIP documents accessibility for questions/items, providing the information needed to make questions accessible for learners with a variety of disabilities and special needs. It was originally created separately from QTI but is now also included within IMS QTI v3.
AccessforAll / PNP	From 1EdTech (previously IMS)	Here	Free	IMS (1EdTech) initiative on accessibility and personal needs preferences.
WCAG	Web Content Accessibility Guidelines	Here	Free	These general guidelines apply to all web content, not just assessments, but they are the core of any attempt to make assessments effective for people with disabilities.

Accreditation of assessments or testing programs

These standards allow accreditation of certification or other kinds of testing programs, and so are relevant to those building systems to make sure that your customers can meet their needs.

	Formal name	URL	Available	Description
ICE 1100	Standard for Assessment-Based Certificate Programs	Here	$	This standard from the Institute for Credentialing Excellence(ICE) and approved by the US ANSI specifies requirements for assessments for certificate programs. It’s possible to get accredited by ICE against the standard.
ISO 17024	Conformity assessment — General requirements for bodies operating certification of persons	Here	$	This ISO standard (also distributed by ANSI others) defines how a certification organization that certifies people should operate. A certification body can be certified against ISO 17024 to demonstrate credibility and that it is following good practice.
ISO 10667	Assessment service delivery — Procedures and methods to assess people in work and organizational settings	Here and here	$	Defines good practice in using assessments in the workplace. Has two parts—one aimed at service providers and one aimed at companies who use assessments, and you can be certified on either part.
NCCA Standards	NCCA Standards for the Accreditation of Certification Programs	Here	$	The NCCA standards are well respected and developed by the Institute for Credentialing Exercise. They allow accrediting of certification programs.

Building and using technology

Technical standards are useful but there also are a few widely reviewed guideline documents on building systems. The most significant and valuable from the following selection is from ITC and the ATP.

	Formal name	URL	Available	Description
ITC/ATP TBA Guidelines	Guidelines for Technology-based Assessment	Here	Free	This mammoth document was published late in 2022 and provides guidance on good practice in technology assessment from the ATP and ITC.  Anyone building an assessment system should at least skim these guidelines—see my earlier blog for Learnosity on them.
ISO 23988	Information technology — A code of practice for the use of information technology (IT) in the delivery of assessments	Here	$	This standard is quite old (dating from 2007) but provides general good practice in using technology for assessment delivery. I actually came up with the idea for this standard and led the team that produced the British standard that was in due course updated to be an international standard.
Principles of AI use in Testing	(from ATP)	Here	Free	Short general principles on using artificial intelligence in testing.
Testing and data integrity in the administration of statewide student assessment programs	(from NCME)	Here	Free	Some general guidance from NCME about on ensuring testing data integrity. Around 10 years old, so a little dated, but still potentially of value.

Content packaging and metadata standards

In an already dry standards world, metadata and content packaging are particularly specialist, but they can be useful.

	Formal name	URL	Available	Description
Dublin Core LRMI	Learning Resource Metadata Initiative	Here	Free	A collection of classes, properties and concept schemes for markup and description of educational resources.
IEEE Standard for Learning Object Metadata (LOM)		Here	$	A conceptual data schema that defines the structure of a metadata instance for a learning (or assessment) object.
IMS Content Packaging		Here	Free	A very old but still used definition of a data structure for describing or passing content packages (that can include assessments) from one system to another.

Credentials and badging

If you are working in the space of competencies, credentials and badging, these standards are worth looking at to see if you can use rather than reinventing the wheel.

	Formal name	URL	Available	Description
CTDL	Credential Transparency Description Language	Here	Free	This is a schema for describing credentials from a variety of sources from the non-profit Credential Engine organization.
EDCI Standards	European Digital Credential Standards	Here and Here	Free	These hold definitions of credentials used within the European Union, which allow digital versions of credentials and qualifications.
IEEE Recommended Practice for Defining Competencies		Here	$	This 2022 standard from IEEE focuses on recommended practice to define competencies but also how to interoperate them
Open Badges		Here	Free	This specification now maintained by IMS/1EdTech is a way for packaging information about accomplishments (including assessment results) and embedding it into portable image files as digital badges. If you are creating digital badges, you should probably use this standard.

Data exchange (between systems)

Another kind of technical standard is one that allows you to pass data from one system to another. They typically have data models and ways of interchanging data like test results from one system to another.  There are several such standards, here are some of the key ones:

	Formal name	URL	Available	Description
A4L SIF	Assessment 4 Learning Community Systems Interoperability Framework	Here	Free	The SIF specifications are a technical blueprint for allowing diverse educational applications to interact and share data. There are versions for the US, Australia, New Zealand, the UK and for global use.
CEDS	Common Education Data Standards	Here	Free	CEDS is an education data management initiative that includes a common vocabulary and data models.
Ed-Fi Data Standard		Here	Free	The Ed-Fi Data Standard is a US-focused data standard allowing interchange of education data, including assessment data.
IMS QTI	Question and Test Interoperability	Here	Free	IMS QTI (now 1EdTech QTI) is one of the oldest and most widely used assessment specifications. I was part of the team that created QTI v1 back in 2000, and my Learnosity colleague Steve Lay led the team that produced QTI v2. The latest version is v3. Although it has other use cases including results reporting, one of the key QTI use cases is to exchange questions being systems. In theory, a question written in QTI in one system can be exported to another system and used there.
PESC	Postsecondary Electronic Standards Council	Here	Free	PESC has a range of technical standards allowing data interchange in areas such as US Higher Education admissions. Their data standards include college and high school transcripts and test scores.

Security and privacy

Anything edtech has to be secure and must respect learner privacy. Here are some useful standards and guidelines in this space.

	Formal name	URL	Available	Description
ISO 27001	Information security, cybersecurity and privacy protection – Information security management systems	Here	$	This is a general standard, not specifically aimed at assessments but since information security is so important, anyone who is delivering assessments will likely want to comply with it. Many will choose to be certified against it, as Learnosity has done. There are also other useful security standards in the ISO 27000 family.
ITC Security Guidelines	ITC Guidelines on the Security of Tests, Examinations, and Other Assessments.	Here	Free	These guidelines focus on reducing cheating in tests and exams. They are quite old but still useful. Chapter 8 of the ITC/ATP Technology-based Assessment guidelines also provide similar and more up to date advice on security.
Privacy Guidance When Using Video In The Testing Industry	From the ATP	Here	$	This ATP guidance (where I was the lead author) provides guidance when using video to record test takers, either in remote proctoring or in test centers to ensure a proper balance between security and privacy.
Student Privacy Pledge		Here	Free	The Student Privacy Pledge is a set of principles respecting learner privacy that many edtech vendors have signed up to, including Learnosity.

Technical standards to call assessments and/or pass results back

Let’s start with standards that allow a system such as an LMS (learning management system), CMS (course management system) or applicant tracking system to call an assessment and/or get results back. Most of them can also be used to call e-learning as well as assessments.  

	Formal name	URL	Available	Description
AICC HACP	HTTP AICC CMI Protocol	Here	Free	The aviation industry need for e-learning interoperability preceded the Internet. The Aviation Industry Computer-Based Training Committee (AICC) defined its first standards in the early 90s, and even though the AICC itself disbanded a decade ago, its AICC HACP protocol is still widely used to call assessments from LMSs.
cmi5	Computer Managed Instruction	Here and here	Free	cmi5 was started by the AICC as a proposed replacement for AICC HACP and SCORM, but eventually became a component of xAPI. It allows calling an assessment and returning the results in a more effective way than AICC HACP or SCORM. Cmi5 is used, but is not yet as widely adopted as AICC or SCORM.
HR Open Standards (aka HR-XML)	HR Open Standards	Here	$	HR Open Standards provide a range of XML and JSON definitions that support interoperability of various human resources activities, including calling a recruitment assessment from an applicant tracking system and passing results back.
IMS LTI	Learning Tools Interoperability	Here	Free	IMS LTI (now called 1EdTech LTI) is a learning tools interoperability standard widely used in K12 and higher education systems. Among its uses is the ability to call an assessment from a course management system.
SCORM	Shareable Content Object Reference Model	Here	Free	SCORM was defined by the US Department of Defence and allows learning content and assessments to be easily launched and tracked from a learning or other management system. There are two flavors (SCORM 1.2 and SCORM 2004) and although it has some flaws (including that all communication is via the browser), SCORM is still widely used to deliver assessments and learning content from LMSs.
xAPI	Experience API	Here	Free	xAPI is widely used in corporate and other learning to capture data about learning. The main use of xAPI for assessments is to pass back answers to questions and/or assessment results for amalgamation with other learning data in a learning record store.

Some more specialist guidelines

There are a wide variety of more specialist guidelines out there. Here is a selection.

	Formal name	URL	Available	Description
ETS Best Practices for Constructed Response Scoring		Here	Free	ETS have huge experience in manual and automated constructed response and essay scoring and these guidelines have been widely reviewed.
Computer Adaptive Testing (IMS/1EdTech)		Here	Free	This public candidate from 1EdTech (previously IMS) is a way of providing interoperability between testing platforms and computer adaptive test engines.
ITC Guidelines for Translating and Adapting Tests		Here	Free	Widely respected guidelines from the International Test Commission on translating and adapting tests and exams.
Proctoring Best Practices	(from ATP and NCTA)	Here	$	These guidelines date back to 2015 but still have some useful guidance on proctoring.
The Design and Delivery of Assessment Centres	From the British Psychological Society	Here	Free	Provides guidance on setting up assessment centres in a workplace context.

Last but not least

This article focuses on standards and guidance that help you build assessment systems, but you should be aware of the pre-eminent document in psychometric good practice—“the Standards” below:

	Formal name	URL	Available	Description
“The Standards”	The Standards for Educational and Psychological Testing	Here	Free online (print version $)	This is the gold standard on psychometric good practice from the US APA, AERA and NCME. It focuses on psychometric precepts like validity, reliability, and fairness.

Many of the above documents and organizations are trademarks or registered trademarks of their owners, all rights respected. 

I hope this list of learning and assessment standards will help some of you take ideas, data models, or ways to interoperate from other parts of the community and help build better systems to educate and assess the world.

If you have any suggestions for other documents we can add to the list, feel free to get in touch.