39 standards to help build a learning or assessment product
“Standing on the shoulders of giants” is a well-known metaphor for advancing on the foundations of the past. These standards and guidelines provide a framework for quality in edtech.
We are fortunate in the learning and assessment space that there are a lot of smart people who have laid down useful standards and guidelines. These provide a navigational compass for product builders when developing products, supporting interoperable systems, providing technical safeguards for future clients, and generally improving the quality and usability of the work being done.
We thought it would be useful to give edtech builders to a repository of the key standards in the space.
Accessibility standards
The main document to look at for accessibility is the general WCAG standard, but the two IMS/1EdTech documents are worth noting.
| Formal name | URL | Available | Description | |
| APIP | Accessible Portable Item Protocol | Here | Free | APIP documents accessibility for questions/items, providing the information needed to make questions accessible for learners with a variety of disabilities and special needs. It was originally created separately from QTI but is now also included within IMS QTI v3. |
| AccessforAll / PNP | From 1EdTech (previously IMS) | Here | Free | IMS (1EdTech) initiative on accessibility and personal needs preferences. |
| WCAG | Web Content Accessibility Guidelines | Here | Free | These general guidelines apply to all web content, not just assessments, but they are the core of any attempt to make assessments effective for people with disabilities. |
Accreditation of assessments or testing programs
These standards allow accreditation of certification or other kinds of testing programs, and so are relevant to those building systems to make sure that your customers can meet their needs.
| Formal name | URL | Available | Description | |
| ICE 1100 | Standard for Assessment-Based Certificate Programs | Here | $ | This standard from the Institute for Credentialing Excellence(ICE) and approved by the US ANSI specifies requirements for assessments for certificate programs. It’s possible to get accredited by ICE against the standard. |
| ISO 17024 | Conformity assessment — General requirements for bodies operating certification of persons | Here | $ | This ISO standard (also distributed by ANSI others) defines how a certification organization that certifies people should operate. A certification body can be certified against ISO 17024 to demonstrate credibility and that it is following good practice. |
| ISO 10667 | Assessment service delivery — Procedures and methods to assess people in work and organizational settings | Here and here | $ | Defines good practice in using assessments in the workplace. Has two parts—one aimed at service providers and one aimed at companies who use assessments, and you can be certified on either part. |
| NCCA Standards | NCCA Standards for the Accreditation of Certification Programs | Here | $ | The NCCA standards are well respected and developed by the Institute for Credentialing Exercise. They allow accrediting of certification programs. |
Building and using technology
Technical standards are useful but there also are a few widely reviewed guideline documents on building systems. The most significant and valuable from the following selection is from ITC and the ATP.
| Formal name | URL | Available | Description | |
| ITC/ATP TBA Guidelines | Guidelines for Technology-based Assessment | Here | Free | This mammoth document was published late in 2022 and provides guidance on good practice in technology assessment from the ATP and ITC. Anyone building an assessment system should at least skim these guidelines—see my earlier blog for Learnosity on them. |
| ISO 23988 | Information technology — A code of practice for the use of information technology (IT) in the delivery of assessments | Here | $ | This standard is quite old (dating from 2007) but provides general good practice in using technology for assessment delivery. I actually came up with the idea for this standard and led the team that produced the British standard that was in due course updated to be an international standard. |
| Principles of AI use in Testing | (from ATP) | Here | Free | Short general principles on using artificial intelligence in testing. |
| Testing and data integrity in the administration of statewide student assessment programs | (from NCME) | Here | Free | Some general guidance from NCME about on ensuring testing data integrity. Around 10 years old, so a little dated, but still potentially of value. |
Content packaging and metadata standards
In an already dry standards world, metadata and content packaging are particularly specialist, but they can be useful.
| Formal name | URL | Available | Description | |
| Dublin Core LRMI | Learning Resource Metadata Initiative | Here | Free | A collection of classes, properties and concept schemes for markup and description of educational resources. |
| IEEE Standard for Learning Object Metadata (LOM) | Here | $ | A conceptual data schema that defines the structure of a metadata instance for a learning (or assessment) object. | |
| IMS Content Packaging | Here | Free | A very old but still used definition of a data structure for describing or passing content packages (that can include assessments) from one system to another. |
Credentials and badging
If you are working in the space of competencies, credentials and badging, these standards are worth looking at to see if you can use rather than reinventing the wheel.
| Formal name | URL | Available | Description | |
| CTDL | Credential Transparency Description Language | Here | Free | This is a schema for describing credentials from a variety of sources from the non-profit Credential Engine organization. |
| EDCI Standards | European Digital Credential Standards | Here and Here | Free | These hold definitions of credentials used within the European Union, which allow digital versions of credentials and qualifications. |
| IEEE Recommended Practice for Defining Competencies | Here | $ | This 2022 standard from IEEE focuses on recommended practice to define competencies but also how to interoperate them | |
| Open Badges | Here | Free | This specification now maintained by IMS/1EdTech is a way for packaging information about accomplishments (including assessment results) and embedding it into portable image files as digital badges. If you are creating digital badges, you should probably use this standard. |
Data exchange (between systems)
Another kind of technical standard is one that allows you to pass data from one system to another. They typically have data models and ways of interchanging data like test results from one system to another. There are several such standards, here are some of the key ones:
| Formal name | URL | Available | Description | |
| A4L SIF | Assessment 4 Learning Community Systems Interoperability Framework | Here | Free | The SIF specifications are a technical blueprint for allowing diverse educational applications to interact and share data. There are versions for the US, Australia, New Zealand, the UK and for global use. |
| CEDS | Common Education Data Standards | Here | Free | CEDS is an education data management initiative that includes a common vocabulary and data models. |
| Ed-Fi Data Standard | Here | Free | The Ed-Fi Data Standard is a US-focused data standard allowing interchange of education data, including assessment data. | |
| IMS QTI | Question and Test Interoperability | Here | Free | IMS QTI (now 1EdTech QTI) is one of the oldest and most widely used assessment specifications. I was part of the team that created QTI v1 back in 2000, and my Learnosity colleague Steve Lay led the team that produced QTI v2. The latest version is v3. Although it has other use cases including results reporting, one of the key QTI use cases is to exchange questions being systems. In theory, a question written in QTI in one system can be exported to another system and used there. |
| PESC | Postsecondary Electronic Standards Council | Here | Free | PESC has a range of technical standards allowing data interchange in areas such as US Higher Education admissions. Their data standards include college and high school transcripts and test scores. |
Security and privacy
Anything edtech has to be secure and must respect learner privacy. Here are some useful standards and guidelines in this space.
| Formal name | URL | Available | Description | |
| ISO 27001 | Information security, cybersecurity and privacy protection – Information security management systems | Here | $ | This is a general standard, not specifically aimed at assessments but since information security is so important, anyone who is delivering assessments will likely want to comply with it. Many will choose to be certified against it, as Learnosity has done. There are also other useful security standards in the ISO 27000 family. |
| ITC Security Guidelines | ITC Guidelines on the Security of Tests, Examinations, and Other Assessments. | Here | Free | These guidelines focus on reducing cheating in tests and exams. They are quite old but still useful. Chapter 8 of the ITC/ATP Technology-based Assessment guidelines also provide similar and more up to date advice on security. |
| Privacy Guidance When Using Video In The Testing Industry | From the ATP | Here | $ | This ATP guidance (where I was the lead author) provides guidance when using video to record test takers, either in remote proctoring or in test centers to ensure a proper balance between security and privacy. |
| Student Privacy Pledge | Here | Free | The Student Privacy Pledge is a set of principles respecting learner privacy that many edtech vendors have signed up to, including Learnosity. |
Technical standards to call assessments and/or pass results back
Let’s start with standards that allow a system such as an LMS (learning management system), CMS (course management system) or applicant tracking system to call an assessment and/or get results back. Most of them can also be used to call e-learning as well as assessments.
| Formal name | URL | Available | Description | |
| AICC HACP | HTTP AICC CMI Protocol | Here | Free | The aviation industry need for e-learning interoperability preceded the Internet. The Aviation Industry Computer-Based Training Committee (AICC) defined its first standards in the early 90s, and even though the AICC itself disbanded a decade ago, its AICC HACP protocol is still widely used to call assessments from LMSs. |
| cmi5 | Computer Managed Instruction | Here and here | Free | cmi5 was started by the AICC as a proposed replacement for AICC HACP and SCORM, but eventually became a component of xAPI. It allows calling an assessment and returning the results in a more effective way than AICC HACP or SCORM. Cmi5 is used, but is not yet as widely adopted as AICC or SCORM. |
| HR Open Standards (aka HR-XML) | HR Open Standards | Here | $ | HR Open Standards provide a range of XML and JSON definitions that support interoperability of various human resources activities, including calling a recruitment assessment from an applicant tracking system and passing results back. |
| IMS LTI | Learning Tools Interoperability | Here | Free | IMS LTI (now called 1EdTech LTI) is a learning tools interoperability standard widely used in K12 and higher education systems. Among its uses is the ability to call an assessment from a course management system. |
| SCORM | Shareable Content Object Reference Model | Here | Free | SCORM was defined by the US Department of Defence and allows learning content and assessments to be easily launched and tracked from a learning or other management system. There are two flavors (SCORM 1.2 and SCORM 2004) and although it has some flaws (including that all communication is via the browser), SCORM is still widely used to deliver assessments and learning content from LMSs. |
| xAPI | Experience API | Here | Free | xAPI is widely used in corporate and other learning to capture data about learning. The main use of xAPI for assessments is to pass back answers to questions and/or assessment results for amalgamation with other learning data in a learning record store. |
Some more specialist guidelines
There are a wide variety of more specialist guidelines out there. Here is a selection.
| Formal name | URL | Available | Description | |
| ETS Best Practices for Constructed Response Scoring | Here | Free | ETS have huge experience in manual and automated constructed response and essay scoring and these guidelines have been widely reviewed. | |
| Computer Adaptive Testing (IMS/1EdTech) | Here | Free | This public candidate from 1EdTech (previously IMS) is a way of providing interoperability between testing platforms and computer adaptive test engines. | |
| ITC Guidelines for Translating and Adapting Tests | Here | Free | Widely respected guidelines from the International Test Commission on translating and adapting tests and exams. | |
| Proctoring Best Practices | (from ATP and NCTA) | Here | $ | These guidelines date back to 2015 but still have some useful guidance on proctoring. |
| The Design and Delivery of Assessment Centres | From the British Psychological Society | Here | Free | Provides guidance on setting up assessment centres in a workplace context. |
Last but not least
This article focuses on standards and guidance that help you build assessment systems, but you should be aware of the pre-eminent document in psychometric good practice—“the Standards” below:
| Formal name | URL | Available | Description | |
| “The Standards” | The Standards for Educational and Psychological Testing | Here | Free online (print version $) | This is the gold standard on psychometric good practice from the US APA, AERA and NCME. It focuses on psychometric precepts like validity, reliability, and fairness. |
Many of the above documents and organizations are trademarks or registered trademarks of their owners, all rights respected.
I hope this list of learning and assessment standards will help some of you take ideas, data models, or ways to interoperate from other parts of the community and help build better systems to educate and assess the world.
If you have any suggestions for other documents we can add to the list, feel free to get in touch.
Related articles
ATP’s new best practice guidelines for Technology-Based Assessment—a summary
A major set of guidelines for technology based assessment recently launched. Here, Learnosity EVP John Kleeman reveals its key takeaways.
“Building the bullseye”: How product demos can address diverse prospect needs
Keeping things simple by focusing on what’s important is how you demonstrate both understanding and compatibility with a client's needs.
High product standards begin with strong design principles
Creating strong core design principles plays an essential part in maintaining high product and user experience standards.