Each year, the assessment community gathers for the ATP Innovations in Testing conference. This year’s conference—held in Orlando, FL, from March 23 to 26—was themed “Better Together: Testing the Limits.”
The transformative impact of artificial intelligence (AI) was a major theme throughout the event, highlighting its practical applications and its potential in test development, scoring, and security. Sessions also highlighted the ethical implications, emphasizing that AI’s integration into testing requires careful consideration of algorithmic bias, data privacy, equity, and responsible implementation.
While AI dominated many conversations, other critical topics emerged: How do we create a positive candidate experience while maintaining test security? How do we ensure assessments are fair and inclusive? These questions drove discussions on flexible testing models, robust security measures, and the integration of diversity, equity, and inclusion (DEI) principles.
Meazure Learning actively contributed to this year’s conference, presenting in nine sessions and championing five team members serving on ATP committees. Our own Maria Incrocci—Senior Vice President of Psychometric and Assessment Services and immediate past-chair of the ATP Certification & Licensure Division—moderated the conference’s sole session on performance-based testing.
Maria and panelists Elizabeth Santana (Inteleos), Marcus Bolton (Performance Testing Council), and Maurilio Gorito (MG Credentialing, Performance Testing Council) highlighted seven areas of consideration when exploring the possibilities of performance-based testing as a component of assessment strategies.
1. Value Add—Why performance-based testing?
Performance-based testing enhances an assessment’s validity and purpose. It moves beyond simple information recall, assessing a candidate’s ability to practically apply knowledge by completing tasks that a credential-holder would be expected to perform. By mirroring real-life scenarios, performance-based testing provides true evidence of a candidate’s capacity to perform core job functions in a way that traditional multiple-choice questions can’t capture. This approach, which can be effectively paired with traditional assessment, is supported by the U.S. Equal Employment Opportunity Commission for its job relevance and is favored by candidates who appreciate its direct connection to on-the-job tasks.
2. Applications—When should performance-based testing be considered?
Performance-based testing is beneficial when evaluating skills that cannot be effectively assessed through traditional methods alone. Consider the road test portion of a driver’s license exam—most people would agree that a simple written assessment cannot determine a candidate’s ability to operate a vehicle correctly and safely. Students must show knowledge of laws and mastery of vehicle operation to prove competency. Similarly, performance-based testing is a smart choice when core job competencies involve practical application and not just textbook knowledge, particularly in high-stakes fields such as healthcare and trades where proven proficiency is critical to safety and positive outcomes. Ultimately, the decision to implement performance-based testing should align with a program’s specific objectives and needs.
3. Feasibility—What does the roadmap for a performance-based testing program look like?
Developing a performance-based testing program requires a structured and informed approach. Begin by reviewing existing content outlines and collaborating with subject matter experts (SMEs) to identify skills that are best evaluated through direct observation or simulation. If there is a strong justification for performance-based testing, gaining support from stakeholders is crucial. Establish a shared understanding within your organization by asking:
- Does leadership have all the necessary information to align on this approach?
- Are the appropriate resources (e.g., technological, experts, financial) for development and administration available?
- Is the effort required to develop the assessment realistic for your organization?
Researching established simulation-based testing practices within the industry can also provide valuable insights and guide program development.
4. Operations—How should we prepare candidates for performance-based testing?
Preparing candidates for performance-based testing follows many of the same principles as traditional testing—it requires standardized procedures and communication. Be sure to provide clear information about the assessment format, length, and required activities. Safety protocols, especially in simulations for high-risk scenarios like high-voltage power line work or surgical procedures, should also be meticulously defined. For technology-based performance testing, allowing candidates to familiarize themselves with the assessment controls and tools before the exam is essential for preparing them to focus on the task at hand. The goal? Eliminate any informational, environmental, or procedural barriers to create a positive and fair experience for candidates.
5. Measurement—How should we think about measurement in performance-based testing?
Begin by clearly defining the assessment’s purpose: Is it foundational, measuring personality and aptitude? Is it technical, focusing on job knowledge? Or is it a true performance-based evaluation designed to evaluate competency (the ability to perform a task) and not just competence (recalling learned information)? Then, work with SMEs to establish standardized rubrics and clear definitions for what is considered “acceptable” or “unacceptable” performance. Appropriate measurement models and targeted training for expert evaluators are vital for maintaining integrity, consistency, and equity.
6. Security—What are the key security considerations?
Security considerations in performance-based testing mirror those of traditional assessments. While the method differs, the need for vigilance remains constant. Safeguarding against potential threats like proxy testing and the use of unauthorized aids is still a priority, as is protecting exam integrity. Proactivity should always be part of the overall test security strategy.
7. Resources—How should sponsors think about budgets?
Performance-based testing typically requires a more substantial investment than traditional assessments. This is due to factors such as developing realistic simulations, acquiring supplies and specialized equipment, and continuous training for evaluators. Budget planning should account for the entire life cycle of the assessment, including design, implementation, and ongoing maintenance. Exploring cost-effective strategies, like leveraging existing resources or partnering with experienced providers, can also help optimize budget allocation.
While the investment is significant, so are the potential returns on investment. Performance-based testing can enhance the interpretation and use of outcomes by demonstrating a candidate’s competency in a well-defined area, improve candidate perception and experience, and strengthen a program’s reputation by producing professionals who are proven to perform at a satisfactory level.
“Performance testing is gaining more traction because technology is providing test sponsors with ways to evaluate candidates’ mastery of tasks performed on the job in an efficient way.”
—Maria Incrocci, SVP of Assessment and Psychometric Services, Meazure Learning
Advancing Assessment Through Partnership and Continued Learning
Meazure Learning’s active involvement at the ATP conference reflects our commitment to the organization’s mission: to promote the merit of testing, foster collegial relationships, and encourage professionalism within the industry. These industry gatherings are invaluable for facilitating the exchange of ideas, and we aim to continue the conversation through learning opportunities like our Expert Series of webinars. This year will bring a fresh slate of webinars—including a deep dive into the practical applications of performance-based testing—to help the testing community stay abreast of industry trends. Sign up to get notifications about upcoming webinars and join us as we continue to explore the evolving landscape of assessment.