ProctorU will become the largest test security provider to use trained human proctors for every test session.
HOOVER, Ala. (May 24, 2021, 8:00 a.m. EDT) – ProctorU—the academic division of Meazure Learning—which is the nation’s leading provider of remote proctoring and integrity safeguards for online testing and assessments, today announced it will no longer offer services that do not include trained human test proctors, effectively eliminating the use of AI-only products company-wide.
The new policy details:
- Effective immediately, ProctorU will no longer offer test proctoring services that are fully automated and that only record a test session and use AI or similar analytics tools to “flag” potential violations of test rules for review by the school or test provider.
- ProctorU will transition its education and test partners that currently use fully automated, record-only services to professional, human review proctoring with the goal to complete those migrations by the 2021/22 academic year.
“We believe that only a human can best determine whether test-taker behavior is suspicious or violates test rules,” said Scott McFarland, CEO of ProctorU. “Depending exclusively on AI and outside review can lead to mistakes or incorrect conclusions as well as create other problems.”
The company is adopting this policy after a complete review of its own data and service offerings and in consultation with education and testing partners.
While ProctorU did not use AI-only review systems to determine misconduct – only the school or test provider can reach those conclusions – the company has nonetheless determined that using technology alone, without trained human proctors, has three main deficiencies and side effects that significantly undercut its effectiveness: a failure to consistently review test sessions, increased opportunity to unfairly implicate test-takers in misconduct, and increased workload for instructors.
By definition, record-only proctoring does not meet the published standards for test integrity. According to Proctoring Best Practices: Association of Test Publishers and National College Testing Association:
“While record and review falls short of this document’s definition of proctoring with respect to security and should not be used for programs requiring high levels of security, it does potentially offer improved security over testing in an unproctored setting, provided the videos are all actually reviewed following the test administrations and individuals reviewing the videos are given detailed information on how to identify suspicious behavior during an online test.”
ProctorU’s review also shows that school or instructor reviews of AI-flagged test sessions are largely not happening, which means those methods fail even the most minimum standards.
Analysis of data from ProctorU showed that only about 11 percent of test sessions tagged for suspicious activity by AI tools are reviewed by the school or testing authority. An unrelated audit of proctoring at the University of Iowa showed similar findings, that just 14 percent of test sessions flagged for possible rules violations such as cheating were reviewed by faculty.
ProctorU’s analysis also revealed that AI-only proctoring systems could return anomalies that trained humans would easily recognize as meaningless. For example, AI could flag a test session inappropriately if a student repeatedly rubbed their eyes or if there was an unusual background noise, like a dog barking. And while this should not result in a finding of misconduct, human proctors are trained to discern and dismiss innocuous actions or sounds.
ProctorU has also determined that AI-only systems unfairly and unreasonably increase workload and expectations for instructors who are already overburdened. For example, reviewing the recordings of just a single, 60-minute exam for a class of 150 students can take nine hours, time that neither instructors nor their teaching assistants should be expected to invest.
“Faculty want to teach, not be hall monitors. No one becomes a teacher to sit around watching recorded test sessions to catch cheaters,” McFarland said. “But the critical point here is that people can tell when someone is trying to be dishonest, but computers aren’t so good at that.”
Independent studies continue to confirm that proper remote proctoring effectively protects academic integrity, safeguarding the work of the test-giver as well as the test-taker. Exam proctoring should be part of a robust and reinforcing academic integrity culture that includes institutional honor codes, student supports, modern assessment design, smaller class sizes, strong instructor-to-student relationships, and clearly stated and visible and enforced policies related to expectations and the consequences of misconduct.
“AI-only and record-and-ignore proctoring practices are no replacement whatsoever for trained humans,” McFarland said.
About ProctorU
ProctorU, the academic division of Meazure Learning, provides a full suite of online proctoring and identity management solutions for education, professional development and credentialing organizations. With patented, 24/7 live proctoring backed by powerful analytics, ProctorU offers a complete, convenient and cost-effective alternative to physical test centers and a solution to the dangers of unproctored assessments. ProctorU increases access to online learning and career credentials while ensuring exam integrity and accountability for every test-taker. ProctorU.com MeazureLearning.com
###
Media Contacts:
Jennifer Harrison for ProctorU
[email protected]
+1-916-716-0636