Data analytics are now considered table stakes in higher education and credentialing assessments. The widespread adoption of computer- and internet-based testing means we have more exam-related data at our fingertips than ever before. Moreover, the rising availability of AI tools has made it easier to analyze large amounts of this data and apply the insights to assessment security strategies.
Put simply, if you’re not actively engaging with all the data at your disposal, you’re missing out on critical opportunities to further safeguard your assessments.
In this article, we explore some of the exam-related data you can analyze, where you can find it, and how scrutinizing it allows you to make more strategic decisions about common security threats, like proxy testing, content theft, collusion, and the use of unauthorized aids.
Let’s dive in.
Understanding and Capturing Exam-Related Data
So, what do we mean by “exam-related data”? More importantly, how can you ensure that both you and your provider(s) are maximizing the use of this data to inform and improve your security strategies?
Useful and usable data can be collected before, during, and after a test administration or proctoring session. To be clear, the exam data we’re referring to is not what would be considered personally identifiable information (PII). We’re talking about scheduling and registration data, question response patterns, item performance metrics, online proctoring touchpoints, and more.
This type of information is often captured, managed, and stored in a variety of testing technologies, such as scheduling platforms, test-taker management systems, exam delivery platforms, and proctoring software. While different parties interact with these systems, their analytical capabilities vary widely. Each party—whether it be your institution or organization, your solution vendor, or a specialized third party—brings distinct strengths to the table. However, the most sophisticated partners have a wide range of sources they can tap into and layers they can sift through to draw valuable conclusions.
Using Insights from Data Analytics to Enhance Exam Security
Data alone doesn’t tell us anything. It’s the human analysis that transforms this data into meaningful and actionable insights. Whether analyzed individually or collectively, this information provides a window into every aspect of the testing process. Let’s examine how data insights can be applied to four common test irregularities.
Proxy Testing
When an imposter either physically sits in place of a test-taker or remotely controls their computer to complete an exam on their behalf, the validity of the exam itself becomes compromised. Identifying a proxy can be difficult though—not least because test-takers and those offering proxy services continually refine their tactics to evade detection.
However, by analyzing key data points gathered at different stages of the assessment life cycle, you can not only detect proxy testing as it happens but also identify and prevent potential instances after the exam concludes. Here are a few examples:
- Biometric comparison of ID photos taken at various points of the testing process can affirm the test-taker’s identity.
- Keystroke data can pinpoint inconsistencies suggestive of third-party involvement and identify similarities across different test sessions, potentially uncovering a serial proxy testing provider.
- Network data can reveal patterns or anomalies associated with proxy testing, such as sudden changes in a test-taker’s network connection.
Say, for example, your provider is analyzing data about the geographical distribution of your test-takers to provide more localized support. During this process, they notice an unusually high concentration of logins from a specific area where your program doesn’t have any registered test-takers. They dig a little deeper and learn these logins took place on VPN services, which are used to encrypt internet traffic and disguise a user’s actual location. This pattern raises concerns about the possibility of proxy test-takers, so they launch an investigation and explore security measures that can address this issue, like implementing stricter identity verification processes or requiring continuous proctor involvement.
Exam Content Theft or Harvesting
Another common threat to exam security occurs when a test-taker steals exam content with the intent of sharing or selling it. Implications of content theft or harvesting can be severe, ranging from compromised integrity to reputational damage, devaluation of degrees or credentials, and legal or financial repercussions.
Data collected from secure browsers, test delivery platforms, or proctoring solutions can serve as valuable indicators of potential content theft. These systems are often designed to collect data points such as:
- Attempts to access restricted computer functionality, like the ability to copy and paste exam content, take screenshots, or access external applications
- Screen, webcam, and audio recordings that simultaneously capture a test-taker’s virtual and physical testing environment from eye level in addition to providing a 90-degree angle view
- Live monitoring by a professional proctor, who can create a structured dataset by timestamping and documenting any suspicious activity or behavior
- Completion times and response patterns that reveal patterns indicative of a test-taker’s prior access to exam content
Imagine this: You’re reviewing data about the total time spent on an exam and notice a particular test-taker completed theirs faster than average. This may mean the test-taker had previous knowledge of the questions or answers, so you turn to the internet to check for potential leaks. Your team scours public and private social media platforms, internet forums, chat rooms, and test prep sites. Soon, you discover that some of your watermarked exam content (i.e., embedded with a marker or code) has been posted publicly. This watermarking allows you to trace the leaked material back to the person(s) responsible for stealing it. And since you know your exam has been compromised at this point, you make strategic security decisions to mitigate the damage, such as rotating your content, submitting a takedown request for the exposed content, randomizing your existing item order, or assembling multiple versions or forms of the test.
“Thanks to the digital nature of most tests today, nearly all attempts to steal or solicit exam content leave a trail of breadcrumbs you can follow.”
—Cory Clark, Vice President of Security, Training, & Quality at Meazure Learning
Test-Taker Collusion
Collusion occurs when test-takers collaborate with each other, a proctor, or an administrator to cheat on exams. To check for signs of collusion, you and your vendor(s) have a variety of data sources you can scrutinize. Here are just a few:
- Scheduling patterns that reveal some test-takers have a high frequency of rescheduled exams or consistently book their exams at the same time
- Proctor recordings or live proctor observations that show visual or auditory signs of collusion, like off-screen glances, background conversations, or test-takers appearing to be in the same room
- Secure browser data that reveals characteristics typical of virtual machines, which could indicate someone is trying to share or remotely access a testing environment
- Answer patterns and response times within the delivery platform that highlight unusually similar answers or answering speeds
Consider this scenario: After analyzing item- and test-level performance metrics, you notice a cluster of test-takers who answered questions too quickly and in a suspiciously similar way. These test-takers, all from the same geographic region and having taken the exam within a narrow time frame, even shared the same incorrect responses. To address this suspected collusion, you begin removing or altering the compromised questions. You also re-evaluate your security framework, potentially implementing more rigorous monitoring or advanced data forensic techniques.
Unpermitted Resources
A test-taker gains an unfair advantage of their peers when using a resource—whether virtual or physical—that’s prohibited by exam rules, they gain an unfair advantage over their peers. This action violates the integrity and fairness of the testing process and skews the overall test results.
A plethora of data gathered during the exam can signal whether unpermitted resources are used. Located in a test delivery platform, proctoring platform, or secure browser, these touchpoints include:
- Visual recordings captured by webcams about a test-taker’s physical environment and actions
- Audio recordings from microphones that suggest the presence of prohibited materials or assistance
- Screen captures displaying the test-taker’s computer screen
- Log data generated by algorithmic checks or scripts detailing attempts to use banned applications or visit restricted websites
Let’s bring this into the real world. Say your proctoring provider offers a wide-angle or secondary camera to enhance their understanding of a test-taker’s physical environment. This camera’s feed records the test-taker subtly yet consistently checking a laser-projected answer on the palm of their hand. Upon reviewing this footage, your proctoring provider flags this incident for your analysis. And after confirming that the test-taker did, in fact, use a wearable device to project answers onto their hand, you revise your exam policies to directly address this resource and work with your provider to develop targeted detection strategies for this new form of cheating.
Conclusion
Throughout this article, we’ve examined many types of exam-related data sources and their potential application in addressing common security threats. The biggest takeaway is this: Data is your friend. In fact, it may be your best friend as assessment technology advances, data sources expand, and our understanding of how to leverage it all deepens. The insights derived can help you and the organization(s) you work with bolster your exam security protocols and uphold the sanctity of the assessment process.
For more insights into the role that data analytics can—and should—play in exam security, check out our e-book “The Benefits of Online Testing.”