Proctored testing: a necessary feature or an overused hype?

Online proctoring definition

If you think that a similar situation wouldn't happen in your firm or educational institution, you are very likely wrong.

Nowadays, talent recruitment, periodic workforce training, online course testing, certification, and other business processes depend on online assessments. Any organization using remote tests is likely to face the problem of decreasing results reliability, caused by cheating.

To be effective, online tests must be reliable. Their results must represent respondents’ true skills and knowledge. The greatest threat to the reliability of the results is online cheating. This behavior is not honest, has damaging effects on the performance of any business, and has to be dealt with properly.

There are many ways for less-than-honest test takers to get better results without making an effort. We talk about how to prevent online test cheating in another of our blog posts.

It is possible to solve the problem of dishonest behavior on online tests and exams. One idea is to have them proctored meaning that a person and/or software monitors students for suspicious activities.

Proctoring features of AI-based solutions

AI-based proctoring operates by identifying potential dishonest respondents attempting to use different ways to cheat on an online test. It can be pre-set to issue a warning or terminate the test for a responder caught red-handed.

Although online proctoring platforms are different in detail, they offer similar security measures:

• Respondent identity recognition
  Facial recognition allows the proctoring software to determine if the respondent present in front of the camera is the one who should be taking part in the assessment. AI scans the face and verifies it against the ID card that a respondent must show to the camera.
  
  Test organizers can also have proctoring software repeat the process several times during the assessment, for a higher level of security. The software alerts a human proctor when it detects a possible violation, who then decides how to proceed. ID recognition is the best means of catching cases of cheating by impersonation.
  
  

• Respondent and other person presence detection
  Artificial intelligence solutions can easily notice when a respondent is not visible or only partially visible in the camera feed. In a similar way, proctoring software can spot the appearance of other persons.
  
  

• Gaze direction detection
  This feature of an AI can detect when a respondent is not looking at the device screen. If it takes more than a moment, they are possibly gazing at some sort of cheat sheet.
  
  

• Voice detection
  Sometimes less-than-honest respondents try to cheat by having another person dictate them answers to test questions. Voicing the questions out loud can indicate such an attempt. The software can target specific phrases, for example, 'Hey Siri' or 'Ok Google', which can start a voice-controlled search engine.
  
  Proctoring software can detect respondents' voices and flag possible cheating attempts, which are then notified to a live human proctor.
  
  

• Smartphones and other gadgets detection
  Electronic devices make cheating easier for test respondents. Remote test proctoring platforms can detect the use of calculators, smartphones, smartwatches, tablets, earphones, and other gadgets during the assessment.
  
  

• Screencasting and virtual machine detection
  Another popular way of cheating during tests and exams is the use of software. Respondents utilize TeamViewer, Skype, and similar screencasting software. They also can use a virtual machine or thin client to let proxy answer questions for them.
  
  During an exam that is remotely proctored, it is possible to detect the use of this software and then stop the respondent's dishonest activity immediately.
  
  

• Web browser lockdown
  To make looking up answers online impossible, remote proctoring platforms create a secure environment. It restricts access to other websites and disables keyboard shortcuts. Test-takers are not allowed to copy and paste, look up information in search engines, or open other apps while taking the test.

To understand the basics of online proctoring, watch this short AI + live proctors presentation. Examus offers strict proctoring solutions, useful in high-stakes educational exams and business certification processes.

Test proctor definition

Proctoring AI can notify a human proctor. In an online world, it is a person, who looks for signs of cheating during the test, reviews the session recording, and reacts to situations flagged by the AI. In such a case, this real person, not the artificial intelligence, makes the ultimate decision whether a situation was a cheating attempt or not.

Why strict proctoring solutions? 5 useful advantages

Is online test proctoring a necessary integrity measure for business and educational organizations? Or is it a modern hype, overused security that can do more harm than good? We'll look at both sides of the argument to decide whether proctored testing is necessary for online assessments.

1. It’s a much-needed security measure
   The first argument in favor of proctoring is that it increases result reliability. Proctoring online tests helps to ensure that the test-takers are who they say they are and that they are not cheating on the test. Particularly when an assessment is a one-off online test, without any follow-up meetings or calls.
   
   This is important for businesses, which need to be able to trust the results of their employees' online assessments. False results undermine the validity of testing and can lead to unfairness in the workplace.
   
   

2. It’s a smart security measure
   The second argument in favor of proctoring is that it can help various organizations adapt to the ever-changing needs and ensure a positive experience for test-takers.
   
   For example, if a business changes its testing requirements, the proctoring system can be updated to reflect these changes. This helps to ensure that the test results are accurate and reliable. This is especially important in businesses where the stakes are high and the competition is fierce.
   
   

3. It offers a better assessment experience
   The third argument in favor of remote test proctoring is that it is much easier for respondents to take their tests and exams online, from the comfort of their homes.
   
   When a test is remotely proctored, meaning that human supervisors and/or AI proctoring solutions oversee it online, it benefits both the test organizer and the respondents. It is also much more convenient for the K12 /higher education or business institutions that organize the evaluation.
   
   

4. It’s a value for money solution
   The fourth argument in favor of online exam proctoring is that it saves money. With the use of artificial intelligence, fewer human proctors must oversee a test. All they need to do is react whenever a proctoring platform flags a possible cheating attempt.
   
   Online proctoring is also scalable. This feature ensures that organizations pay for what they actually use. It also provides test organizers with detailed reports and analytics. Not only on cheating attempts but also the general performance of their respondents.
   
   

5. It saves time
   The fifth, and final argument in favor of remote proctoring is that it increases the effectiveness. AI can proctor dozens of test respondents simultaneously. It also ensures that human proctors have to rewatch only flagged fragments of recordings.
   
   This creates a much better environment for the respondents, who get their results faster and don't have to wait anxiously.

To understand the basics of online proctoring, watch this short AI + live proctors presentation by our partner, Examus. They offer strict proctoring solutions, useful in high-stakes educational exams and business certification processes.

Or maybe it’s not for me? 4 reasons to avoid strict online proctoring

1. It invades test-takers' privacy
   The main argument against virtual proctoring is that it can be intrusive and stressful. Many respondents feel anxiety knowing that live proctors or AI are supervising them via video meeting software.
   
   This can significantly impact their experience. Even if they enjoyed a training session or recruitment process, ending it with a strictly remotely proctored test might turn a positive impression into a negative one.
   
   

2. It is even more stressful with artificial intelligence supervision in place
   Automatic virtual proctoring can become problematic. Test-takers very often complain about AI solutions. The reason behind their attitude is obvious. The software which judges their every movement and looks for signs of cheating might be considered unfair.
   
   

3. It creates an entry barrier for people with disabilities
   Respondents suffering from very common behavior disorders, such as ADHD (Attention deficit hyperactivity disorder) might find it extremely difficult to take a test in such a way that doesn't trigger system warnings.
   
   We have to mention here that generally, online assessments are more accessible for people with disabilities than on-site ones. However, in this scenario, using strict proctoring by AI solutions or live proctors might have a negative impact on the performance of some respondents.
   
   

4. It offers empty promises of full security
   Even the most innovative and accurate remote proctoring automation systems won't be able to track all violations of rules. In the case of high-stakes assessments, respondents might be tempted to outsmart them.
   
   We must stress that although it increases test integrity and results reliability, it doesn't guarantee 100% cheating deterrence or detection.

How to proctor an online exam: full AI, human proctoring, or both?

Many online proctoring apps offer:

• automatic, artificial intelligence-based solutions,

• AI + a post-evaluation review by proctors,

• a combination of a ‘thinking' computer and live human proctoring.

Using online test proctoring based only on artificial intelligence is a great solution for low-stakes assessments. This anti-cheating measure is fully automated, so it doesn't require any human action after the test is prepared. This set-and-forget measure will be perfect at the lower levels of K12. Tests that are done often but don't have a lot of long-term impact are not that important.

If we want the proctoring features of AI-based solutions to be effective and ethical we must ensure one thing. Respondents must know that they will be overseen by AI only. Some options should also be switched off, for example, gaze detection, as it is likely to trigger warnings as with no human proctor available, no one can decide whether cheating occurred, or not.

In the case of assessments proctored by humans, personnel availability is often the biggest challenge. It is possible to deter and detect cases of online cheating. But to achieve that, a ratio of no more than 1 proctor per 10 respondents is a must. Anything more and supervisors simply won't be able to keep up as they will have to constantly switch from one test-taker live camera feed to another.

The only possible way to solve this problem is to record every test session and review everything afterward. However, this gives human proctors even more work, as they will likely have to rewatch hours of recordings.

Artificial intelligence should be responsible for verifying ID and flagging all kinds of suspicious behavior. But at the same time, a human proctor needs to supervise AI all the time. The final decision on whether a candidate is cheating should be made by a real person, not an algorithm.

How much does online proctoring cost and is it worth the money?

The cost of virtual proctoring solutions varies, depending on the kind of security we expect. Different levels of supervision are recommended for specific types of remote assessments. Make sure you consider your needs and the price tag attached when selecting a solution for your organization.

AI-only proctoring costs

Many companies offering online proctoring solutions require new customers to pay an initial, one-off set-up fee. In this case, the payment, regardless of the size of the client organization, will start around $1.500 - $2.000, sometimes even more.

They also require pre-payment for a significant number of test sessions, the minimum being usually around 1.000, and the lowest prices available for orders in excess of 5.000 test sessions.

If an organization wants to assess skills and knowledge online with an extra layer of security and give their tests a 'human face', it can do that by using proctors. There are two ways to do that.

The first is to use an organization's own personnel to supervise assessments and respond to possible violations of test rules spotted by artificial intelligence. The second includes professional proctors supplied by the proctoring company. There is a significant difference between the two.

Internal proctors costs

Using your own human proctors is a low-cost option. It usually requires no additional expenses as proctors are usually already employed by the test organizers. Unfortunately, those employees might not be proficient with various procedures of proctoring and necessary training might take time and cost money.

External proctors costs

Opting for online proctoring by company-supplied personnel guarantees a much higher level of expertise and better security. It is also much more expensive. Additionally, it might also not be very popular with the respondents, who in general would rather be supervised by people they know - their teachers, trainers, managers, or coworkers.

Do I need proctoring for my online assessments?

There is a fine line between proctoring and invigilation. You must remember that in the end, your respondents will decide whether you crossed it or not. Look after their experience and you will succeed. In fact, for most of your online assessments, you don't really need strict online proctoring! No artificial intelligence, and/or human proctors, constantly looking over your test-takers shoulders.

It is best to avoid strict remote proctoring in mid-term summative assessment, formative assessment and most business use cases. For example, regular, low-stakes assessments or quizzes, as well as most recruitment processes and company trainings will do fine without any AIs and human proctors. It is much better to create an atmosphere of trust. Too much security can lead to respondents becoming resistant to proctoring even when it is really necessary.

In summative assessments, such as final exams, high-stakes tests, as well as business certification, etc. the level of security should be higher. But still, these evaluations are possible without the support of artificial intelligence solutions or human proctors. There are other options available, such as Testportal’s Honest Respondent Technology.

It can also disable a respondent’s access to the test if they leave the test tab too many times. This is a good example of a feature that ensures assessment results reliability and, at the same time, respects the privacy of respondents.

Strict online proctoring - unnecessary in most cases

Remote proctoring solutions are certainly considered hype now, after their boom during the COVID-19 pandemic. It is a relatively new technology that became famous due to its popularity and controversies in remote assessments for education.

Many business professionals think of using extra security for their online recruitment, training, or certification. It seems that many managers fall for proctoring platform marketing efforts. They check the available options and pricing and think that “the more we pay, the more secure our tests will be”. They fail to account for the quality of the exam experience for their respondents.

That happens even in situations when their business online tests don't really need such a serious level of security. Because of the high prices of offered solutions, it might be a good idea to start looking for other options on how to ensure the reliability of remote test results.

It is much better to replace strict, invasive proctoring with light solutions that are available at hand and described in the section below.

Summary. Choose proctoring based on what you need, not on what they advertise

To get an efficient service that’s cost-effective and puts your respondents first, remember to adjust the platform you’re going to use to your requirements. Also, keep in mind that too much security can start feeling like unwanted surveillance.

The policy of trust toward your respondents is much more effective. Light security solutions you can find in the leading online test platforms, such as the Honest Respondent Technology mentioned above, and other smart, non-invasive security features, have several important benefits. It is very effective in deterring and detecting, as well as easy on your budget. Another advantage of this solution is that test-takers don’t consider it an invasion of their privacy.

Both business and educational remote assessment respondents are more likely to have a positive experience without strict proctoring.

Light proctoring, on the other hand, keeps online test results reliable and helps respondents focus on their tasks, making it the best available option.