Sherlock: AI-Driven Online Proctoring Agent for Ensuring Fairness in Remote Interviews and Exams

What is Sherlock?

Sherlock is an AI-driven online proctoring agent developed by WeCP to replace traditional human proctors, offering a more efficient and intelligent solution for remote test monitoring. It analyzes video, audio, and system signals in real time to detect AI-assisted cheating behaviors, such as hidden devices, voice patterns, and tab-switching, to prevent fraudulent activities.

Key Features

1. Multi-Modal Real-Time Monitoring

Sherlock combines computer vision, audio analysis, and behavior pattern recognition technology to monitor the candidate’s behavior in real-time and identify potential cheating. Key monitoring functionalities include:

• Device Detection: Identifies the use of external devices such as smartphones, extra monitors, multi-screen setups, and virtual machines.

• Screen and Clipboard Monitoring: Prevents unauthorized screen sharing, copy-paste, and screenshot behaviors.

• Audio Monitoring: Detects human speech, test-related keywords, and AI-generated voice.

• Environmental Awareness: Detects multiple faces or other people in the test environment.

• Activity Log Recording: Records all activities with timestamps for later review.

2. Behavior Pattern Analysis

Sherlock analyzes candidates’ behavior patterns, such as abnormal typing speed, frequent tab-switching, and unusual response patterns, to identify potential cheating behavior.

3. Real-Time Alerts and Report Generation

When suspicious behavior is detected, Sherlock sends real-time alerts to administrators and generates detailed reports, providing video evidence and behavior analysis to assist recruiters in decision-making.

4. Security and Privacy Protection

Sherlock complies with security standards such as SOC 2 Type II and ISO 27001, ensuring data security and privacy protection.

Technology Principles

Sherlock is based on advanced AI and machine learning technologies, combining computer vision, audio analysis, and behavior pattern recognition to create a multi-layered monitoring system. It can analyze video streams in real-time to detect whether a candidate is using AI-assisted tools, such as voice prompts or screen sharing. Additionally, Sherlock employs deep learning models that continuously learn and adapt to new cheating techniques, ensuring ongoing effectiveness in monitoring.

Project Links

• Official Website: https://www.withsherlock.ai/

• GitHub Repository: https://github.com/WeCP/Sherlock

Use Cases

• Online Recruitment: Ensures fairness in remote interviews by preventing candidates from using AI-assisted tools to cheat.

• Online Exams: In the education field, prevents students from cheating during remote exams, ensuring the integrity of test results.

• Remote Training: Monitors learners during corporate training programs to prevent proxy learning and cheating.

• Legal Compliance: In industries with strict compliance requirements, ensures transparency and fairness in the recruitment and exam processes.