EyesOff: How I built a screen contact detection model
8 days ago
- #privacy
- #machine-learning
- #computer-vision
- The author built a custom model named EyesOff to detect when someone is looking at your screen to prevent shoulder surfing.
- Due to the lack of an existing dataset, the author hand-labeled over 20,000 images and created synthetic gaze labels for pre-training.
- Initial approaches like Eye-Contact-CNN, MediaPipe, and gaze detection models were considered but deemed unsuitable for various reasons.
- The author developed a two-stage training approach: Phase 1 involved pre-training on a gaze regression task, and Phase 2 fine-tuned the model for screen contact classification.
- A key challenge was the lack of data, leading the author to create a custom dataset from sources like the Video Conferencing Dataset (VCD) and YouTube videos.
- The final model achieved ~71% accuracy across close and mid-range distances, with the best performance coming from models trained on expanded datasets.
- Future improvements include collecting more data, reducing model size, and potentially using a dual-model approach for close and far-range detection.