🚗 AI Driver Monitoring System

A real-time AI-powered driver monitoring system built with Flask, OpenCV, and YOLO that detects various driver behaviors and safety violations to enhance road safety.

📋 Table of Contents

• Features
• Demo
• Architecture
• Installation
• Usage
• Project Structure
• Detection Modules
• Alert System
• Configuration
• Models
• Contributing
• License

✨ Features

Real-Time Detection Capabilities

• 👁️ Blink Detection - Monitors eye aspect ratio (EAR) to detect blinking and drowsiness
• 😷 Face Mask Detection - Detects whether the driver is wearing a mask
• 📱 Phone Usage Detection - Identifies when the driver is using a mobile phone
• 🚬 Smoking Detection - Detects smoking behavior while driving
• ✋ Hand-on-Mouth Detection - Identifies potential distraction from eating/drinking

Smart Alert System

• Real-time WhatsApp Alerts - Sends instant notifications via Twilio
• Cloud Image Upload - Captures and uploads violation snapshots to Cloudinary
• Intelligent Cooldown - Prevents alert spam with configurable cooldown periods
• Multi-Behavior Tracking - Monitors multiple safety violations simultaneously

Web Interface

• Live Camera Feed - Real-time video analysis through webcam
• Image Upload - Analyze static images for violations
• Interactive Dashboard - Visual feedback with bounding boxes and confidence scores
• Responsive Design - Bootstrap-based UI that works on all devices

🎯 Demo

The system provides two modes of operation:

1. Live Analysis Mode: Real-time monitoring using your webcam
2. Image Upload Mode: Analyze pre-recorded images or screenshots

Detection Examples

• Drowsiness alerts when eyes remain closed for extended periods
• Phone detection with hand-phone overlap verification
• Mask compliance monitoring
• Smoking behavior identification

🏗️ Architecture

┌─────────────────┐
│   Web Browser   │
│  (HTML/JS UI)   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Flask Server   │
│   (app.py)      │
└────────┬────────┘
         │
    ┌────┴────┬─────────┬──────────┬──────────┐
    ▼         ▼         ▼          ▼          ▼
┌───────┐ ┌───────┐ ┌───────┐ ┌────────┐ ┌────────┐
│ Blink │ │ Mask  │ │ Phone │ │ Smoke  │ │  Hand  │
│Module │ │Module │ │Module │ │ Module │ │ Module │
└───┬───┘ └───┬───┘ └───┬───┘ └────┬───┘ └────┬───┘
    │         │         │          │          │
    └─────────┴─────────┴──────────┴──────────┘
                       │
                       ▼
              ┌────────────────┐
              │  Alert System  │
              │  - Cloudinary  │
              │  - Twilio      │
              └────────────────┘

🔧 Installation

Prerequisites

• Python 3.8 or higher
• Webcam (for live detection)
• Internet connection (for alerts)

Step 1: Clone the Repository

git clone https://github.com/yourusername/Driver_Monitoring_System.git
cd Driver_Monitoring_System

Step 2: Create Virtual Environment (Recommended)

# Windows
python -m venv venv
venv\Scripts\activate

# macOS/Linux
python3 -m venv venv
source venv/bin/activate

Step 3: Install Dependencies

pip install -r requirements.txt

Step 4: Download Model Files

Place the following model files in the models/ directory:

• best.pt - Face mask detection model
• yolov8n.pt - YOLO nano model for general object detection
• handdsa.pt - Hand detection model
• detection_module.pt - Smoking detection model (optional)

Step 5: Configure Alert System

Edit alert_system.py and add your credentials:

# Cloudinary Configuration
CLOUDINARY_NAME = "your_cloud_name"
CLOUDINARY_KEY = "your_api_key"
CLOUDINARY_SECRET = "your_api_secret"

# Twilio Configuration
TWILIO_SID = "your_account_sid"
TWILIO_TOKEN = "your_auth_token"
TWILIO_FROM_NUMBER = "whatsapp:+14155238886"

# Your WhatsApp Number
YOUR_WHATSAPP_NUMBER = "whatsapp:+1234567890"

🚀 Usage

Running the Application

1. Start the Flask Server

python app.py

2. Access the Web Interface Open your browser and navigate to:

http://localhost:5000

3. Choose Detection Mode
   • Live Analysis: Click "Start Camera" to begin real-time monitoring
   • Image Upload: Upload an image for analysis

Command Line Options

# Run on a specific port
python app.py --port 8080

# Run in debug mode
python app.py --debug

📁 Project Structure

Driver_Monitoring_System/
│
├── app.py                      # Main Flask application
├── requirements.txt            # Python dependencies
├── train.ipynb                 # Model training notebook
│
├── detectors/                  # Detection modules
│   └── __pycache__/
│
├── models/                     # YOLO model files
│   ├── best.pt                 # Mask detection model
│   ├── yolov8n.pt             # General YOLO model
│   ├── handdsa.pt             # Hand detection model
│   └── detection_module.pt    # Smoking detection model
│
├── templates/                  # HTML templates
│   └── index.html             # Main web interface
│
├── alert_system.py            # WhatsApp alert integration
├── blink_detection.py         # Eye blink/drowsiness detection
├── hand_detection.py          # Hand detection module
├── mask_detection.py          # Face mask detection
├── phone_detection.py         # Mobile phone detection
├── smoke_detection.py         # Smoking detection
└── utils.py                   # Utility functions (IoU, bbox operations)

🔍 Detection Modules

1. Blink Detection (blink_detection.py)

Uses MediaPipe Face Mesh to calculate Eye Aspect Ratio (EAR):

• EAR Threshold: 0.25 (indicates closed eyes)
• Drowsiness Threshold: 2.0 seconds of continuous eye closure
• Blink Counting: Tracks total blinks per session

Algorithm:

EAR = (|p2 - p6| + |p3 - p5|) / (2 * |p1 - p4|)

2. Mask Detection (mask_detection.py)

YOLO-based face mask classification:

• Detects: with_mask, without_mask, mask_weared_incorrect
• Hand-on-mouth overlap detection to avoid false negatives
• Configurable overlap threshold: 40%

3. Phone Detection (phone_detection.py)

Context-aware phone detection:

• Requires hand-phone overlap (5% threshold)
• Fallback to person-phone overlap (2% threshold)
• Minimum phone size filtering to reduce false positives
• Confidence threshold: 30%

4. Smoking Detection (smoke_detection.py)

Detects smoking behavior:

• YOLO-based cigarette/smoking detection
• Real-time visual feedback with bounding boxes
• Integrated with alert system

5. Hand Detection (hand_detection.py)

Hand position tracking:

• Used for context in phone and mask detection
• Detects hand-on-mouth scenarios
• Improves overall system accuracy

📢 Alert System

The system uses Twilio and Cloudinary for real-time alerts:

Features:

• Automated Snapshots: Captures frame when violation is detected
• Cloud Storage: Uploads images to Cloudinary for permanent storage
• WhatsApp Notifications: Sends instant alerts with image links
• Smart Cooldown: 60-second cooldown between alerts to prevent spam

Alert Triggers:

• 🚨 Drowsiness detected (eyes closed > 2 seconds)
• 📱 Phone usage while driving
• 🚬 Smoking behavior
• 😷 Mask violations (when applicable)

Setup Requirements:

1. Create a Cloudinary account
2. Create a Twilio account
3. Enable Twilio WhatsApp sandbox
4. Configure credentials in alert_system.py

⚙️ Configuration

Adjustable Parameters (app.py)

# Detection Thresholds
HAND_ON_MOUTH_THRESHOLD = 0.40      # Hand-mouth overlap ratio
EAR_BLINK_THRESHOLD = 0.25          # Eye closure threshold
DROWSY_SECONDS_THRESHOLD = 2.0      # Drowsiness time limit

# Alert Settings
ALERT_COOLDOWN_SECONDS = 60         # Cooldown between alerts
DETECT_HEAVY_EVERY_N = 3            # Process every Nth frame

# Phone Detection
PHONE_CONF = 0.30                   # Phone detection confidence
PHONE_IOU = 0.45                    # IoU threshold for NMS

🤖 Models

Required Models:

Model	Purpose	Size	Download
best.pt	Face mask detection	~6 MB	Custom trained
yolov8n.pt	General object detection	~6 MB	Ultralytics
handdsa.pt	Hand detection	~6 MB	Custom trained
detection_module.pt	Smoking detection	Variable	Custom trained

Training Your Own Models

See train.ipynb for training pipeline examples. The notebook includes:

• Data preprocessing
• Model training with YOLOv8
• Validation and testing
• Export for deployment

🤝 Contributing

Contributions are welcome! Here's how you can help:

1. Fork the repository
2. Create a feature branch

   git checkout -b feature/AmazingFeature

3. Commit your changes

   git commit -m 'Add some AmazingFeature'

4. Push to the branch

   git push origin feature/AmazingFeature

5. Open a Pull Request

Areas for Contribution:

• 🎯 Improve detection accuracy
• 🚀 Add new detection modules
• 🎨 Enhance UI/UX
• 📚 Improve documentation
• 🧪 Add unit tests
• 🌍 Add internationalization

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Ultralytics YOLOv8 - Object detection framework
• MediaPipe - Face mesh detection
• Flask - Web framework
• OpenCV - Computer vision library
• Twilio - WhatsApp API
• Cloudinary - Cloud image storage

🔮 Future Enhancements

• Multi-driver support
• Historical data analytics dashboard
• GPS integration for location tracking
• Voice alerts in addition to WhatsApp
• Mobile app for monitoring
• Advanced fatigue detection using head pose
• Integration with vehicle CAN bus
• Cloud deployment (AWS/GCP/Azure)
• Docker containerization
• API for third-party integrations

---

⭐ If you find this project useful, please consider giving it a star! ⭐