🤟 DeepSign — Real-Time Sign Language Translation

An individual university project. A real-time American Sign Language (ASL) recognition system built in Python using Convolutional Neural Networks and live webcam video processing.

---

🧠 How It Works

1. Data Collection — custom script captures hand gesture images via webcam
2. Data Augmentation — augments training data to improve generalisation
3. Model Training — trains multiple CNN architectures and selects the best
4. Ensemble Inference — combines predictions from multiple models in real time
5. Live Translation — processes webcam feed and overlays predicted sign letter

---

🏗️ ML Pipeline

Stage	File
Data collection	collect_training_data.py
Augmentation	data_augmentation.py
Dataset loading	dataset_loader.py
Model definition	model.py / enhanced_model.py
Training	train_cnn.py / train_enhanced.py
Fine-tuning	fine_tune_model.py
Ensemble training	ensemble_train.py
Live inference	ensemble_inference.py
Entry point	main.py

---

🤖 Models Used

• EfficientNet — primary high-accuracy model
• MobileNetV3 — lightweight, optimised for real-time performance
• ResNet — used in ensemble for robustness
• Ensemble — combines all three for final prediction

Training history and confusion matrices are included in the repo as .png files.

---

🛠️ Tech Stack

Python TensorFlow/Keras OpenCV NumPy CNN Transfer Learning

---

🚀 Getting Started

Prerequisites

• Python 3.9+
• Webcam

Install & Run

git clone https://github.com/Mololola/DeepSign_CNN_2.git
cd DeepSign_CNN_2

pip install tensorflow opencv-python numpy

python main.py

---

⚠️ Note

Model was trained and optimised to run on a standard laptop CPU — architecture choices reflect hardware constraints of the development environment.