🧩 Sudoku AI: Advanced Computer Vision Solver

A comprehensive AI-powered Sudoku solver that combines computer vision, deep learning, and intelligent error correction to process puzzle images and provide interactive solving assistance.

🌟 What Makes This Special

This isn't just another Sudoku solver—it's a complete AI system with several key innovations:

🔄 Multi-Prediction Error Correction: Tracks multiple possible values per cell and automatically fixes recognition errors
🎯 Smart Auto-Correction: Identifies conflicting cells and tries alternative digit combinations
🎨 Synthetic Dataset Generation: Creates unlimited realistic training data with fonts, handwriting, and backgrounds
💡 Interactive Solving: Provides intelligent hints, manual correction, and step-by-step guidance
📊 Comprehensive Evaluation: Complete model training, comparison, and performance analysis framework

✨ Core Features

🤖 Advanced AI Recognition

98-99% Accuracy on digit recognition and cell classification
Multi-Architecture Models: Simple CNN, deeper CNN with BatchNorm, ResNet-like with residual connections
Dual Classification: Recognizes both digits (0-9) and cell types (empty/printed/handwritten)
Confidence Scoring: Tracks prediction reliability for error detection

🔍 Robust Computer Vision

Smart Board Detection: Automatically finds Sudoku grids in photos using contour analysis
Perspective Correction: Handles tilted, rotated, and angled puzzle images
Adaptive Preprocessing: Gaussian blur, adaptive thresholding, morphological operations
Cell Extraction: Precise 9×9 grid segmentation with configurable parameters

⚡ Intelligent Problem Solving

Multiple Prediction Tracking: Keeps top-N predictions per cell for error recovery
Automatic Validation: Real-time Sudoku rule checking with conflict detection
Smart Auto-Correction: Tries alternative predictions to fix invalid boards
Manual Override System: User-friendly interface for correcting recognition errors
Context-Aware Hints: Difficulty-graded solving suggestions

🎨 Synthetic Data Generation

Realistic Rendering: Multiple fonts, handwriting styles, backgrounds, and augmentations
Configurable Difficulty: Easy (clean), medium (distorted), hard (heavily augmented)
Multiple Modes: Printed, handwritten, and mixed puzzles
Export Pipeline: Generates labeled datasets for training and evaluation

🌐 Interactive Web Interface

Streamlit Application: Clean, responsive design with real-time feedback
Step-by-Step Guidance: From image upload to final solution
Visual Problem Solving: Color-coded boards, progress tracking, hint visualization
Manual Correction Tools: Interactive cell editing with immediate validation

🚀 Quick Start

1. Installation & Setup

git clone https://github.com/yourusername/sudoku-ai.git
cd sudoku-ai
python -m venv sudoku-env
source sudoku-env/bin/activate  # Windows: sudoku-env\Scripts\activate
pip install -r requirements.txt

2. Train Models (First Time Setup)

python notebooks/03_model_training.py  # Trains digit recognition & cell classification models

3. Launch Web Application

streamlit run sudoku_app.py

Navigate to http://localhost:8501 and start solving!

4. Using the Interface

📁 Upload a clear Sudoku puzzle image
🔍 Process - AI detects board and recognizes digits
🎯 Solve - Get hints, apply corrections, or view complete solution
✏️ Fix - Manually correct any recognition errors
🎉 Complete - Step through the solution or solve instantly

🎯 How It Works

Computer Vision Pipeline

Image Input → Board Detection → Perspective Correction → Cell Extraction → Digit Recognition → Board Validation

Detection: Finds Sudoku grid using contour analysis and corner detection
Correction: Applies perspective transform to straighten the board
Extraction: Segments into 81 individual cells with preprocessing
Recognition: Dual-model classification for digits and cell types
Validation: Checks Sudoku rules and identifies conflicts

AI Recognition System

Digit Model: Classifies 0-9 with confidence scoring
Cell Type Model: Identifies empty/printed/handwritten cells
Multi-Prediction: Stores top-N alternatives for error correction
Auto-Correction: Uses alternative predictions to fix invalid boards

Intelligent Solving

Hint Generation: Finds cells with minimal valid options (MRV heuristic)
Difficulty Grading: Easy (single option), medium (few options), hard (complex logic)
Manual Interface: Allows user corrections with real-time validation
Solution Visualization: Color-codes original vs. solved cells

📁 Project Architecture

sudoku-ai/
├── 🌐 sudoku_app.py                 # Streamlit web application
├── 📂 src/                          # Core Python package
│   ├── 🔍 processing/               # Computer vision & solving
│   │   ├── board_detection.py       # Board detection and extraction
│   │   ├── data_processor.py        # Dataset loading and preprocessing  
│   │   ├── solver.py                # Basic Sudoku solving logic
│   │   ├── improved_solver.py       # Advanced multi-prediction solver
│   │   └── interactive_solver.py    # Web interface backend
│   ├── 🎨 generation/               # Synthetic data generation
│   │   ├── image_generator.py       # Main image generation orchestrator
│   │   ├── providers.py             # Font and MNIST digit providers
│   │   ├── puzzle_generator.py      # Sudoku puzzle generation
│   │   ├── renderers.py             # Digit, background, and grid rendering
│   │   └── dataset_generator.py     # Complete dataset creation pipeline
│   ├── 🧠 modeling/                 # Machine learning models
│   │   ├── models.py                # CNN architectures (simple, deeper, ResNet-like)
│   │   ├── evaluation.py            # Training and evaluation utilities
│   │   └── experiment.py            # End-to-end experiment management
│   ├── ⚙️ config/                   # Configuration
│   │   └── generator_config.py      # Generation parameters
│   └── 🛠️ utils.py                 # Utility functions
├── 📓 notebooks/                    # Demonstration notebooks (.py format)
│   ├── 01_board_detection.py       # Computer vision examples
│   ├── 02_dataset_generation.py    # Synthetic data creation
│   ├── 03_model_training.py        # Training pipeline
│   ├── 04_advanced_solving.py      # Multi-prediction solver demos
│   └── 05_interactive_solver.py    # Interactive interface examples
├── 🧠 models/                       # Saved neural networks
├── 📁 data/                         # Datasets and images
├── 📋 requirements.txt              # Dependencies
└── 📖 README.md                     # This file

🎮 Development Usage

Synthetic Dataset Generation

from src.generation.dataset_generator import generate_sample_dataset

# Create training data with specific parameters
generator, samples = generate_sample_dataset(
    num_samples=1000,
    modes=['printed', 'handwritten', 'mixed'],
    difficulties=['easy', 'medium', 'hard'],
    background_styles=['paper_color', 'texture', 'unified']
)

Advanced Processing

from src.processing.improved_solver import ImprovedSudokuProcessor

# Process with multi-prediction error correction
processor = ImprovedSudokuProcessor(top_n=3, confidence_threshold=0.5)
board = processor.process_image('sudoku.jpg', auto_correct=True)

if board.is_valid:
    processor.solve_board(board)
    print("Solution found!")
else:
    # Auto-correct failed, try manual correction
    corrected = board.auto_correct_board(max_iterations=5)

Interactive Solving

from src.processing.interactive_solver import InteractiveSudokuSolver

solver = InteractiveSudokuSolver()
solver.process_image('puzzle.jpg')

# Get intelligent hints
hint = solver.suggest_hint()
print(f"Hint: Place {hint.value} at ({hint.row+1}, {hint.col+1})")

# Apply hint and continue
solver.apply_hint_and_next()

Model Training

from src.modeling.experiment import SudokuExperiment

# Complete training pipeline
experiment = SudokuExperiment()
experiment.setup_dataset('my_dataset')
experiment.prepare_data()

# Train and compare multiple architectures
experiment.build_models()
digit_models = experiment.train_digit_models(epochs=20)
cell_models = experiment.train_cell_type_models(epochs=15)

# Evaluate performance
comparison = experiment.evaluate_digit_models()
print(comparison)

📊 Performance Results

Our AI models achieve state-of-the-art performance:

Model Accuracy (on test sets)

Model Type	Architecture	Accuracy	F1 Score
Digit Recognition	Deeper CNN	98.8%	98.3%
Digit Recognition	ResNet-like	94.9%	92.6%
Cell Classification	Deeper CNN	99.7%	99.7%
Cell Classification	Simple CNN	98.4%	98.1%

End-to-End Performance

Board Detection Success: >95% on clear images
Complete Solving Success: >90% with auto-correction
Processing Speed: <3 seconds per image on Apple Silicon
Error Recovery: 80%+ of invalid boards corrected automatically

🔧 System Requirements

Minimum Requirements

Python 3.8+
4GB RAM
2GB disk space for models and datasets
Camera or image files of Sudoku puzzles

Recommended Setup

Apple Silicon Mac (M1/M2/M3/M4) with Metal GPU acceleration
8GB+ RAM for large dataset processing
Good quality camera or high-resolution puzzle images

Dependencies

Streamlit: Interactive web interface
TensorFlow: Deep learning framework (with Metal acceleration on Apple Silicon)
OpenCV: Computer vision operations
NumPy/Matplotlib: Data processing and visualization
Pillow: Image processing
scikit-learn: Evaluation utilities

💡 Usage Tips

📸 Best Images for Recognition

✅ Ideal Conditions:

High contrast between digits and background
Straight-on camera angle (minimal perspective distortion)
Even lighting without shadows or glare
Sharp focus on the puzzle grid
Complete grid visible in frame

⚠️ Challenging Conditions (but often still work):

Moderate rotation or perspective distortion
Mixed printed and handwritten digits
Slight shadows or uneven lighting
Lower resolution images

❌ Avoid:

Extreme angles or heavy perspective distortion
Severe blurriness or out-of-focus images
Heavy shadows obscuring digits
Reflections or glare on the puzzle surface

🎯 Solving Strategies

Start with automatic processing - let the AI handle everything first
Check board validity - if invalid, examine highlighted problematic cells
Use manual correction for any obvious recognition errors
Get hints for learning - understand solving logic step-by-step
View complete solution when ready or stuck

🤝 Contributing

We welcome contributions! Here's how to get started:

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Development Areas

New model architectures for improved accuracy
Additional augmentation techniques for robustness
Mobile app development using the core pipeline
Real-time video processing for live solving
Multi-language digit support for international puzzles

📜 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

TensorFlow Team for the excellent ML framework
OpenCV Community for computer vision tools
Streamlit Team for the intuitive web framework
Google Fonts for typography resources
MNIST Dataset for handwritten digit training data

📞 Support & Contact

Having Issues?

🐛 Report a Bug
💡 Request a Feature
📖 Check the notebooks for detailed examples
📚 Review the code documentation

Quick Troubleshooting:

Models not found: Run the training notebook first
Board detection fails: Ensure good image quality and proper Sudoku format
Low accuracy: Try generating more training data with diverse conditions
Performance issues: Enable Metal acceleration on Apple Silicon

🎯 Bringing AI to Classic Puzzles

⭐ Star this repo • 🔄 Fork it • 📢 Share it

Made with ❤️ and lots of ☕ by passionate AI developers

Name		Name	Last commit message	Last commit date
Latest commit History 9 Commits
.streamlit		.streamlit
.vscode		.vscode
models		models
models_big		models_big
notebooks copy		notebooks copy
notebooks		notebooks
src		src
.DS_Store		.DS_Store
.gitignore		.gitignore
README.md		README.md
requirements.txt		requirements.txt
sudoku_app.py		sudoku_app.py
test1.jpg		test1.jpg
test2.jpg		test2.jpg

cwirex/sudoku-ai

Folders and files

Latest commit

History

Repository files navigation