Skip to content

Aid for blinds. This AI will describe the surrounding, it will tell who is in front of him (if that person is a known person to AI using Facial Recognition) and it will also help him to know what is written (Optical Character Recognition)

License

Notifications You must be signed in to change notification settings

jashan20/Vision-For-Blind

Repository files navigation

Title: God's Eye (AI based spectacles for Blind people) (Hardware Project)

  • This project was the winner of Ionathon 1.0 on global level

  • SIH finalist 2022

1. Description

photo_2022-03-26_03-15-10

Solution:

AI-based spectacles that will tell blind people about their surroundings in real-time.

Features:

  1. It will assist visually impaired person in navigating from one place to another.
  2. It will have OCR technology which will help the visually impaired person to read books and newspapers.
  3. The facial recognition module will help the visually impaired person to know the person sitting in front of him/her.

Working:

Methodology

Modules:

A) Module 1 (Voice Module) -

It uses GTTS library(Google Text to Speech) to convert string to voice and Playsound Library is used to play the voice returned by GTTS

B) Module 2 (Optical Character Recognition) -

It uses Tesseract library w, which takes opencv frame as input, recognizes text in it and return text as string.

C) Module 3 (Live-Environment Captioning) -

We have trained our own deep learning model. It works on a multimodal neural network that uses feature vectors obtained using both RNN and CNN, so consequently, for training, two inputs have to be taken. One is the image we need to describe, a feed to the CNN, and the second is the words in the text sequence produced till now as a sequence as the input to the RNN. This module takes OpenCV frame as input and returns a description of the frame.

D) Module 4 (Facial Recognition Module) -

It works on face_recognition that uses dlib's deep learning algorithm implementation to recognize the person in the image. It takes OpenCV frame as input and returns name as string.

2. Methodology:

Arnav-01

3. Live Links:

Youtube videos:

4. Instructions to run God's Eye ?

Step 1: Download the repository as zipped file
Step 2: Extract zip
Step 3: Install dependencies from requirements.txt
Step 4: Run main.py
Step 5: Web Cam will start working. We have 3 Modes, Initially Live-environment captioning module will work (Mode 1, press 1 to start this mode), enter 2 to start Facial Recognition mode and enter 3 to start Optical Character Recognition Mode (i.e enter 1, 2 or 3 to switch between modes)
Step 6: Enter ESC button to end

About

Aid for blinds. This AI will describe the surrounding, it will tell who is in front of him (if that person is a known person to AI using Facial Recognition) and it will also help him to know what is written (Optical Character Recognition)

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published