Using Google Chrome, access: https://overhear.app
by: Linpei Duan, Max Henry, and Karl Michel Koerich
One of the main issues with assistive software for visually-impaired users is that it cannot effectively communicate where and how web page components are placed on the screen. A regular sighted person can quickly identify objects and map them to certain functionalities; and what’s more, the layout of a website is an important part of sharing in the common experience of being online. For example, sighted users are familiar with the idea that a “closing button” is typically located on one of the top corners of the webpage. This shared convention is useful for website designers, who don’t have to build up every interface from scratch. In contrast, a visually impaired person cannot easily map these functionalities to locations.
To solve this problem, our system presents web pages as their own sonic rooms: 3D sonic environments in which each element of the page is "physically" situated. This allows users to explore web pages as if they were a series of connected chambers, navigating from one room to another through the use of aural cues just as many non-sighted users do in their everyday experience.
The system's engine you are about to use is built upon a simple demo website that contains 4 pages. Each page contains a rectangular button emitting a spatialized sound.
By moving the cursor, you will hear the sound change: by dragging the mouse towards the sound source, you will hear the sound "getting closer" and more centered as if you were approaching an object in real life. When the cursor meets and hovers over a button you are able to click, the system switches into a “focus mode”: plays a new distinct ticking sound.
When you click a button and a new page has been successfully loaded, the system plays a pleasant, high-pitched bell tone. Each new page will have new aural elements. You are invited to follow the aural cues to find the button that leads you to the next page. On the demo website, you may repeat this process until reaching the fourth and final page.
Note: The goal of this system is not simply to go from the first to the last page but to see if and how the sound environment helps you to get a sense of the layout of a page without using your eyes. In practice, our engine should help you to situate yourself on any website in a comfortable way that is both familiar and new. The sound elements should guide you while also being pleasing to the ears.
We welcome you to this new user experience, in which navigating feels like going from room to room instead of page to page.