A team of researchers at the University of Georgia, USA, has developed a backpack equipped with artificial intelligence (AI) that allows a visually-impaired person to walk alone without the assistance of a guide dog or a cane. This high-tech product also warns the user of possible obstacles to avoid with the use of simple vocal indications.
The backpack holds a smart AI system running on a laptop, and is fitted with OAK-D cameras (which, in addition to providing obstacle information, can also provide depth information) hidden in a vest and also in a waist pack. The cameras run Intel’s Movidius VPU and are programmed using the OpenVINO toolkit. The waist pack also holds batteries for the system. The AI system was trained to recognize objects a sighted pedestrian would see when walking around in a town or city, such as cars, bicycles, other pedestrians or even overhanging tree limbs.
The system was also trained to recognize typical terrain, such as sidewalks, grass, curbs and pavement—and also a host of road signs. The system can read the words and convert them to messages for the user. The system also has a GPS device and a receiver connected to a microphone so that the user can speak to the system. The user receives spoken messages from the system via Bluetooth earpiece.
In practice, a user puts on the backpack and fannypack and heads out into the real world (for up to eight hours). The user can choose to listen to a stream of comments that describe the immediate vicinity, and the system responds to questions. In either case, the system alerts the user to obstacles such as curbs, benches, potted plants and other people. It also gives them advance warning of upcoming crosswalks, and because of the depth information data, can alert the user to impending inclines and declines in the path ahead.
To achieve this, the mini camera is attached to a vest or fannypack and connected to the computer unit in the backpack. Analyzed in real-time, these images provide precise information on the depth of field and the various elements that make up the pedestrian’s environment.
