Anupam Basu
Ph.D, FNAE, FAScT
Development of Technology for Differently-abled People - In collaboration with IIT Kharagpur
Technology for the People with Neuro-Motor Disorders and Speech Impairment
>> Sanyog
The aim of the project Sanyog is to develop a vernacular Augmentative and Alternate(AAC) system for the people with severe speech and neuro-motor disorders. This segment of the population can not use the keyboard and the mouse, thus being alienated from normal computer usage. This factor, coupled with their speech impairment impedes their communication.
Presently Sanyog supports three languages, e.g., Bengali, Hindi and English. In each language, there are three communication modes. These are,
Iconic mode
Alpha-numeric mode and
Stored message mode
In the iconic mode, a user can construct simple sentences by the selection of icons and can have them spoken out using a text to speech(TTS) system. It supports generation of simple sentences for different tense, aspect and modality from the same sequence of icons. This mode also produces an iconic form for the sentence formed. The iconic communication mode is primarily aimed at people with low language skills.
Fig. Iconic Communication Interface
However, for advanced users, the system provides the alphanumeric mode of communication. In this mode, the user can construct complex messages with the help of an on-screen soft alphanumeric keyboard with standard editing facilities. Moreover, the keyboard is supported by word prediction techniques and is speech enabled.
The selection of icons or the keys in the virtual keyboard can be achieved using special access mechanism, that includes auto-scan and special selection switches to alleviate the use of mouse and keyboard.
Fig. Alphanumeric Mode
In the stored message mode, a set of pre-stored messages is kept in the system. Each of these messages is accompanied by one/more images. The interface component shows these messages to the user and the user communicates by selecting one/more of these messages. This mode is also speech enabled.
Fig. Stored Message Mode
>> Aakash Bani
A Voice Output Communication Aid (VOCA), Aakash Bani, is an electrical device that assists people who are unable to use natural speech to express their needs and exchange information with other people during a conversation. The VOCAs store pre-recorded messages, which are produced in the form of digitized speech when the user presses an image. The most elaborate VOCAs include software that allows users to create and combine words to produce novel utterances in the form of computerized synthetic speech. SNLTR has recently developed a VOCA system for people with autism. This would definitely enable autistic persons to use tablets running on Android platform seamlessly.
Versions: The first edition of Aakash-Bani, so named as it is being launched with Aakash tablets, will have three versions to challenge various levels of disability. These applications should help persons with disability communicate better with those around them.
Touch Mode: A person can select a desired icon simply by touch. The popular touch screen phenomenon is used here.
Auto Scan mode: This has the touch facility through the auto-scan feature. This means that the user can touch any part of the screen and the highlighted icon will get selected.
Switch Mode: This version allows the user to communicate with two bright red switches. “By pressing the first switch, an icon gets selected (for instance, two girls crossing a road) and on pressing the second one its message (be careful while crossing) becomes audible.
Fig. Switch Mode & Auto Scan Mode
Key Features:
Can be customized as per the needs of the user.
No. of grids may be increased or decreased.
Grids can be modified, changed, edited.
Provision for sound recording in .mp3 format.
>> Sweepsticks
SweepSticks is an adaptive user interface for a special community of users who have severe neuro-motor impairments mainly caused by cerebral palsy. In particular, it is a virtual mouse to be used by this population of users since they do not have sufficient motor control to use a normal mouse. It is equipped with methods of scanning the desktop interface of the computer where two “sticks” sweep the screen. The user, through some special switches can control the sweep and thereby reach form every part of the screen to every other part. A prediction scheme has been incorporated into Sweepsticks which predicts action to minimize the overall navigation effort. The Sweepsticks adapt to the user’s preference.
>> Special Access Switch
All the above systems mentioned above are supported by Special Access Mechanism, which is required to make the systems usable by severely disabled persons. The mechanism consists of two components, namely the software component called scanning and the hardware component called the access switches. The scanning mechanism scans the items in the iconic interface sequentially in such a way that the icon selection effort is minimized. In access switch mechanism different special access switches, such as palm switch, foot switch and leap switches, are used by which the disabled users can use the system with less effort.
Technology for the Blind
>> Sparsha
Sparsha is a speech enabled bi-directional automatic Indian language text to Braille transliteration system. The system provides a unified framework for a large number of popular Indian languages. The salient features of the system are discussed as follows:
-
Automatic, forward and reverse transliteration from digital text to Braille and Braille to text.
-
Supports most of the drivers for embossing the Braille code produced on Braille embossers.
-
Editing facility of the generated Braille code.
-
Transliteration of Mathematical Symbols and Scientific notations to Nemeth Braille.
-
Speech enabled file editor with keyboard support.
-
Apart from English, other languages supported by Sparsha include Hindi, Bengali, Assamese, Marathi, Gujarati, Oriya, Telugu Kannada and Dzongkha.
>> Speech Enabled Keyboard
The speech enabled keyboard allows a sightless user to type texts through normal English QWERTY or Bengali “Baishakhi” keyboard and obtain speech feedback at the character, word or sentence level.
This system will enable sightless users write examinations without the intervention of “writers”.
>> MyMagic - an android app for people with low vision
People with low vision often face challenges in pursuing their education, employment and living independently. MyMagic, an android application, fosters people with low vision by enabling them to read textual information properly. The app magnifies text in real time so that individuals with a problem in reading small or blurry texts can read the text easily and conveniently.
>> DAISY format Book
SNLTR, along with IIT Kharagpur, with support from MHRD, is involved in developing Accessible Study Material for the Blind Students. SNLTR is specifically responsible for converting Bangla text books into the DAISY format that is accepted as the standard for the documents to be made accessible to the blind people.
On-going projects :
Web Browser for the Blind
Project sponsored by Ministry of Information Technology, Govt. of India, jointly with IIT Kharagpur. The objective of this project is to build a light-weight web browser with special features that can be easily accessible by the visually impaired people. The World Wide Web is a vast resource of information; it is an excellent medium of communication and has brought about many advantages in our lives.
