Face-to-face conversation systems for deaf people with speech to text

Review by Joe Stewart from Australia (Graphic and web designer, cochlear implantee with interest in disseminating info on hearing issues)

DH Chat is a mobile phone and/or tablet application for face-to-face (or nearby) communication between deaf and hearing people that runs on two Bluetooth linked devices. For a deaf person it enables speech to be converted to text, which is sent to the device he/she is using and then, as necessary, responded to. In circumstances where one party is Deaf and uses sign language and the other doesn't, communication via text is possible. It is available for both Android (4.2 and higher) and Apple iOS (version requirements unknown) platforms and is downloadable from the appropriate sites. Most of what follows in this post relates to my assessment of DH Chat on two Android devices.

Overall description
The Google Play's description for DH-Chat states that it has the potential to allow a deaf person to communicate with hearing people in a shop, a hospital, a bank, etc. To this I would add that the app can be useful in noisy environments and, is also applicable for communications between two deaf people. For DH Chat to work there needs to be two devices available with the DH-Chat application installed on both. Actually, there are two complementary versions of the DH Chat application, one named Deaf-Hearing Chat Device-H for the hearing person, the other, Deaf-Hearing Chat Device-D, for the deaf persons. In certain situations it may be more appropriate to use the Deaf-Hearing Chat Device-H on both devices.


 
Cost are: Android version, Deaf-Hearing chat device H, A$7.50 and for Deaf-Hearing chat device D, A$4.99. The author states that funds raised will be used for further development.
The main difference between the apps is that DH-Chat for device D doesn't feature a direct “Speak and Send” function. In my circumstances, being a deafened adult who retains verbal ability, it probably would have been better to use two instances of DH-Chat H and this arrangement is said to work just as well.

My appraisal
1. Setup DH-Chat
I installed the DH-Chat D on a 6.4-inch Xperia Z Ultra as this is my mobile phone/tablet for texting and data. Because I am profoundly deaf I can’t successfully use mobile voice calls. Next I acquired a budget priced 4-inch Zet T815 to install the DH-Chat H on. Both installations were relatively simple and after pairing the devices I had no trouble in getting speech-to-text working using either WiFi or a G3 connection. Overall, the system worked well and provided I got the hearing person to speak clearly and not too quickly, reliable text messages flowed between devices.

2. Offline speech to text -limitations
Where I had problems was getting the offline speech to text recognition to work on the Zet T815 device and consequently the “Speak and Send” function did not work on the Zet device when WiFi and G3 were turned off. One possible explanation I was able to find was that Google excluded from some lower cost Jellybean devices the feature of downloading offline language files due to hardware limitations. On the Xperia Z Ultra I used the voice input from the virtual keyboard and this worked offline; admittedly not as well as when online. It appears to me that the downloadable language files, required to make the offline speech to text work, do not have the same accuracy (a limited dictionary) as the online conversion process.

3. Online speech to text functions worked well  
When WiFi or G3 mobile networks were available (with sufficient signal strength) the speech to text function worked well; there was little delay and the conversion was reasonable accurate. The “Say and Send” and “Voice Input” features worked best for creating a natural flow of conversation and detected a normal pause between sentences. The alternative voice input i.e. using the microphone on the virtual keyboard, requires a “Tap to pause” and then a “Send” tap to transfer the text.
     
4. Testing in noise
I tested the system in noisy conditions, where I found it almost impossible to hear via my implant and DH-Chat system delivered text output that I could understand. 

5. Suggested additional features
(a) It would be of benefit if the conversation text could be saved to a text file and, shared with other applications.
(b) The displayed text is large and easily read, but perhaps some setting on text size would be appropriate; particularly on small screen devices.
(c) Maybe one application to cover both users (the deaf and hearing) may be more appropriate than two separate ones. Then, features appropriate to the particular user could be turned on or off.

Review by Jeannie Krull (Senior Assistive Technology Consultant with North Dakota Interagency Program for Assistive Technology)

Face-to-face communication for a person who is deaf with someone who does not sign, can be difficult. Interpreters are not always available. Writing doesn’t always work because it can be slow and not everyone has terrific penmanship. Texting is great sometimes, but it doesn’t work for everyone or when the service is down. There are two-way text communication devices specifically for this situation called the Ubiduo or Interpretype; however, these are expensive, not portable, and meant to be used while sitting at a table.
Enter a set of Android apps called “Deaf/Hearing Chat Device”. The purpose of this app system is to establish a running conversation in text form between a person who is deaf and does not speak with a person who can hear. This requires two Android devices (tablets or smartphones) and a Bluetooth connection. No Wi-Fi, 3G, or 4G required. The person who is deaf uses an Android device with the Deaf/Hearing Chat Device-Deaf App ($5.99), and they communicate by typing or Swyping. The person who they want to communicate with is given another Android device with the Deaf/Hearing Chat Device-Hearing App ($8.99), and they communicate by typing, Swyping or entering their text by voice with the built-in Google speech recognition engine.
We tried this at IPAT with an Asus tablet and a Samsung Galaxy Note 2, both running Android 4.1 (Jellybean) with Wi-Fi and 4G turned off. The Bluetooth took a minute to connect, but once connected, the conversation commenced with virtually no delay between transmissions. The voice recognition worked well even without Wi-Fi or 4G. When the transmission was sent to the phone, it vibrated to alert the user (does not work with tablets), which is really nice if communicating with someone who is “long-winded” and you doze off.
The best features, in my opinion, were not having to pass a device back and forth and being able to see the threaded conversation in real time. In addition, the font size of the app user interface was very large, even on small devices, which would be great for someone with low vision.

Review by William Nau from USA

Works great.
Excellent app for its intended purpose. My father who is 84 has been deaf for several years. Until I found this app I have been communicating either with paper and pencil or email. I have tried other chat apps but their font size were small and hard to read. This afternoon I spent 2 hours talking with my father.
I installed this app (H) on an old phone (android 2.2). I installed the "D" app on my father's tablet (Android 4.0). Using the voice input (voice to text) I would speak into phone and then press send to transmit the resulting text to my Dad's tablet. He found the font size easy to read. Since it uses Bluetooth it is much faster than sms or email. Also the large text input button was much easier for my 85 year old mother to find with her bad eyes. I installed this app on an old non-contract phone . I did not activate the phones and just use my father's WIFI. Internet connectivity is need for voice to text. For areas WIFI is not available 3G or 4G is need for voice to text. I will install the H app on my activated phone in the future.
The price is nothing to be able to take my Dad to dinner and "talk" to him. The developer of this app gave me outstanding customer service.

Review by Victor Ivanov from Russia

это просто фантастика! Использую систему, чтобы общаться с глухим семилетним ребенком. Результаты просто фантастические! Для примера, действия, которые мальчик узнавал по табличкам, но ни разу не употреблял сам, до использования комплекса (апрель’12): дай, на, бежит, стоит, сидит, спит, ест. Итого 7 глаголов, остальное показывал примитивными первобытными жестами. Слова его вобще не интересовали. По состоянию на октябрь’12 глаголы, которые ребенок знает и использует сейчас (пишет сам в своих сообщениях): бежит, стоит, сидит, ест, спит, пьет, кушает, лежит, ловит, бросает, смотрит, плачет, болит, толкает, тянет, смеется, летит, закрывает, открывает, ходит, плавает, едет, танцует, рисует ,залезает, падает, кормит, показывает, читает, пишет, приносит, уносит, гуляет, дает, берет, включает, выключает, ползет, прыгает, ищет, находит, прячет, играет, печатает, фотографирует, говорит, слушает, кричит, обнимает, целует, гладит, одевает, снимает, поднимает, моет, подметает, строит, болеет (58 глаголов!) + повелительная форма для многих глаголов, такие как принеси, возьми, дай, найди, покажи, смотри и т.д. Так и ребенку-то очень интересно с нами общаться, полностью преобразился мальчик.