By successful, I mean made it out to the large world market or being massively funded. After looking online it seems that the most relevant research is the one done by Bach-y-Rita about the seeing with tongue in the 70's. Since then there were no significant breakthroughs. Is there a reason for that? Did someone prove that brain plasticity is limited?
3 Answers
Bach-y-Rita's Tactile Vision Substitution System (TVSS) project was initiated in 1963 and he has since been regarded as the founding father of sensory substitution. The concept of sensory substitution refers to the process of obtaining information about the world from a functional sensory system (e.g. touch) that would normally be obtained from a lost sensory system (e.g, vision in the blind).
Bach-y-Rita developed various devices, but his most cited one consisted of 400 vibrotactile solenoids mounted in the back of a dental chair. Camera images were converted into binary ("black and white") tactile representations onto the back of a subject ('seeing with the 'skin') (Bach-y-Rita 1969):
The results obtained with the TVSS were very promising: after sufficient practice (20 – 150 hours), blind subjects were able to perceive movement, recognize shapes and objects, and even identify faces. It also enabled the appreciation of more abstract concepts quite unfamiliar to the blind, such as shadow, depth, looming, zooming and parallax (Bach-y-Rita, 2003). A face projected onto the skin would have felt like something like this (photo of an oscilloscope image representing the voltage stimuli sent from the camera to the vibrotactile display).
However, as the first figure shows, the TVSS was large and bulky. Moreover, it was noisy and power hungry and simply not very useful outside the laboratory. Although various portable variants were constructed, they too suffered these limitations in various degrees:
Bach-y-Rita's work has been extended into electrotactile approaches and the company Wicab is currently marketing the BrainPort V100 device in Europe. It features a head-worn camera to scan the environment and translates these images onto a grid of electrodes that stimulate the tongue to elicit electrotactile sensations (much like a sip of champagne or a soda would do):
It uses electrodes instead of bulky vibrotactors which greatly reduces power consumption and this approach has made a portable application feasible. Electrodes, however, need a moist surface or very high voltages are necessary. A straightforward approach was to apply the electrode grid onto the tongue. The tongue is moist and has excellent resolution. The BrainPort currently features 400 electrodes and is able to provide graded voltages to present gray scale images onto the tongue.
The BrainPort has been marketed fairly recently in Europe (2013) and an FDA study in the US has just been finished. As far as I know they are busy with commercializing there as well.
Regarding your brain plasticity question: There is a wealth of information available on neuroplastic changes in the brain due to the use of the BrainPort or other sensory substitution approaches (VoiCE, EyeMusic etc etc). Basically it has been shown that the deafferented visual cortices are recruited for tactile information processing (e.g. Sadato, 2002), and hence, the brain actively adapts to the new source of information. So no, plastic changes are not the limiting factor.
Hence, the BrainPort (and other sensory substitution approaches like the SonicGuide etc.) is useful, portable, and not less importantly, the brain actively adapts in using the alternative visuo-tactile input by recruiting the visual cortex for processing visuotactile information. Why then haven't sensory substitution approaches conquered the world of visual rehabilitation of the blind"? There are a number of important downsides to sensory substitution devices (SSDs):
- SSDs substitute vision (by tactile or acoustic information), they do not restore it;
- The limited resolution of SSDs does not allow for detailed target analysis, i.e., discerning the difference between a fork and a knife will be very difficult;
- Instead, relying on well-established coping mechanisms and sticking to their usual routines may be far more effective in everyday life. For example, simply touching the fork or spoon will immediately tell them the difference, while scanning them with a SSD takes them ages (i.e., SSDs are not very practical);
- The restricted resolution of state-of-the-art SSDs also aids little in orientation and mobility (O&M). In other words, it does not help blind folks to go about. Instead, established methods such as the guide cane and guide dog are far more effective for O&M;
- Learning to use a SSD effectively takes a lot of time. Not many people are willing to undergo an arduous rehabilitation program;
- In case of the BrainPort - it is very visible to the outside world and not very discreet when a blind person is using it. It is known that 29% of consumers with disabilities abandon the use of assistive devices for reasons unrelated to device performance (Phillips and Zhao, 1993).
References
- Bach-y-Rita 1969. Nature 221:963-4
- Bach-y-Rita 2003. JHCI 15:285-95
- Philips & Zhao 1993. Assistive technology 5:36-45
- Sadato 2002. Neuroimage 16:389-400
-
1$\begingroup$ wow thanks Chris for this extensive overview of the current state of such devices! $\endgroup$– MehdiJan 23, 2015 at 6:15
Chris gave an excellent answer for visual-to-tactile sensory substitution devices. I will not argue about the extent to which sensory substitution is successful, but the most widespread sensory substitution system for the blind is nowadays The vOICe, a class of visual-to-auditory sensory substitution devices that is available globally. Its Android version has been installed over a quarter million times in the last few years, and there are a number of recent peer-reviewed scientific publications based on use of its Microsoft Windows version, covering various aspects of brain plasticity. Upcoming augmented reality glasses are likely to improve convenience of use. But yes, like Chris stated, learning to use a SSD effectively takes a lot of time.
-
$\begingroup$ Nice answer! +1 are you related to The vOICe? I focused my question pretty much on Bach-y-Rita's legacy given the question. I think the main downsides of BrainPort, except for discreteness, also apply to other SSDs. $\endgroup$– AliceD ♦Jan 24, 2015 at 10:15
-
2$\begingroup$ Thanks @Chris. Yes I'm related to The vOICe so I could be biased. However, The vOICe is far cheaper than the BrainPort device, offers a much higher resolution (e.g. 176 x 64 versus only 12 x 12 for the 144-electrode BrainPort), and it works with mass-market hardware. Oh, and you can speak and eat while using it. :-) On the other hand, the BrainPort should work better for people who are deaf-blind, and it may work better for tracking motion. $\endgroup$ Jan 24, 2015 at 14:32
-
-
1$\begingroup$ @Chris You asked the same question by e-mail last year. :-) Not this year, but there was an invited presentation on The vOICe at Asia-ARVO 2013 in New Delhi, India (PDF of slides). $\endgroup$ Jan 25, 2015 at 19:42
-
1$\begingroup$ The 176 x 64 hardware/software resolution is mentioned on the product pages for the Microsoft Windows and Android versions of The vOICe. Perceptual resolution with The vOICe in the form of "visual" acuity is discussed in the peer-reviewed PLOS ONE and Frontiers in Psychology papers. $\endgroup$ Feb 6, 2015 at 17:57
Tadoma is a method of communicating used by some deaf blind individuals. The system was developed prior to 1967 and predates TVSS, although there is no hardware involved. I think this video demonstrates the extent to which the approach is successful, and hence not limited by brain plasticity. Tadoma has fallen out of favor because it is difficult to learn and teach. There has been some research into vibrotactile hearing aids and there are some commercially available.
-
$\begingroup$ Tadoma is a non-electronic way of sensory substitution. Guide dogs and the guide cane pre-date the TVSS as well, obviously. The link deafness.about.com/cs/vibrotactile/f/vibrotactile.htm is quite obscure, as it links to dead websites, and weird unintelligible Chinese matter, unfortunately. $\endgroup$– AliceD ♦Feb 6, 2015 at 13:28