Our very own Cristiano Micheli, Andreas Forsland and Lucas Steuber were featured in the latest issue of MIND. MIND is an international collaboration among organizations in the Neurotech community. It serves as a well of knowledge exploring myriad aspects of Neurotechnology, hoping to educate, inspire, and instill wonder regarding recent discoveries and promising possibilities. And we are so honored to be featured.
I Have Something To Say
But I Can’t Tell You...
The majority of nations have successfully extended the average lifespan of their citizens. However, the quality of life in the extra years gained from this extension is often overlooked. While it is difficult to approach this from a medical perspective, the widespread adoption of current technologies of human-computer interaction such as portable devices, wearables, mixed reality products, and brain-computer interfaces (BCIs) have made it possible to complement the healthcare infrastructure and augment the quality of life of geriatric populations.
Primarily, these devices can improve their lived experience by facilitating person-to-person communication. This article presents some of those cutting-edge technologies available to support the elderly and those with limited communication abilities, followed by a few prominent use-cases for the aging population.
By Cris Micheli, Utkarsh Sarawgi, Lucas Steuber, Andreas Forsland
Technology can assist the elderly in a variety of ways by leveraging a diversified selection of access interfaces and assistive technologies that augment or compensate for deficits in their physical and cognitive abilities. These portable human-technology interfaces aid humans, without substituting or invading their bodies. This non-invasive approach is called ‘assistive technology’ as it gently assists human function similar to a walking stick providing stability to an elderly person.
As people age, communication becomes more and more challenging, eventually leading to social isolation. As such, all available instruments able to support or ‘augment’ communication will assume a very important role in the years to come for aging populations. These technologies are referred to as Augmentative Alternative Communication, or AAC. Specifically, AAC is described as “encompass[ing] the communication methods used to supplement or replace speech or writing for those with impairments in the production or comprehension of spoken or written language.”[1]
The most ubiquitous modern form of AAC is accessed via a touch screen – often on a commercially available device. Such devices allow for solutions that assist or augment the interaction of users with their surroundings, thereby solving communication roadblocks with their peers. In fact, a substantial percentage of the population (7.6% of adults in the US [2]) has some form of speech impediment. As a consequence, the market has shown lately a high adoption rate of portable devices with AAC technologies across all ages, including the geriatric segment.
In particular, there are an increasing number of apps on the market that use the built-in accessibility functions of mobile devices to produce synthesized or recorded language. These apps also employ specialized interfaces designed for specific target populations to provide access to communication (Figure 1). In addition to speaking aids, most portable devices can help communication by easing cognitive functioning. For example, by engaging the user’s attention and memory systems through gaming or providing dashboards that produce simple sentences, the cognitive workload of communication can be distributed and made easier.
Figure 1: A useful depiction of Augmentative Alternative technologies or AAC (from Elsahar et al 2019). Licensed under creative commons by 4.0
Due to recent innovations - like the consumer-grade touchscreen devices mentioned above - we can now design and create affordable high-tech solutions with the potential to exponentially improve the abilities of aging individuals affected by speech or general communication impediments. A handful of companies with clearly defined missions have taken upon this task of delivering consumer-level products that will help solve these challenges, forming a niche technological ecosystem described below.
A Brief and Incomplete Overview of the Tech Ecosystem
CTRL-Labs, originally based in New York, developed an armband to decode arm gestures. This assistive device could be used to convey intended movement or to decode a simple hand or arm gesture with a stipulated meaning. Facebook recently acquired CTRL-Labs for over 500 million dollars, making it part of the Facebook Reality Labs. The financial scale of this acquisition, among many others, demonstrates the consumer electronics industry’s commitment to transform what we currently call “accessibility” into simply another component of user “personalization”, where the product adapts to the consumer rather than the other way around.
Another company, Neuralink, adopts an invasive approach that substitutes the loss of function with technology. Their interface rests on top of the user’s head and transmits information wirelessly from tiny flexible electrode threads embedded in the brain. It is supposed to provide a near-to-the source interface to assist decoding movement or language intentions where the biological systems are no longer able to do so. As opposed to the earlier mentioned approaches, Neuralink advances an invasive framework by reaching into the human body, rather than having an external product that can be used as a ‘plug-and-play’ device.
Cognixion, a company based in Santa Barbara and Toronto, provides a speech-generating device as an assistive communication solution that interacts with users via multiple non-invasive access points. The app efficiently serves as a prosthetic to provide speech to non-verbal individuals through a user-friendly tablet or phone interface which generates grammatically accurate sentences from a fast access interface with the goal to increase their speed of communication. In technical terms, it seeks to enhance ‘word rate’, a measurement of the amount of words effectively communicated in one minute. The interface is completely customizable and supports eye-tracking, multiple keyboard layouts, and a range of gesture swipes corresponding to the desired messages. For example, a swipe or tap on the iPad screen could select the ‘I love you dad’ tile; alternatively, a blink during directed eye gaze could generate a spoken request for water. The company is also developing a cutting-edge brain-computer interface to augment daily-life conversational skills for people in need (Figure 2).
Figure 2: Three of the many interfaces available in Speakprose® (Cognixion’s AAC app): swipe or tap on an iPad and eye-tracking (right) and Augmented Reality (left). Among other features (not pictured): brain-machine interface. The app vocalizes the desired statement on behalf of the user from a customizable vocabulary
Another striking example of AAC technology is project AlterEgo from MIT Media Lab. It is a non-invasive and wearable real-time silent speech interface that helps people communicate in natural language without using their voice or externally observable movements. All the user has to do is articulate their words internally, and the peripheral neural interface records electrical activity using multiple surface electrodes around the speech articulators (mouth and throat areas), which are then decoded into speech. The feedback to the user is given through audio, via bone conduction, without disrupting the user’s usual auditory perception, and making the interface closed-loop. This feature is commonly adopted in rehabilitation protocols, and in this specific case it leverages neuroplasticity by providing feedback via the bone conduction device about the correctness of a silent speech word.
As opposed to brain-computer interfaces which record electrical activity from the brain, this wearable merely acquires intended speech signals, i.e. internal articulations which are inherently silent and unobtrusive. While a primary focus of this project is to help support communication for people with speech disorders in conditions like Amyotrophic Lateral Sclerosis (ALS) and Multiple Sclerosis (MS) among others, the system has the potential to seamlessly integrate humans and computers such that computing, The Internet, and AI would weave into our daily life as a “second self” and augment our cognition and abilities. This can thereby help facilitate unobtrusive and real-time access to personalized content and publicly available information for the elderly people and any other target population with similar needs, while also providing an AI assistant to help form intelligible speech for the affected. (Figure 3).
Figure 3: An elderly Multiple Sclerosis patient using an early prototype of AlterEgo
Many other companies have joined the effort to restore, augment, or supplement the ability to communicate for affected individuals. The use cases detail how technology can aid communication among the elderly.
Aging and Stroke
The use of portable devices is important among the aging population. Their general availability and accessible interface make them useful for a population with senses that often are naturally in decline. In some cases, this decline is abrupt, such as after a cerebrovascular insult (CVI), more commonly known as a stroke. One frequent consequence of stroke is aphasia, a structural disruption in the brain conducive to impaired language comprehension and/or production. Aphasia is a complex syndrome that presents itself with different symptoms and, while individuals with aphasia can benefit from AAC, their needs are often unique. Thus, a personalizable interface with images describing the items to be selected would support such recovering patients. For them, a picture is really worth a 1000 words. The ideal app would support recovering patients affected by stroke and aphasia by providing an effective neurofeedback tool; by viewing the image and accompanying verbal or textual stimulus, they quite literally are able to “recall” a forgotten word or phrase.
Aging and Mild Cognitive Impairment
AAC technology can also support individuals affected by age-related mild cognitive impairment. The symptoms of such conditions are memory loss as well as working memory impairment, which is the inability to retain a thought for a medium to long span of time.
Working memory impairment has a clear impact on communicative efficacy because it makes communication much more cognitively difficult. Assistive technologies can provide resources to help this by reducing the cognitive workload of communication and helping the elderly remember the sentences they want to communicate. For example, having graphic elements that reinforce the context of the discourse within a speech generating app facilitates the retainment of words in working memory and therefore promotes fluid communication.
In general, AAC interfaces can assist communication by lowering the cognitive workload related to word selection by visualizing, or otherwise facilitating the understanding of, both concepts and context relevant to conversation. Such solutions, paired with personalized vocabulary that learns from patients’ routines and the myriad factors that constitute their immediate context, can boost communication and give useful neurofeedback to the user, over time improving their cognitive and communicative abilities. The promise of such AAC technologies is to promote cognitive plasticity and reinstate weakened verbal abilities.
Aging and Hospitalization
Every so often, in a clinical environment, patients wake up in the Intensive Care Unit and find out that their ability to speak is temporarily impaired. Some individuals must receive invasive surgeries such as tracheostomy to restore respiration, and the lack of verbal communication affects the patients as well as their families.
Critical care is a dramatic example of AAC as a communication aid. Most often, aging individuals in assistive nursing conditions or under caretaking regimens experience a gradual decline in their ability to speak. In such cases, speech-generating devices can make a difference in communicating intentions and needs. Portable lightweight solutions with a set of contextual sentences can also make a difference in constrained clinical settings where visiting times are often limited (Figure 4).
COVID-19 Flipbook
Figure 4:Portable solutions as communication aids in critical care can make the difference for non-speaking patients (Speakprose ®, from Cognixion)
The previous examples cover use cases within wearable or lightweight portable assistive technologies. Speech assistive technologies can also compose emergency numbers on behalf of the customer. Another example of such technologies is the fall-detection device, an automatic dialling switch hanging from the neck of elderly people. The device contacts family members with an emergency preprogrammed call in instances of sudden falls. All these discussions lead towards a question that still remains unanswered– with such apparent diversity of options, how does this all come together for the user – and why would an industry develop dedicated solutions to generating such tools?
Why This Is Important
Let’s briefly review how assistive technologies affect the quality of life of elderly people. Due to recent technological innovations such as portable and wearable devices to aid communication by supporting different levels of functionality and human interaction, especially for non-speaking individuals: they provide neural feedback mechanisms. By providing visual or audio feedback to the elderly, they stimulate more communication and improve quality of life and interaction with other individuals:
They are lightweight, easily concealed, and cost-effective therapeutic solutions;
Some solutions allow the caretaker or the individual to personalize the vocabulary;
Multimodal access for items selection such as eye tracking, gestures, touchscreen displays, brain-computer interfaces, further improve customizability as the elderly can choose their preferred communication modality.
Where once a nonverbal or a minimally verbal individual was limited by the options presented to them, they now have personalized access to the ability to communicate, control their environment, and contribute in ways that were until recently very difficult, causing profound implications at the individual, familial and societal levels.
For the individual, it is not a significant recovery of cognitive function due to effective communication support and neurofeedback of such tech solutions is not unlikely. We, as tech contributors and developers, are motivated to imagine the improved relationship with family and friends, the feelings of integration or diminished isolation, and a better sense of agency and control over their own life. Importantly, such interfaces enable the elderly to communicate their life experiences as part of the human need to pass on their culture and experience to the next generations.
The impact of AAC on society is already – and will continue to be - profound. Among other benefits, assistive technologies help reduce the burden on staff and caretakers, thus reducing costs for palliative care and psychological therapies and reducing the possibility of misunderstandings and errors.
Every human being deserves to be given the opportunity to share their thoughts and experiences. Communication is not just a functional or transactional act; it is the means by which they expand the span of their well-being; it is a form of self-actualization. For many of us, this key facet of our identity will slowly slip away as we continue to age. Fortunately, we can prevent this with the right technology, allowing the oldest members of society to participate in it once again.
Notes: [1] https://en.wikipedia.org/wiki/Augmentative_and_alternative_communication, [2] https://www.nidcd.nih.gov/health/statistics/quick-statistics-voice-speech-language
References: Balandin and Morgan, 2001, Beukelman et al, 2009, Elsahar et al 2019, ter Hoorn et al 2016
This article was originally published here. Editors: Jwalin Joshi and Lillian Shallow, Cover: Chris Seo @ NeuroTechX Berkeley
To download the full feature click here.
Want to learn more about how technology and healthcare intertwine? Curious about how BCI works? Give us a follow on our Cognixion Facebook page here where each week we host a Live show of Whats NEW? Whats NEXT?