By sensors we mean technologies that can measure movements or (sound) vibrations. This can be done in a variety of ways and with different types of hardware, creating opportunities for both home and professional applications. Especially in the healthcare sector and the research field, tools with sensors offer many opportunities. Moreover, when a game element is added, as in exergames, this can make treatments more effective and increase therapy compliance. People often become more motivated to perform their exercises carefully and completely. In addition, tools with sensors can collect valuable usage data. This helps healthcare professionals and researchers improve treatment methods and gain new scientific insights.
In this article we discuss different types of tools with sensors that have great potential for the research field and healthcare: products with speech technology, rehabilitation games & communication games.
One of our current projects is the DigiJuf, which we are developing in collaboration with the Center for Language and Speech Technology at Radboud University. Through Digi Juf, children are challenged to create a story together using speech technology. Children take turns speaking in sentences that are converted to text, eventually leading to a self-created story. In this way, children practice their reading and speaking skills in a fun and playful way, which also gives them a greater sense of language.
Another example of a game using speech technology is the app Fluency Friends by SAY IT Labs. This video game is aimed at children with speech impairments such as stuttering, who receive feedback on their speaking techniques through AI and speech recognition. In the story, you meet different characters to whom you have to talk, then learn how to improve your speech. For example, children learn about their speaking rate, pronunciation, breathing and speech rhythm.
People doing rehabilitation at home can do their exercises playfully while their progress is measured, thanks to gaming with sensors. Serious gaming not only makes rehabilitation more fun, it is also a good support in the recovery process. People are encouraged to complete their exercises to the best of their ability to advance in the game. Moreover, games with sensors also provide a wealth of information for researchers. By measuring movements, the researcher or healthcare professional gains accurate insight into how a person moves during (home) exercises: where is the weight placed in the feet? How is eye-hand coordination going? What is a bottleneck? This is important information for assessing recovery and adjusting the treatment regimen.
In 2013, in collaboration with the UMCG the Skating Game, an exercise video game that allows elderly people to train their balance at home to reduce the risk of falling. A Kinect 360 camera registers their movements and translates this into a virtual skating tour on the television screen. In this way, seniors work on their coordination and endurance in a playful way. By turning the exercises into a game, seniors feel more motivated and challenged to continue their treatment at home.
With PhD researcher at RuG Campus-Fryslan Lise Beumeler, we are working on a virtual reality game that helps ICU patients with their rehabilitation exercises, both at home and in the ICU. In the game, the patient is transported to a virtual, cozy living room using VR glasses. Here, the patient must solve a jigsaw puzzle, using the same hand and arm movements taught in rehabilitation therapy. An infrared camera on the VR goggles records all the movements, eliminating the need for other props. In addition, it is possible to play the game on four different levels, allowing even people who are still in the ICU and have low mobility to play the game. Thus, this game playfully offers patients important exercises in a pleasant environment.
We are currently working in collaboration with researchers at the University of Twente and Radboudumc on the study HEROES. This virtual balance game is being developed specifically for people who have suffered a stroke. This game is also adaptive and can be played at multiple levels, which is important for individual rehabilitation. Players stand on a raft that is carried by the current and occasionally bumps into something. The player must then restore the disturbed balance of the raft by making the appropriate movement themselves. These movements are registered by the Intel Realsense Processor’s sensor, which simultaneously displays the player’s movements on the television screen.
Tools with sensors are, in short, an accessible and valuable way to gain knowledge about recovery processes. In addition, this way patients are actively (and in a fun way) involved in research and improvement processes, which can lead to valuable new insights.
In the near future, Johan van der Meulen from 8D will provide several workshops on the role that tools with sensors can play in the research field and healthcare. Also interested in a guest lesson or workshop on this topic for a science event? Feel free to contact at.
At 8D, we support researchers in integrating a design approach into their work, from project design to creative product development and process supervision. This participatory methodologies help solve complex, human-centered issues and embed research results sustainably in society.
Maarten likes to think along!
Serious Games as a Research Tool Games as a Method Serious games can contribute to scientific research in many different ways and at many different stages. Think of collecting quantitative…
Participatory research can solve three bottlenecks that researchers face: data collection, effective knowledge transfer and dissemination. Potential forms are game-based research and design-based research. We are happy to inspire you with examples from our portfolio.