Interactive Sonification of Images in Serious Games as an Education Aid for Visually Impaired Children

Autor/inn/en	Radecki, Andrzej; Bujacz, Michal; Skulimowski, Piotr; Strumillo, Pawel
Titel	Interactive Sonification of Images in Serious Games as an Education Aid for Visually Impaired Children
Quelle	In: British Journal of Educational Technology, 51 (2020) 2, S.473-497 (25 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Radecki, Andrzej) ORCID (Bujacz, Michal) ORCID (Skulimowski, Piotr) ORCID (Strumillo, Pawel)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0007-1013
DOI	10.1111/bjet.12852
Schlagwörter	Children; Visual Impairments; Educational Games; Elementary School Students; Handheld Devices; Assistive Technology; Interaction; Nonverbal Communication; Geometric Concepts; Learning Processes; Audio Equipment; Color; Students with Disabilities + Suchen Sie Ihr Suchwort? Child; Kind; Kinder; Visual handicap; Sehbehinderung; Educational game; Lernspiel; Interaktion; Non-verbal communication; Nonverbale Kommunikation; Elementare Geometrie; Learning process; Lernprozess; Audio-CD; Colour; Farbbezeichnung; Farbe; Student; Students; Disability; Disabilities; Schüler; Schülerin; Studentin; Behinderung
Abstract	The paper presents an application for interactive sonification of images intended for use on mobile devices in education of blind children at elementary school level. The paper proposes novel sonification algorithms for converting colour and grayscale images into sound. The blind user can interactively explore image content through multi-touch gestures and select image sub-regions for sonification with real-time control of the synthesized sound parameters such as amplitude, frequency and timbre. Additionally, images may contain text fields read by text-to-speech synthesizer. The usability of one of the proposed sonification schemes is tested by collecting data on the tracking accuracy and recognition speed of basic shapes, such as lines, curves, as well as figures and simple functions. In order to facilitate the learning process for blind children, a number of games were proposed that use the devised sonification schemes. The first game--"Hear the Invisible"--is intended for two players--one child draws a shape and the task of the other one is to guess the displayed shape by means of the available sonification methods. The second proposed game is "Follow the Rabbit" and is intended for a single player who tracks a colourful "Rabbit" that runs along a path representing a given geometric shape. The obtained results show that the proposed sonification methods allow to reach previously unattainable levels of flexibility in exploration of shapes and colours. The main application of the described interactive tool is to teach geometry and mathematics in schools for blind children. The developed games are meant to enhance the learning process and motivate the children. (As Provided).
Anmerkungen	Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com/WileyCDA
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01