Visible Speech: analysis, synthesis and application in deaf education

Kupchinaus S.Ju., Gitlin V.B., Korobeinikova I.V.
Software Dpt. Udmurt State University,
Computers Dpt. Izhevsk State Technical University

Speech plays a great part in children's development. Deaf children don't have speech training and their verbal speech and intellect doesn't have necessary development. This problem is being solved not only by traditional methods, but also by use modern personal computers with their video- and audio-channels. One of directions is called artificial Visible Speech, that differs from natural visual products of human speech. Speaking about Visible Speech we mean not only special languages of deaf communication (sign languges, lip reading), but also artificial visible products of speech (animated pictures of hands or speaking man face) and results of recognition of sounds, phonems and other parts of speech. The following thing is taken as a principle of use of PC by teaching the deaf: lack of natural reverse connection through injured acoustic channel is compensated by reverse connetion through visual channel. It's real possibility the substitution informational scheme "speech-hearing-speech" to more branched one "speech-remaining hearing-sight-visual analysator-speech". Including PC in this scheme opens new and great possibilities for correctional work. We elaborated and use training programs of Russian Sign Language for work with deaf kids. We have chosen the direction of algorithmical transformations of basic images (parts of face and head, hands and body) with the aim of creating compact animation files. We too provide program modelling of speech signals with the aid of linear prediction and got speech tract profile for isolated sounds. We show the deaf kid what he has said by demonstrating the chain of recognized isolated symbols on the screen. He compared them with the model symbol or word and gets new attempts.