Samenvatting

Communication between healthcare professionals and deaf patients has been particularly challenging during the COVID-19
pandemic. We have explored the possibility to automatically translate phrases that are frequently used in the diagnosis and
treatment of hospital patients, in particular phrases related to COVID-19, from Dutch or English to Dutch Sign Language
(NGT). The prototype system we developed displays translations either by means of pre-recorded videos featuring a deaf
human signer (for a limited number of sentences) or by means of animations featuring a computer-generated signing avatar
(for a larger, though still restricted number of sentences). We evaluated the comprehensibility of the signing avatar, as compared
to the human signer. We found that, while individual signs are recognized correctly when signed by the avatar almost
as frequently as when signed by a human, sentence comprehension rates and clarity scores for the avatar are substantially
lower than for the human signer. We identify a number of concrete limitations of the JASigning avatar engine that underlies
our system. Namely, the engine currently does not offer sufficient control over mouth shapes, the relative speed and intensity
of signs in a sentence (prosody), and transitions between signs. These limitations need to be overcome in future work for the
engine to become usable in practice.