Translate, Then Rewrite, or Translate Straight into the Avatar’s Words? Signable English from Greek, French, Spanish and German for a Sign Language Avatar

Abstract

Some sign language avatars play a recorded sign for each word in their dictionary and fingerspell the other meaningful words, such as nouns, verbs and names, spelling them letter by letter with the hand alphabet. Input in another language must be translated before it is signed, and the translation can be fitted to the avatar’s small vocabulary in two ways, by translating as usual and rewording afterwards, or by letting a language model translate directly while it is restricted to the words that the avatar can sign. This paper compares the two designs on written sentences in Greek and French, translated into English for an International Sign avatar with several hundred signs, and applies the first design also to Spanish and German, with lists of the most frequent English words standing in for larger dictionaries. Translations by NLLB and EuroLLM needed as much fingerspelling as English originals. Replacing each missing word by a similar word with a sign removed more than half of it almost instantly, at some cost in meaning. Both restricted designs removed nearly all of it but, with the avatar’s own dictionary, kept only part of the meaning. Translating first and rewording afterwards kept at least as much meaning by COMET as direct restricted translation, needed less computing and produced fewer unrelated words, while direct translation stayed closer to the wording of the reference by chrF. Translating first is therefore the better design, and with a small dictionary, word replacement after translation is the practical choice.