Transcribing sign languages with Typannot: a typographic system which retains and displays multiple levels of information. - Archive ouverte HAL
Conference Papers Year : 2020

Transcribing sign languages with Typannot: a typographic system which retains and displays multiple levels of information.

Transcrire les langues des signes avec Typannot: un système typographique avec plusieurs niveaux d'informations

Trascrivere le lingue dei segni con Typannot, un sistema tipografico che conserva più livelli di informazione

Abstract

Sign Languages (SL) are among the scarce languages which resist to be written. We will explain some of the reasons why it is so difficult to write them down and will expose the main solutions that have been developed to transcribe their key components, i.e., hand gestures. The two main transcription systems (HamNoSys [Prillwitz 1989] and SignWriting [Sutton 1995]) approach handshape representation with a fixed set of hand structures, in which orientation and location of the hands are defined in relation to the egocentric space (left, right, forward, backward, etc.). Other phonological approaches - e.g., Hand Tier Model (Sandler, 1989), Prosodic Model (Brentari, 1998), Hold-Movement Model (Johnson & Liddell, 2010) -, also deal with hand gestures using a relative perspective: orientation refers to the palm orientation according to different body planes (sagittal, etc.), location refers to the hand position in relation to the whole body, while movement refers to its course in space. This centralized perspective puts the focus on the hand and diminishes the role of other segments. This induces the idea that meaning is only carried by the hand and its trajectory/figure in space, while in fact it can also be carried by forearms, arms and shoulders. Although those approaches are very efficient at representing hand gestures from an external perspective, they disregard important informative dimensions by ignoring the internal perspective. We would like to promote another way to examine SL, in which the whole upper limb participates in expression of meaning. While the phonological description of handshape is relevant, its location and orientation are determined by the kinesiological system of the upper limb. This novel integration of the body parts configuration opens new possibilities to describe the active components of gestures and not just to show the final results (the hand behavior in space). This intrinsic point of view permits the study of gestures in new ways (i.e., investigating if and how meaning could originate from the structural various inner transformations [Boutet, 2015]) but presents new challenges on both conceptual and practical levels. How are the different levels of information organized? How can such complex phonological descriptions achieve readability? How can the linguist easily describe gestures from an intrinsic perspective that is experienced but not seen? How will the researcher transcribe several parameters at the same time? Typannot is a typographic system that structures phonological descriptions of SL signs into four distinct levels of information: the first level defines the parameters of the sign (i.e., handshape, initial localization, etc.); the second level describes the parts defining each parameter (i.e., fingers, thumb, etc.); the third level describes the features of the parts (i.e., form of the finger, angle of rotation, etc.); finally the fourth level shows the values of those features (i.e., curved, closed, etc.). Typannot allows both a linear and a multilinear organization of the information in transcription software like ELAN using a dedicated typographic system. The designs and functionalities of the system are built upon four principles: genericity, readability, modularity and inscriptibility. Genericity refers to the discretization of all four levels components (features levels in phonology) that structure SL. Each sign can be described by a sequence of generic characters (mapped in Unicode) using a predefined syntax to carry information consistently. This fundamental principle allows transcription to be analyzed and researched at various levels. Readability refers to the ease of reading conferred by a certain form of writing (McMonnies, 1999). While the generic form favours an analytical and fundamentally decomposed layout of components, the concept of readability requires to synthesize the string of information into a single, immediately recognizable, form. Visual analogy is used to design composed glyphs that can be viewed as a “logographic summary” of the parameter (i.e., handshape). The most important property of composed glyphs is that they retain all the pieces of information from their generic forms, preserving the analytic purpose of transcription a.k.a. data searchability. Users can switch between the two forms of reading using standard OpenType functionalities (i.e., ligature or contextual alternate). Modularity is a design principle which allows an automatic generation of any composed glyphs using glyphic modules and computer algorithms. Because of Unicode’s technical limit of 64 000 glyphs per font, Typannot’s repertoire of composed glyphs will start with a standard set, to be progressively extended by collecting “unmapped” combinations that users transcribe, making it a corpus-driven font system. Finally, the concept of inscriptibility refers to the writer’s ability to associate the transcription system (forms and functionalities) with the multiple performative dimensions of his/her language. The gestural-visual modality used by SL allows to consider a writing system based on corporal analogies as the mode of representation of components. It is worth reminding that, for SL, oral and written forms share the same gestural-visual modalities. The written word is not limited to conventional symbols disconnected from the dimensions of orality, as in the case of vocal language. To conclude, it is necessary to develop dedicated input systems in order to help users dealing with the multiple dimensions of SL signs, especially when they are seen from an intrinsic perspective. Two important ideas have to be kept in mind: first, the input systems must provide the necessary control over the complex parametric levels of information; second, they should visualize the effect of the parametric input to serve as an external visual feedback of the internal phonological encoding. Also, the current development of motion capture and recognition technologies has a deep impact on how transcription can be performed “directly” using the body. We will show the integration of such a possibility for inputting handshape phonological transcriptions and promoting the principle of inscriptibility. Bibliographic references Brentari, D. (1998). A prosodic model of sign language phonology. The MIT Press. Boutet, D. (2015). Conditions formelles d’une analyse de la négation gestuelle. Vestnik of Moscow State Linguistic University "Discourse as social practice: priorities and prospects", 6 (717): 116-129 Johnson, R.E., & Liddell, S.K. (2010). Toward a phonetic representation of signs: sequentiality and contrast. Sign Language Studies, 11(2), 241-274. https://doi.org/10.1353/sls.2010.0008 Johnston, T. (2008). Corpus linguistics and signed languages: no lemmata, no corpus. 3rd Workshop on the Representation and Processing of Sign Languages, 82-88. O. Crasborn, E. Efthimiou, T. Hanke, E.D. Thoutenhoofd, I Zwitserlood eds. McMonnies, C.W. (1999). Chart construction and letter legibility/readability. Ophthalmic and Physiological Optics, 19(6), 498-506. https://doi.org/10.1046/j.1475-1313.1999.00460.x Prillwitz, S., Leven, R., Zienert, H., Hanke, T., & Henning, J. (1989). Hamburg notation system for sign languages: an introductory guide. Signum Press, Hamburg. Sandler, W. (1989). Phonological representation of the sign: linearity and nonlinearity in American Sign Language. vol. 32. Walter de Gruyter, Dordrecht. Sutton, V. (1995). Lessons in SignWriting. DAC. La Jolla CA

Domains

Linguistics
4C C054 2020-presentazione Grapholinguistics_II Paris.pdf (23.4 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-02555881 , version 1 (26-08-2020)

Identifiers

  • HAL Id : hal-02555881 , version 1

Cite

Claire Danet, Dominique Boutet, Patrick Doan, Claudia S. Bianchini, Adrien Contesse, et al.. Transcribing sign languages with Typannot: a typographic system which retains and displays multiple levels of information.. G21C 2020 "Grapholinguistics in the 21st century", Université de Paris 3, Jun 2020, Paris, France. ⟨hal-02555881⟩
327 View
5 Download

Share

More