Converter of Indonesian sign language into text and voice, text and voice to sign language to build between inclusion vocasional school student and teacher

Andriana Andriana¹, Zulkarnain Zulkarnain², Olly Vertus², Sutisna Abdul Rahman², Ida Hamidah¹, Iwan Kustiawan¹, Mokhamad Syaom Barliana¹, Tutin Aryanti¹, Dedi Rohendi¹, Lala Septem Riza¹

1. ¹ Indonesia University of Education
2. ² Universitas Langlangbuana

AIP conference proceedings, 2023

The main objective of this research is to build a sign recognition model that allows processing and classification of the greeting language for Indonesian Sign Language communication and to develop it by designing Machine learning to train the recognition model from input sign language to voice and text or vice versa from voice and text to sign language with the help of the OPT8320 type time of flight camera sensor and infrared sensor. To achieve this goal, benchmark efforts are carried out using machine learning methods and algorithms on video processing techniques based on position, rotation of coordinates and finger conditions for conversion from sign language to text and voice and computational intelligence engineering methods to convert speech and text into sign language. The results of this study become digital learning media that can help communication between inclusive students and teachers in schools. In this research, 5 greeting words were tested in Indonesian. Success with a trial of 5 greetings.

DOI10.1063/5.0128916Copy