Aplikasi Web Pembelajaran BISINDO Interaktif dengan Deteksi Gestur Real-Time
DOI:
https://doi.org/10.47065/bulletincsr.v6i5.1305Keywords:
BISINDO; MediaPipe; Conv1D; BiLSTM; Interactive LearningAbstract
Indonesian Sign Language (BISINDO) is the primary language of the Deaf community in Indonesia; however, access to BISINDO learning remains limited, particularly in North Sulawesi. This research develops an interactive web-based BISINDO learning application, BISINDO.app, providing real-time feedback on users' sign gestures via webcam. The system uses MediaPipe Hand Landmarker to extract 21 hand landmarks, classified using low-parameter deep learning models: a one-dimensional Convolutional Neural Network (Conv1D) for static gestures and a Bidirectional Long Short-Term Memory (BiLSTM) network for dynamic gestures. All inference runs client-side via TensorFlow.js, so the application requires no additional installation. The application uses React, Node.js/Express, and SQLite, secured with JSON Web Token (JWT) authentication, with content validated together with North Sulawesi's Indonesian Sign Language Center (PUSBISINDO). Training results show accuracies on the validation set (not yet evaluated on an independent test set) of 97.64% (alphabet, 26 classes), 98.01% (static numbers, 11 classes), 87.69% (dynamic numbers, 10 classes), and 99.26% (basic words, 17 classes), with model weights ranging from 165 KB to 650 KB; evaluation on an independent test set is left for future work. A prediction stabilization algorithm using a 0.8 confidence threshold, three-tick consistency, and Exponential Moving Average smoothing produces stable, responsive feedback. Functional testing across 14 scenarios confirms all features work as specified, and a usability evaluation with 13 respondents yielded an average SUS score of 83.19 (Excellent). The main contribution of this research is the integration of real-time client-side gesture recognition with an interactive feedback mechanism within a single web-based BISINDO learning platform. This research is expected to improve BISINDO learning accessibility and support the Deaf community's social inclusion in Indonesia.
Downloads
References
Z. R. Jannah, “Perbedaan Bahasa Isyarat: BISINDO VS SIBI | Lembaga Bahasa Internasional FIB UI,” Lembaga Bahasa Internasional FIB UI. Accessed: Aug. 18, 2026. [Online]. Available: https://lbifib.ui.ac.id/id/blog/artikel/perbedaan-bahasa-isyarat-bisindo-vs-sibi
B. Asriandhini and C. H. Rahmawati, “Bahasa Isyarat Indonesia Sebagai Konstruksi Identitas Dan Citra Sosial Tuli Di Purwokerto,” JRK (Jurnal Riset Komunikasi), vol. 12, no. 1, 2021, doi: 10.31506/jrk.v12i1.10059.
Direktorat Jenderal Pendidikan Vokasi, “Jadi Alat Komunikasi Kaum Tunarungu, Yuk Mengenal Bahasa Isyarat di Indonesia | Direktorat Jenderal Pendidikan Vokasi PKPLK Kemendikdasmen,” Kementerian Pendidikan Dasar dan Menengah. Accessed: Aug. 18, 2026. [Online]. Available: https://vokasi.kemendikdasmen.go.id/read/b/jadi-alat-komunikasi-kaum-tunarungu-yuk-mengenal-bahasa-isyarat-di-indonesia
R. I. Borman and B. Priyopradono, “Implementasi Penerjemah Bahasa Isyarat Pada Bahasa Isyarat Indonesia (BISINDO) Dengan Metode Principal Component Analysis (PCA),” Jurnal Informatika: Jurnal Pengembangan IT, vol. 3, no. 1, 2018, doi: 10.30591/jpit.v3i1.631.
W. Ahmad priadiyatna, H. Hudiono, and A. Rasyid, “Rancang Bangun Sarung Tangan Pintar Penerjemah Bahasa Isyarat Indonesia (Bisindo) Berbasis Iot,” Jurnal Jartel: Jurnal Jaringan Telekomunikasi, vol. 10, no. 4, 2020, doi: 10.33795/jartel.v10i4.21.
M. Susanty, R. Z. Fadillah, and A. Irawan, “Model Penerjemah Bahasa Isyarat Indonesia (BISINDO) Menggunakan Pendekatan Transfer Learning,” PETIR, vol. 15, no. 1, pp. 1–9, Dec. 2021, doi: 10.33322/PETIR.V15I1.1289.
L. Arisandi and B. Satya, “Sistem Klarifikasi Bahasa Isyarat Indonesia (Bisindo) Dengan Menggunakan Algoritma Convolutional Neural Network,” Jurnal Sistem Cerdas, vol. 5, no. 3, 2022, doi: 10.37396/jsc.v5i3.262.
A. Bayu Pangestu, M. Rafi Muttaqin, and M. Agus Sunandar, “Sistem Deteksi Bahasa Isyarat Indonesia (Bisindo) Menggunakan Algoritma You Only Look Once (YOLO)v8,” JATI (Jurnal Mahasiswa Teknik Informatika), vol. 8, no. 5, 2024, doi: 10.36040/jati.v8i5.10833.
I. G. A. O. Aryananda and F. Samopa, “Comparison of the Accuracy of The Bahasa Isyarat Indonesia (BISINDO) Detection System Using CNN and RNN Algorithm for Implementation on Android,” MALCOM: Indonesian Journal of Machine Learning and Computer Science, vol. 4, no. 3, 2024, doi: 10.57152/malcom.v4i3.1465.
A. Aljabar and Suharjito, “BISINDO (Bahasa isyarat indonesia) sign language recognition using CNN and LSTM,” Advances in Science, Technology and Engineering Systems, vol. 5, no. 5, 2020, doi: 10.25046/AJ050535.
A. F. Deleviar, I. Oktaviani, and H. Permatasari, “Pengembangan Website Speech To Video Bahasa Isyarat Indonesia (Bisindo) Berbasis Algoritma Long Shot Term Memory,” Infotek: Jurnal Informatika dan Teknologi, vol. 8, no. 1, 2025, doi: 10.29408/jit.v8i1.26117.
K. L. Wiguna and Rojali, “Sentence-Level Indonesian Sign Language (BISINDO) Recognition Using 3D CNN-LSTM and 3D CNN-BiLSTM Models,” International Journal of Advanced Computer Science and Applications, vol. 16, no. 9, 2025, doi: 10.14569/IJACSA.2025.0160941.
A. K. Fadzli and M. Rahardi, “Comparative Analysis of MobileNetV3 and EfficientNetv2B0 in BISINDO Hand Sign Recognition Using MediaPipe Landmarks,” Journal of Applied Informatics and Computing, vol. 10, no. 1, pp. 737–746, Feb. 2026, doi: 10.30871/JAIC.V10I1.11878.
I. Rizka Fadhillah, M. Muharrom Al Haromainy, and H. Maulana, “Implementasi Model Transfer Learning Efficientnet Untuk Pendeteksian Bahasa Isyarat Indonesia (BISINDO) Pada Perangkat Android,” JATI (Jurnal Mahasiswa Teknik Informatika), vol. 8, no. 4, 2024, doi: 10.36040/jati.v8i4.10463.
N. Renaningtias, F. Putra Utama, and A. N. A. Sobri, “Detection System Indonesian Sign Language (BISINDO) in Video with YOLOv7,” JSAI (Journal Scientific and Applied Informatics), vol. 8, no. 1, pp. 1–8, Jan. 2025, doi: 10.36085/JSAI.V8I1.7067.
T. M. A. Afwan, R. Gernowo, and H. A. Wibawa, “Deep Learning-Based Recognition of Indonesian Sign Language (BISINDO) Alphabetic Gestures Using Skeletal Feature Extraction and LSTM,” Jurnal Teknik Informatika (Jutif), vol. 7, no. 2, pp. 903–925, Apr. 2026, doi: 10.52436/1.JUTIF.2026.7.2.5337.
S. A. Nurpiena, E. S. Wihidayat, and A. Budianto, “Developing Indonesia Sign Language (BISINDO) Application with Android Based for Learning Sign Language,” Journal of Informatics and Vocational Education, vol. 4, no. 1, pp. 1–11, Feb. 2021, doi: 10.20961/JOIVE.V4I1.48629.
R. Setiawan, Y. Yunita, F. F. Rahman, and H. Fahmi, “BISINDO (Bahasa Isyarat Indonesia) Sign Language Recognition Using Deep Learning,” IT for Society, vol. 9, no. 1, Mar. 2024, doi: 10.33021/ITFS.V9I1.5076.
F. I. Hadinata and S. A. Sanjaya, “BISINDO Sign Language Recognition: A Systematic Literature Review of Deep Learning Techniques for Image Processing,” The Indonesian Journal of Computer Science, vol. 12, no. 6, p. 3281, Dec. 2023, doi: 10.33022/IJCS.V12I6.3539.
G. O. Kindy, G. Leonali, and H. Lucky, “Word-Level BISINDO: A Novel Video Indonesian Sign Language Dataset and Baseline Methods,” Procedia Comput. Sci., vol. 269, pp. 249–258, Jan. 2025, doi: 10.1016/J.PROCS.2025.08.277.
C. Sebastian, J. Limanza, L. Laurentia, J. Harefa, and K. Jingga, “Indonesian Sign Language (BISINDO) Recognition Using Spatially Aware Body Gesture Recognition,” Procedia Comput. Sci., vol. 269, pp. 1002–1011, Jan. 2025, doi: 10.1016/J.PROCS.2025.09.042.
A. Vakunov, C.-L. Chang, F. Zhang, G. Sung, M. Grundmann, and V. Bazarevsky, “MediaPipe Hands: On-device Real-time Hand Tracking,” 2020. Accessed: Aug. 18, 2026. [Online]. Available: https://research.google/pubs/mediapipe-hands-on-device-real-time-hand-tracking/
I. D. A. Rachmawati, R. Yunanda, M. F. Hidayat, and P. Wicaksono, “Deep Transfer Learning for Sign Language Image Classification: A Bisindo Dataset Study,” Engineering, MAthematics and Computer Science Journal (EMACS), vol. 5, no. 3, 2023, doi: 10.21512/emacsjournal.v5i3.10621.
A. Almjally, S. A. Algamdi, N. Aljohani, and M. K. Nour, “Harnessing attention-driven hybrid deep learning with combined feature representation for precise sign language recognition to aid deaf and speech-impaired people,” Sci. Rep., vol. 15, no. 1, 2025, doi: 10.1038/s41598-025-15109-2.
A. Baihan, A. I. Alutaibi, M. Alshehri, and S. K. Sharma, “Sign language recognition using modified deep learning network and hybrid optimization: a hybrid optimizer (HO) based optimized CNNSa-LSTM approach,” Sci. Rep., vol. 14, no. 1, 2024, doi: 10.1038/s41598-024-76174-7.
A. Yuan and A. Vakunov, “Face and hand tracking in the browser with MediaPipe and TensorFlow.js The TensorFlow Blog,” The TensorFlow Blog. Accessed: Aug. 18, 2026. [Online]. Available: https://blog.tensorflow.org/2020/03/face-and-hand-tracking-in-browser-with-mediapipe-and-tensorflowjs.html
Google AI Edge, “Gesture recognition task guide | Google AI Edge | Google for Developers,” Google for Developers. Accessed: Aug. 18, 2026. [Online]. Available: https://developers.google.com/edge/mediapipe/solutions/vision/gesture_recognizer
A. Desai et al., “ASL Citizen: A Community-Sourced Dataset for Advancing Isolated Sign Language Recognition,” Adv. Neural Inf. Process. Syst., vol. 36, pp. 76893–76907, Dec. 2023, doi: 10.52202/075280-3360.
R. Z. Fadillah, A. Irawan, and M. Susanty, “Data Augmentasi Untuk Mengatasi Keterbatasan Data Pada Model Penerjemah Bahasa Isyarat Indonesia (BISINDO),” Jurnal Informatika, vol. 8, no. 2, pp. 208–214, Sep. 2021, doi: 10.31294/JI.V8I2.10768.
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016.
J. Brooke, “SUS: A quick and dirty usability scale,” Usability Eval. Ind., vol. 189, pp. 207–212, Nov. 1996.
A. Bangor, P. Kortum, and J. Miller, “Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale,” J. Usability Stud., vol. 4, pp. 114–123, Apr. 2009.
E. L. Kelana, M. R. Anshori Prasetya, Mambang, and M. Zulfadhilah, “Integrating the CNN Model with the Web for Indonesian Sign Language (BISINDO) Recognition,” Journal of Applied Informatics and Computing, vol. 9, no. 3, 2025, doi: 10.30871/jaic.v9i3.9345.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Aplikasi Web Pembelajaran BISINDO Interaktif dengan Deteksi Gestur Real-Time
ARTICLE HISTORY
How to Cite
Issue
Section
Copyright (c) 2026 Surya Adjie Umar Tenda, Aryanzah Nugrah Maggi, Vicrena Yolita Vanesa Kalangit, Toban T. Pairunan, Anritsu Steven Christian Polii
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
