A Game-Based Eye Tracking Approach To Measuring Interaction Levels In Children With Autism

Anfazul Faridatul Azizah; Arif Djunaidy; Daniel Siahaan; Fendy Suhariadi

doi:10.29284/2zah5d20

Authors

Anfazul Faridatul Azizah Department of Informatics, Institut Teknologi Sepuluh Nopember, Surabaya, Indonesia
Arif Djunaidy Department of Information Systems, Institut Teknologi Sepuluh Nopember, Surabaya, Indonesia
Daniel Siahaan Department of Informatics, Institut Teknologi Sepuluh Nopember, Surabaya, Indonesia
Fendy Suhariadi Doctoral Program in Human Resources Development – Postgraduate School, Universitas Airlangga, Surabaya, Indonesia

DOI:

https://doi.org/10.29284/2zah5d20

Keywords:

Autism, Eye Pupils, Games, Interaction.

Abstract

This study aims to analyze autism games have been used to facilitate the development of the interaction of children with autism, particularly with regard to emotions. However, these games are unable to gauge how the interaction improves as the player progresses to higher levels. Through the recording of pupil movements, computer technology is able to measure the interaction level of children with autism with accuracy. The game application proposed in this study uses pupil positions and movements to see how well children with autism interact. The method used in this study presents the results of experiments with ten children with autism and ten children without autism. For both groups, we used statistical descriptions to analyze the resulting pupil positions. Two descriptive analyses were performed: a comparison of pupil position modes between children with autism and children without autism from 1 to 120 seconds; and a comparison of pupil positions between children with autism and children without autism from frame one. The results of the experiment showed that children with autism were encouraged to interact using video games. Their pupil movements were measured while playing the games. During the interaction, children with autism had inconsistent pupil movement patterns, while the control group, consisting of children without autism, showed consistent pupil movement patterns. Then the other results show, first, that the TFA application provides pupil position patterns that are highly suitable as a biomarker for measuring the interaction level of children with autism. Second, further research is needed to implement therapies for children with autism to improve their interaction levels based on these measurements. Third, most of the research was conducted using data for level 2 autism (“requires substantial support”) so that in the future research may be conducted to further develop the application for level 3 autism (“requires very substantial support”). The conclusion of this study showed that children with and without autism have pupil movement patterns that remain within certain areas. Based on these areas, the severity of autism in children with autism is level 2 (moderate) according to DSM-5, making it possible to divide this level into three sub-levels: low-moderate, moderate-moderate, and high-moderate.

References

[1] M. D. Samad, N. Diawara, J. L. Bobzien, C. M. Taylor, J. W. Harrington, and K. M. Iftekharuddin, “A pilot study to identify autism related traits in spontaneous facial actions using computer vision,” Res. Autism Spectr. Disord., vol. 65, no. September 2018, pp. 14–24, 2019, doi: 10.1016/j.rasd.2019.05.001.

[2] M. Ewa, “Neuropsychologia. A novel machine learning analysis of eye-tracking data reveals suboptimal visual information extraction from facial stimuli in individuals with autism,” vol. 129, no. February, pp. 397–406, 2019, doi: 10.1016/j.neuropsychologia.2019.04.022.

[3] H. J. Nuske, G. Vivanti, K. Hudry, and C. Dissanayake, “Pupillometry reveals reduced unconscious emotional reactivity in autism,” Biol. Psychol., vol. 101, no. 1, pp. 24–35, 2014, doi: 10.1016/j.biopsycho.2014.07.003.

[4] Y. Lin, J. Wong, and Y. Tseng, “Game-Based Social Interaction Platform for Cognitive Assessment of Autism Using Eye Tracking,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 31, pp. 749–758, 2023, doi: 10.1109/TNSRE.2022.3232369.

[5] D. L. Reisinger et al., “Atypical Social Attention and Emotional Face Processing in Autism Spectrum Disorder : Insights From Face Scanning and Pupillometry,” vol. 13, no. February, pp. 1–14, 2020, doi: 10.3389/fnint.2019.00076.

[6] R. Carette, M. Elbattah, G. Dequen, J. L. Guerin, and F. Cilia, “Visualization of Eye-Tracking Patterns in Autism Spectrum Disorder: Method and Dataset,” 2018 13th Int. Conf. Digit. Inf. Manag. ICDIM 2018, pp. 248–253, 2018, doi: 10.1109/ICDIM.2018.8846967.

[7] M. K. Kwon, A. Moore, C. C. Barnes, D. Cha, and K. Pierce, “Typical Levels of Eye-Region Fixation in Toddlers with Autism Spectrum Disorder Across Multiple Contexts,” J. Am. Acad. Child Adolesc. Psychiatry, vol. 58, no. 10, pp. 1004–1015, 2019, doi: 10.1016/j.jaac.2018.12.011.

[8] A. A. Magableh, “Eye Tracking Algorithms, Techniques, Tools, and Applications with an Emphasis on Machine Learning,” Expert Syst. Appl., p. 114037, 2020, doi: 10.1016/j.eswa.2020.114037.

[9] Z. A. Taha Ahmed and M. E. Jadhav, “A Review of Early Detection of Autism Based on Eye-Tracking and Sensing Technology,” Proc. 5th Int. Conf. Inven. Comput. Technol. ICICT 2020, pp. 160–166, 2020, doi: 10.1109/ICICT48043.2020.9112493.

[10] S. Alarcon-Licona, L. Loke, and N. Ahmadpour, “From Autism Educators to Game Designers Integrating Teaching Strategies into Game Design for Autism Education Support,” ACM Int. Conf. Proceeding Ser., pp. 58–62, 2018, doi: 10.1145/3292147.3292208.

[11] M. Kassner, W. Patera, and A. Bulling, “Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-based Interaction,” UbiComp 2014 – Adjun. Proc. 2014 ACM Int. Jt. Conf. Pervasive Ubiquitous Comput., pp. 1151–1160, 2014, doi: 10.1145/2638728.2641695.

[12] M. Rudling, P. Nyström, S. Bölte, and T. Falck-Ytter, “Larger Pupil Dilation to Nonsocial Sounds in Infants with Subsequent Autism Diagnosis,” J. Child Psychol. Psychiatry Allied Discip., vol. 63, no. 7, pp. 793–801, 2022, doi: 10.1111/jcpp.13520.

[13] M. Of and M. Disorders, DSM-5.

[14] C. Linnemann et al., “Predictive Value of Serum Neurofilament Light Chain for Persistent Cognitive Deficits in Elderly Depressive Patients,” J. Affect. Disord. Reports, vol. 4, no. December 2020, p. 100095, 2021, doi: 10.1016/j.jadr.2021.100095.

[15] K. Zhang, L. Gao, J. Chen, X. Liu, G. Wang, and M. Liao, “Eye-Tracking Analysis of Autistic Children’s Attention to Social Stimuli,” Proc. – 2020 Int. Symp. Educ. Technol. ISET 2020, pp. 268–272, 2020, doi: 10.1109/ISET49818.2020.00065.

[16] J. Zou, H. Zhang, and T. Weng, “New 2D Pupil and Spot Center Positioning Technology under Real-Time Eye Tracking,” ICCSE 2017 – 12th Int. Conf. Comput. Sci. Educ., no. Iccse, pp. 110–115, 2017, doi: 10.1109/ICCSE.2017.8085473.

[17] M. Pater, M. Spreen, and T. Van Yperen, “The Developmental Progress in Social Behavior of Children with Autism Spectrum Disorder Getting Music Therapy. A Multiple Case Study,” Child. Youth Serv. Rev., vol. 120, no. July 2020, p. 105767, 2021, doi: 10.1016/j.childyouth.2020.105767.

[18] Y. Ng and M. S. Pera, “Recommending Social-Interactive Games for Adults with Autism Spectrum Disorders ( ASD ),” pp. 209–213.

[19] C. Crowell, J. Mora-Guiard, and N. Pares, “Structuring Collaboration: Multi-user Full-Body Interaction Environments for Children with Autism Spectrum Disorder,” Res. Autism Spectr. Disord., vol. 58, no. July 2017, pp. 96–110, 2019, doi: 10.1016/j.rasd.2018.11.003.

[20] A. F. Azizah, A. Djunaidy, D. Siahaan, and F. Suhariadi, “Improving the Interaction of Autistic Children through Eye Tracking using Gamification Design Framework,” 2021 Int. Conf. Comput. Syst. Inf. Technol. Electr. Eng. COSITE 2021, pp. 57–62, 2021, doi: 10.1109/COSITE52651.2021.9649613.

[21] A. F. Azizah and T. D. Susanto, “Conceptual Model Cognitive, Affective, Physical, and External Factor for Individual Information Technology Acceptance,” IPTEK J. Proc. Ser., vol. 2, no. 1, pp. 171–172, 2016, doi: 10.12962/j23546026.y2015i1.1169.

[22] H. Zhao, A. R. Swanson, A. S. Weitlauf, Z. E. Warren, N. Sarkar, and S. Member, “Hand-in-Hand: A Communication-Enhancement Collaborative Virtual Reality System for Promoting Social Interaction in Children With Autism Spectrum Disorders,” pp. 1–13, 2018.

[23] ADAMU, Amina Sani; ABDULLAHI, Saleh El-Yakub. A Review of Autism Diagnosis/Screening Expert System and Mobile Application. In: 2020 IEEE 2nd International Conference on Cyberspace (CYBER NIGERIA). IEEE, 2021. p. 80-84.