Classification of Autism Spectrum Disorder (ASD) in Children Using the VGG19 CNN Model Based on Facial Landmarks of the Eye and Forehead Areas
Downloads
Early detection of Autism Spectrum Disorder (ASD) is a crucial challenge in child development interventions because conventional screening methods are often subjective and prone to assessor bias. This study proposes an objective solution in the form of a deep learning approach for automatic ASD classification using facial landmark representations that focus exclusively on the eye and forehead areas. The selection of these areas is based on the eye avoidance hypothesis, which states that these regions contain very rich diagnostic information and behavioral biomarkers related to the ASD phenotype. The pre-processing stage involves isolating the eye and forehead areas using Dlib 68-landmark detection to eliminate background visual noise, followed by detailed topological visualization using MediaPipe Face Mesh with 478 landmark points as the model input. The Convolutional Neural Network (CNN) architecture used is the VGG19 model modified with transfer learning techniques and the addition of Dropout layers to improve efficiency and prevent overfitting. The model was trained on a primary dataset of 1,238 images collected under controlled conditions from children in Banda Aceh. The test results showed very promising performance with an overall accuracy of 94.35%. Specifically, the model achieved a recall (sensitivity) of 95.24%, a precision of 93.75%, and an AUC score of 0.9831. This high sensitivity is crucial in a medical context to minimize the risk of misdetection of positive cases. These results demonstrate that landmark visualization in the eye and forehead areas with the VGG19 model is a highly effective, accurate, and practical method for serving as an economical early screening tool for ASD.
[1] O. I. Talantseva dkk., “The global prevalence of autism spectrum disorder: A three-level meta-analysis,” Front. Psychiatry, vol. 14, hlm. 1071181, Feb 2023, doi: 10.3389/fpsyt.2023.1071181.
[2] N. Salari dkk., “The global prevalence of autism spectrum disorder: a comprehensive systematic review and meta-analysis,” Ital. J. Pediatr., vol. 48, no. 1 hlm. 112, Des 2022, doi: 10.1186/s13052-022-01310-w.
[3] R. Revilla dkk., “Facial Emotion Recognition Trainings for Children and Adolescents: A Transdiagnostic Systematic Review and Meta‐Analysis,” Clin. Psychol. Psychother., vol. 32, no. 4, hlm. e70116, Jul 2025, doi: 10.1002/cpp.70116
[4] C. Lord dkk., “The Lancet Commission on the future of care and clinical research in autism,” The Lancet, vol. 399, no. 10321, hlm. 271–334, Jan 2022, doi: 10.1016/S0140-6736(21)01541-5.
[5] K. Allen, J. Harrington, L. B. Quetsch, J. Masse, C. Cooke, dan J. F. Paulson, “Parent–Child Interaction Therapy for Children with Disruptive Behaviors and Autism: A Randomized Clinical Trial,” J. Autism Dev. Disord., vol. 53, no. 1, hlm. 390–404, Jan 2023, doi: 10.1007/s10803-022-05428-y.
[6] I. Scarcella dkk., “Information and communication technologies-based interventions for children with autism spectrum conditions: a systematic review of randomized control trials from a positive technology perspective,” Front. Psychiatry, vol. 14, hlm. 1212522, Jul 2023, doi: 10.3389/fpsyt.2023.1212522.
[7] S. Cored Bandrés, S. Vázquez Toledo, dan M. Liesa Orús, “Social skills, autism and technologies: An analysis of the effectiveness of this triad,” Educ. Inf. Technol., vol. 28, no. 8, hlm. 9285–9304, Agu 2023, doi: 10.1007/s10619-022-10990-0.
[8] N. Stuart, A. Whitehouse, R. Palermo, E. Bothe, dan N. Badcock, “Eye Gaze in Autism Spectrum Disorder: A Review of Neural Evidence for the Eye Avoidance Hypothesis,” J. Autism Dev. Disord., vol. 53, no. 5, hlm. 1884–1905, Mei 2023, doi: 10.1007/s10803-022-05443-z.
[9] M. Rudling, P. Nyström, G. Bussu, S. Bölte, dan T. Falck-Ytter, “Infant responses to direct gaze and associations to autism: A live eye-tracking study,” Autism, vol. 28, no. 7, hlm. 1677–1689, Jul 2024, doi: 10.1177/13623613231203037.
[10] K. Rezaee, H. Attar, dan M. Khosravi, “A review of machine learning-based methods for automatically detecting autism spectrum disorder in children’s faces,” dalam 2023 2nd International Engineering Conference on Electrical, Energy, and Artificial Intelligence (EICEEAI), Zarqa, Jordan: IEEE, Des 2023, hlm. 1–5. doi: 10.1109/EICEEAI60672.2023.10590257.
[11] F. Rosindo Daher De Barros, C. Novais F. Da Silva, G. De Castro Michelassi, H. Brentani, F. L. S. Nunes, dan A. Machado-Lima, “Computer aided diagnosis of neurodevelopmental disorders and genetic syndromes based on facial images – A systematic literature review,” Heliyon, vol. 9, no. 10, hlm. e20517, Okt 2023, doi: 10.1016/j.heliyon.2023.e20517.
[12] D. Mukherjee dkk., “Digital tools for direct assessment of autism risk during early childhood: A systematic review,” Autism, vol. 28, no. 1, hlm. 6–31, Jan 2024, doi: 10.1177/13623613221133176.
[13] M. Sobieski, A. Sobieska, M. Sekułowicz, dan M. M. Bujnowska-Fedak, “Tools for early screening of autism spectrum disorders in primary health care – a scoping review,” BMC Prim. Care, vol. 23, no. 1, hlm. 46, Des 2022, doi: 10.1186/s12875-022-01645-7.
[14] S. Zhang, S. Wang, R. Liu, H. Dong, X. Zhang, dan X. Tai, “A bibliometric analysis of research trends of artificial intelligence in the treatment of autistic spectrum disorders,” Front. Psychiatry, vol. 13, hlm. 967074, Agu 2022, doi: 10.3389/fpsyt.2022.967074.
[15] B. Sistaninejhad, H. Rasi, dan P. Nayeri, “A Review Paper about Deep Learning for Medical Image Analysis,” Comput. Math. Methods Med., vol. 2023, no. 1, hlm. 7091301, Jan 2023, doi: 10.1155/2023/7091301.
[16] M. Li, Y. Jiang, Y. Zhang, dan H. Zhu, “Medical image analysis using deep learning algorithms,” Front. Public Health, vol. 11, hlm. 1273253, Nov 2023, doi: 10.3389/fpubh.2023.1273253.
[17] M. Melinda, F. H. Juwono, I. K. A. Enriko, M. Oktiana, S. Mulyani, dan K. Saddami, “Application of continuous wavelet transform and support vector machine for autism spectrum disorder electroencephalography signal classification,” Radioelectron. Comput. Syst., no. 3, hlm. 73–90, Sep 2023, doi: 10.32620/reks.2023.3.07.
[18] M. Melinda, F. Arnia, A. Yafi, N. A. C. Andryani, dan I. K. A. Enriko, “Design and Implementation of Mobile Application for CNN-Based EEG Identification of Autism Spectrum Disorder”.
[19] A. W. Salehi dkk., “A Study of CNN and Transfer Learning in Medical Imaging: Advantages, Challenges, Future Scope,” Sustainability, vol. 15, no. 7, hlm. 5930, Mar 2023, doi: 10.3390/su15075930.
[20] A. A. Mukhlif, B. Al-Khateeb, dan M. A. Mohammed, “An extensive review of state-of-the-art transfer learning techniques used in medical imaging: Open issues and challenges,” J. Intell. Syst., vol. 31, no. 1, hlm. 1085–1111, Sep 2022, doi: 10.1515/jisys-2022-0198.
[21] H. E. Kim, A. Cosa-Linan, N. Santhanam, M. Jannesari, M. E. Maros dan T. Ganslandt, “Transfer learning for medical image classification: a literature review,” BMC Med. Imaging, vol. 22, no. 1, hlm. 69, Des 2022, doi: 10.1186/s12880-022-00793-7.
[22] G. Quatrosi dkk., “Cranio-Facial Characteristics in Autism Spectrum Disorder: A Scoping Review,” J. Clin. Med., vol. 13, no. 3, hlm. 729, Jan 2024, doi: 10.3390/jcm13030729.
[23] M. Melinda, M. Oktiana, Y. Nurdin, I. Pujiati, M. Irhamsyah, dan N. Basir, “Performance of ShuffleNet and VGG-19 Architectural Classification Models for Face Recognition in Autistic Children,” Int. J. Adv. Sci. Eng. Inf. Technol., vol. 13, no. 2, hlm. 674–680, Apr 2023, doi: 10.18517/ijaseit.13.2.18274.
[24] Md. F. Rabbi dkk., “Autism Spectrum Disorder Detection Using Transfer Learning with VGG 19, Inception V3 and DenseNet 201,” dalam Recent Trends in Image Processing and Pattern Recognition, vol. 1704, K. Santosh, A. Goyal, D. Aouada, A. Makkar, Y.-Y. Chiang, dan S. K. Singh, Ed., dalam Communications in Computer and Information Science, vol. 1704. , Cham: Springer Nature Switzerland, 2023, hlm. 190–204. doi: 10.1007/978-3-031-23599-3_14.
[25] J. Junidar, M. Melinda, D. D. Diannuari, D. D. Acula, dan Z. Zainal, “Face autistic classification based on thermal using image ensemble learning of VGG-19, ResNet50v2, and EfficientNet,” Radioelectron. Comput. Syst., vol. 2025, no. 1, hlm. 153–164, Feb 2025, doi: 10.32620/reks.2025.1.11.
[26] M. A. Witherow, M. D. Samad, N. Diawara, dan K. M. Iftekharuddin, “Facial landmark feature fusion in transfer learning of child facial expressions,” dalam Applications of Machine Learning 2022, M. E. Zelinski, T. M. Taha, dan J. Howe, Ed., San Diego, United States: SPIE, Okt 2022, hlm. 34. doi: 10.1117/12.2641898.
[27] S. Poudel dkk., “Autism Spectrum Disorder Detection using Facial Landmark Detection and Artificial Neural Network”.
[28] A. Lakkapragada dkk., “The Classification of Abnormal Hand Movement to Aid in Autism Detection: Machine Learning Study,” JMIR Biomed. Eng., vol. 7, no. 1, hlm. e33771, Jun 2022, doi: 10.2196/33771.
[29] R. Mohan, S. Kadry, V. Rajinikanth, A. Majumdar, dan O. Thinnukool, “Automatic Detection of Tuberculosis Using VGG19 with Seagull-Algorithm,” Life, vol. 12, no. 11, hlm. 1848, Nov 2022, doi: 10.3390/life12111848.
[30] L. K. Gaddala dkk., “Autism Spectrum Disorder Detection Using Facial Images and Deep Convolutional Neural Networks,” Rev. Intell. Artif., vol. 37, no. 03, hlm. 801–806, Jun 2023, doi: 10.18280/ria.370329.
[31] M. Z. Al-Taie, E. Sabeeh, dan S. Sabeeh, “Comparative Deep Learning–Based Facial Image Analysis for Early Autism Prediction in School-Aged Children,” vol. 4, no. 1, 2025.
[32] M. J. Awan dkk., “Image-Based Malware Classification Using VGG19 Network and Spatial Convolutional Attention,” Electronics, vol. 10, no. 19, hlm. 2444, Okt 2021, doi: 10.3390/electronics10192444.
[33] H. Alkahtani, T. H. H. Aldhyani, dan M. Y. Alzahrani, “Deep Learning Algorithms to Identify Autism Spectrum Disorder in Children-Based Facial Landmarks,” Appl. Sci., vol. 13, no. 8, hlm. 4855, Apr 2023, doi: 10.3390/app13084855.
[34] M.S.Alam dkk., “Efficient Deep Learning-Based Data-Centric Approach for Autism Spectrum Disorder Diagnosis from Facial Images Using Explainable AI,” Technologies, vol. 11, no. 5, hlm.115, Agu 2023, doi: 10.3390/technologies11050115.
[35] M. E. Tezcan, A. E. Ataş, dan H. Ferahkaya, “Can optic nerve morphology in children with autism spectrum disorder be associated with atypical visual-sensory behaviors?,” Front. Psychiatry, vol. 16, hlm. 1619695, Agu 2025, doi: 10.3389/fpsyt.2025.1619695.
[36] M. S. Alam, M. M. Rashid, R. Roy, A. R. Faizabadi, K. D. Gupta, dan M. M. Ahsan, “Empirical Study of Autism Spectrum Disorder Diagnosis Using Facial Images by Improved Transfer Learning Approach,” Bioengineering, vol. 9, no. 11, hlm. 710, Nov 2022, doi: 10.3390/bioengineering9110710.
[37] M. Tsuneki, “Deep learning models in medical image analysis,” J. Oral Biosci., vol. 64, no. 3, hlm. 312–320, Sep 2022, doi: 10.1016/j.job.2022.03.003.
[38] A. Faghihi, M. Fathollahi, dan R. Rajabi, “Diagnosis of Skin Cancer Using VGG16 and VGG19 Based Transfer Learning Models,” 1 April 2024, arXiv: arXiv:2404.01160. doi: 10.48550/arXiv.2404.01160.
[39] “Modified VGG-19 Deep Learning Strategies for Parkinson’s Disease Diagnosis - A Comprehensive Review and Novel Approach,” J. Angiother., vol. 8, no. 3, Mar 2024, doi: 10.25163/angiotherapy.839559.
[40] Z. Zhao, L. Alzubaidi, J. Zhang, Y. Duan, dan Y. Gu, “A comparison review of transfer learning and self-supervised learning: Definitions, applications, advantages and limitations,” Expert Syst. Appl., vol. 242, hlm. 122807, Mei 2024, doi: 10.1016/j.eswa.2023.122807.
[41] K. K. Mujeeb Rahman dan M. M. Subashini, “Identification of Autism in Children Using Static Facial Features and Deep Neural Networks,” Brain Sci., vol. 12, no. 1, hlm. 94, Jan 2022, doi: 10.3390/brainsci12010094.
[42] A. Alias dan M. A. As’ari, “Vision Based Screening of Children with Autism Spectrum Disorders Via Deep Learning Approach,” J. Hum. Centered Technol., vol. 4, no. 2, hlm. 148–154, Agu 2025, doi: 10.11113/humentech.v4n2.114.
[43] Master of Engineering Management, Lamar University, Beaumont, Texas, United States of America dkk., “COMPARATIVE ANALYSIS OF NEURAL NETWORK ARCHITECTURES FOR MEDICAL IMAGE CLASSIFICATION: EVALUATING PERFORMANCE ACROSS DIVERSE MODELS,” Am. J. Adv. Technol. Eng. Solut., vol. 04, no. 01, hlm. 01–42, Mar 2024, doi: 10.63125/feed1x52.
[44] Y. Li, W.-C. Huang, dan P.-H. Song, “A face image classification method of autistic children based on the two-phase transfer learning,” Front. Psychol., vol. 14, hlm. 1226470, Agu 2023, doi: 10.3389/fpsyg.2023.1226470.
[45] A. M. Al-Saadi dan D. Al-Thani, “Mobile Application to identify and recognize emotions for children with autism: A systematic review,” Front. Child Adolesc. Psychiatry, vol. 2, hlm. 1118665, Mar 2023, doi: 10.3389/frcha.2023.1118665.
[46] A. M. Al-Saadi dan D. Al-Thani, “Mobile Applicatio H. A. Mengash dkk., “Automated Autism Spectral Disorder Classification Using Optimal Machine Model”.http://dx.doi.org/10.32604/cmc.2023.032729
[47] M. Radhakrishnan, K. Ramamurthy, K. K. Choudhury, D. Won, dan T. Aarthy, “Performance Analysis of Deep Learning Models for Detection of Autism Spectrum Disorder from EEG Signals”. https://doi.org/10.18280/ts.380332
Copyright (c) 2026 yunidar, Arya Suyanda, Melinda Melinda, Lailatul Qadri Zakaria, Siti Rusdiana (Author)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlikel 4.0 International (CC BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
