Visual Encoding of Behavioral Language Signals for Deep Learning-Based Prediction of Lexical Access
DOI:
https://doi.org/10.29284/7fmmfq81Keywords:
Computational psycholinguistics; behavioural signal analysis; image encoding; lexical access; convolutional neural network; lexical decision task; signal processing; visual word recognition.Abstract
Lexical access is a fundamental cognitive process underlying visual word recognition and language comprehension. While previous psycholinguistic research has primarily relied on statistical modelling of behavioural responses, relatively little attention has been given to representing behavioural language signals as visual data for computational analysis. The present study proposes an interdisciplinary framework that integrates psycholinguistics, signal processing, image analysis, and deep learning to predict lexical access efficiency using secondary behavioural data from the British Lexicon Project. Behavioural indicators, including reaction time, response accuracy, word frequency, familiarity, age of acquisition, and orthographic characteristics, were transformed into image-based representations through behavioural signal encoding techniques. These visual representations served as inputs to a convolutional neural network (CNN) for classifying lexical access efficiency into fast and slow recognition categories. Comparative evaluation against conventional machine-learning algorithms demonstrated the effectiveness of image-based deep learning for capturing complex nonlinear relationships within behavioural language signals. The study contributes to signal and image sciences by introducing behavioural language data as a novel form of computational image representation and demonstrates how computer vision techniques can extend traditional psycholinguistic analyses. The proposed framework establishes a methodological bridge between cognitive language research, artificial intelligence, and behavioural signal imaging, providing new opportunities for interdisciplinary research in computational psycholinguistics.
References
1. Andrews, S. (1997). The effect of orthographic similarity on lexical retrieval: Resolving neighbourhood conflicts. Psychonomic Bulletin & Review, 4, 439–461.
2. Andrews, S. (2012). The development of visual word recognition: Theories and findings. In J. S. Adelman (Ed.), Visual word recognition (Vol. 1, pp. 3–32). Psychology Press.
3. Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., Chatila, R., & Herrera, F. (2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58, 82–115. https://doi.org/10.1016/j.inffus.2019.12.012
4. Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., Neely, J. H., Nelson, D. L., Simpson, G. B., & Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39(3), 445–459. https://doi.org/10.3758/BF03193014
5. Balota, D. A., Yap, M. J., Cortese, M. J., & Watson, J. M. (2008). Beyond mean response latency: Response time distributional analyses of semantic priming. Journal of Memory and Language, 59(4), 495–523.
6. Coltheart, M., Rastle, K., Perry, C., Langdon, R., & Ziegler, J. (2001). DRC: A dual route cascaded model of visual word recognition and reading aloud. Psychological Review, 108(1), 204–256.
7. Eckmann, J. P., Kamphorst, S. O., & Ruelle, D. (1987). Recurrence plots of dynamical systems. Europhysics Letters, 4(9), 973–977.
8. Forster, K. I., & Chambers, S. M. (1973). Lexical access and naming time. Journal of Verbal Learning and Verbal Behavior, 12(6), 627–635.
9. Goldberg, Y. (2017). Neural network methods for natural language processing. Morgan & Claypool Publishers.
10. Gonzalez, R. C., & Woods, R. E. (2018). Digital image processing (4th ed.). Pearson.
11. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
12. Grainger, J. (2008). Cracking the orthographic code: An introduction. Language and Cognitive Processes, 23(1), 1–35.
13. Grainger, J., & Jacobs, A. M. (1996). Orthographic processing in visual word recognition: A multiple read-out model. Psychological Research, 59(3), 154–166.
14. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778).
15. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25.
16. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436–444. https://doi.org/10.1038/nature14539
17. Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., Van Der Laak, J. A. W. M., Van Ginneken, B., & Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60–88.
18. Mallat, S. (2009). A wavelet tour of signal processing: The sparse way (3rd ed.). Academic Press.
19. McClelland, J. L., & Rumelhart, D. E. (1981). An interactive activation model of context effects in letter perception: Part 1. An account of basic findings. Psychological Review, 88(5), 375–407.
20. Morton, J. (1969). Interaction of information in word recognition. Psychological Review, 76(2), 165–178.
21. Norris, D. (2013). Models of visual word recognition. Trends in Cognitive Sciences, 17(10), 517–524. https://doi.org/10.1016/j.tics.2013.08.003
22. Plaut, D. C., McClelland, J. L., Seidenberg, M. S., & Patterson, K. (1996). Understanding normal and impaired word reading: Computational principles in quasi-regular domains. Psychological Review, 103(1), 56–115.
23. Ratcliff, R. (1993). Methods for dealing with reaction time outliers. Psychological Bulletin, 114(3), 510–532.
24. Samek, W., Montavon, G., Vedaldi, A., Hansen, L. K., & Müller, K.-R. (Eds.). (2019). Explainable AI: Interpreting, explaining and visualizing deep learning. Springer.
25. Seidenberg, M. S., & McClelland, J. L. (1989). A distributed developmental model of word recognition and naming. Psychological Review, 96(4), 523–568.
26. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
27. Wang, Z., & Oates, T. (2015). Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, 40–46.
28. Yap, M. J., & Balota, D. A. (2015). Visual word recognition. In A. Pollatsek & R. Treiman (Eds.), The Oxford handbook of reading (pp. 26–43). Oxford University Press.
29. Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology. Perspectives on Psychological Science, 12(6), 1100–1122.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
