Artificial intelligence application in synaesthesia research

Abstract

Synaesthesia is an unusual neurological condition in which a cognitive process or stimulation in one sensory modality triggers another type of sensory sensation. One of the most common forms of synaesthesia is colored hearing – when sounds elicit an automatic involuntary sensation of color. To test the authenticity of the case, when studying synaesthesia, a test of genuineness (TOG) is usually applied. A positive TOG result confirms the temporal stability of synesthetic experiences. This study analyses a reported case of color hearing synaesthesia of a girl (SB) with a visual deficiency. She can only see colorless silhouettes and reports people develop a person-specific color after communicating with them. The case was confirmed to be genuine with a 75% TOG result. This study hypothesized that SB synesthetic sensations are evoked by different parameters of people’s voices. To confirm this hypothesis, we extracted 68 voice features from study participants’ voice recordings and used them to train a multilayer perceptron (MLP). The MLP classified previously unseen recordings with 94% accuracy. To find the most relevant features a random forest classifier was applied. It was determined, that by reducing the number of features to 21 of the most relevant ones, the MLP classification increased to 97%. This study used a novel approach to investigate a case of synaesthesia. The approach confirmed the different color synesthetic experiences to be linked to certain voice features.