Decoding-Neural-Activity-to-Reconstruct-and-Generate-Music-Stimuli

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

Decoding-Neural-Activity-to-Reconstruct-and-Generate-Music-Stimuli

★ 18 Open GitHub ↗

This project uses deep regression and generative models to reconstruct music from brain responses as participants passively listen to full length music.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Decoding-Neural-Activity-to-Reconstruct-and-Generate-Music-Stimuli
Published as: https://arxiv.org/pdf/2207.13845.pdf

This project uses deep regression and generative models to reconstruct music from brain responses as participants passively listen to full length music.

The project was inspired by Brain2Image (Kavasidis et al., 2017) model where generative models took in EEG data of participants looking at images as input and outputted reconstructions of the image stimuli.

For more details: Kavasidis, I., Palazzo, S., Spampinato, C., Giordano, D., & Shah, M. (2017, October). Brain2image: Converting brain signals into images. In Proceedings of the 25th ACM international conference on Multimedia (pp. 1809-1817).

In our project we continue this type of 'Mind Reading' research but with EEG responses to music. We take two directions, both a generative model approach and a neural decoding/signal processing approach. The latter is of focus as we posit a signal recovery functionality from CNN based deep regressions. Instead of learning a normal distribution of features from each stimulus class, we use deep regression to map a 1 to 1 from input to desired output. This puts the hypothesis foward that EEG signals can be treated as noisy versions of the auditory stimulus and allow for the continous reconstruction of a full length song instead of just the mean identity. For every 1 second of EEG data we can reconstruct 1 second of music the participant was listening to at that time.

![alt text](https://raw.githubusercontent.com/AGRamirezz/Decoding-Neural-Activity-to-Reconstruct-and-Generate-Music-Stimuli/main/Figures/Model_input_output1.png)

![alt text](https://raw.githubusercontent.com/AGRamirezz/Decoding-Neural-Activity-to-Reconstruct-and-Generate-Music-Stimuli/main/Figures/stim_rec1.png)

Decoding-Neural-Activity-to-Reconstruct-and-Generate-Music-Stimuli

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README