ArtSpeechIRMFr a MRI database of static and dynamic images (around 1 hour of continuous speech for 2 male French speakers).
This film was recorded at Göttingen Max Planck Institute with the real time system developed by Jens Frahm and his team.The speech sound was denoised with software developed by Emmanuel Vincent (MultiSpeech team a LORIA).
The database is available. Contact Yves.Laprie@loria.fr for further details.
Since october 2019 new films have been recorded using the recently purchased system thanks to a grant of the CPER « Humanités Numériques ».
This film recorded with the new system shows the MRI film together with the spectrogram and the phonetic segmentation. Speech has been acquired with an optical microphone and then denoised. Signal without speech is used to calibrate denoising.
Those film exhibit the result of tracking lip and tongue contours with a DNN approach. The image intensity has been reduced so that the contour (the black or red points depending on the video) appears more clearly.The sound is available for the BW video.
