Tracking vocal tract in real-time MRI
Projects | | Links: code | dataset | poper
Convolutional encoder-decoder models to track articulators of human vocal tract in real-time MRI.
Real-time MRI can help understand human speech product processes. Automatic segmentation of articulators (e.g., tongue, lips) is vital for study of vocal tract dynamics in rtMRI.
The overall methodology and examples are shown below. For more details, see the paper
Inspired by semantic segmentation work, we propose a full convolutional encoder-decoder network to predict articulatory boundaries (colored lines shown on the MRI image in Fig. 2). The model performance was further improve by adding a penalty to the loss function to peanalize heavily when articulator pixels are confused with that of air vs. tissue. This helped reinforce the tissue boundary segmentation with a higher confidence.
