Auditory filterbanks benefit universal sound source separation

For separating two arbitrary sources from monaural recordings, the encoder-separator-decoder framework is popular in recent years. We investigated three kinds of filterbanks in the encoder: free, parameterized, and fixed. We proposed parameterized Gammatone and Gammachirp filterbanks, which improved performance with fewer parameters and better interpretability. Next, the properties of different filterbanks were investigated. Through training the network, an entirely freely learned filterbank emerges with properties similar to a series of bandpass filters spaced on a nonlinear scale - similar to the auditory system. We also explored the underlying separation mechanisms learned by the network through a classic auditory segregation experiment, revealing that the model separates mixtures based on the general principle (proximity of frequency and time). In summary, results demonstrate that the separation network automatically picks up the filterbank properties and separation mechanisms that are similar to those which have developed over millions of years in humans.