Non-professional music recordings shared on online platforms often suffer from background noise and reverberation, which compromise their perceived quality and hinder their potential for reuse.
To address this issue, this paper proposes DSME a novel Music Enhancement model, based on Dual time-frequency Spectral representations, which are tailored to the characteristics of music.
The DSME employs short-time Fourier transform (STFT) and constant-Q transform (CQT) spectra for prediction and discrimination, respectively, within the framework of a generative adversarial network (GAN).
On one hand, leveraging the fixed-window, invertibility, and predictability of the STFT, the generator of DSME predicts clean music amplitude–phase spectra from their degraded counterparts, and then reconstructs the enhanced music waveform via inverse STFT.
On the other hand, utilizing the log-frequency-scale and variable-window of the CQT, which better align with the octave-based structure of music, we propose an octave-segmented CQT spectrum discriminator for training.
In addition, we introduce a chroma-spectrum-based loss to further emphasize pitch and harmonic consistency, enhancing the alignment of the enhanced waveform with the underlying musical structure and characteristics.
Experimental results demonstrate that our proposed DSME outperforms the baseline models in both objective and subjective evaluations, confirming the effectiveness of the dual-spectrum approach.