Neural Transform-Domain Mappings

While the Fourier transform domain is computationally convenient, other domains such as the log power spectral, cepstral, and LPC domains are often more desirable to work in. For example, the log spectral domain is thought to offer a more perceptually relevant measure of distortion than the spectral domain, and also allows homomorphic filtering [27] to better compensate for channel distortion. If the goal of the enhancement is to improve perceptual quality, then there are clear advantages to minimizing distortion in a perceptually relevant domain. Furthermore, if the enhancement is for a front end to an ASR system, the transform-domain chosen for recognition usually dictates the domain in which speech enhancement occurs.

Figure: Transform-Domain Enhancement. A fixed transform is used to preprocess the data before it is modified. A neural network is used to perform the nonlinear enhancement mapping. Postprocessing transforms the signal back into the time-domain.

Unfortunately, removing noise in these alternative transform-domains can no longer be done with a simple linear subtraction of the noise from the speech. Because the transformed noise and speech are combined in a nonlinear way ( e.g., through a log function), a nonlinear form of ``subtraction'' is required. The necessary nonlinear function depends on the transform-domain and can be approximated by a neural network. This neural network ``subtraction'' or transform-domain mapping is illustrated in Figure 14.5. As in the time-domain approach, a representative training set of clean speech and corresponding noisy speech is used to train the network. However, one advantage of the transform-domain is that time-alignment of the inputs and targets is not as critical as with a time-domain mapping. This allows for generation of training data in more realistic settings using less controlled recording devices.

Although neural network ``subtraction'' allows for the use of nonlinear transforms, this flexibility comes at considerable cost. In classic spectral subtraction, the ``mapping'' is simply a subtraction and is functionally independent of the noise level and noise spectrum. This is precisely due to the linearity of the Fourier transform. However, if another transform is used, the neural network must be able to provide different mappings for different noise types and levels. This can be attempted by incorporating additional inputs to the network which encode estimates of the SNR and/or noise distribution in some way. Clearly, this requires that the training set includes a representative sample of the noise levels and distributions which are expected to be encountered by the final system.

During operation, errors in the noise spectrum estimates will result in degradation of performance. The problems associated with estimating the noise statistics are fundamentally the same as in classic spectral subtraction. However, artifacts such as musical noise may be less severe due to properties of the transform domain chosen. For example, the variance in the short-term spectral estimation using an LPC analysis will be lower than a direct DFT approach.⁶ Also, the nature of the nonlinear mapping is less likely to produce values that require the kind of strict truncation used in linear spectral subtraction. Finally, the use of a nonlinear mapping allows for a fair amount of freedom in the choice of transform domain, which can therefore be chosen for its inherent robustness and perceptual qualities. The associated enhancement technique might then be less affected by variations in the noise spectral estimation.

Like the noise source, the distribution of the speech signal also affects the neural network mapping. Training with a single network averages across all speech signals and effectively assumes stationarity of speech (both within a speech signal and between different speakers). This assumption is the same as in the direct time-domain approaches. However, it is not fully understood whether this assumption is more severe in the time-domain versus some transform domain. Approaches which skirt this issue through the use of multiple networks will be discussed in the next sections.

A number of researchers have performed preliminary investigations based on neural transform domain mappings. We summarize some of this work here.

• The use of a feedforward neural network in the log power spectral domain has been demonstrated by Xie in [28] (an estimate of the mean of the speech signal and the noise variance provide additional input to the network). Informal listening tests showed favorable acceptance (no musical noise was reported). On a simple HMM speech recognition system (speaker dependent, 100 word digit database), using the method as a preprocessor resulted in a substantial improvement in recognition rates, as shown below:

  F16 Noise (SNR)	-6dB	0dB	6dB	12dB	18dB
  No Processing	16	45	81	94	99
  Neural preprocessing	50	89	96	95	97
  Difference	34	44	15	1	-2

  Car Noise (SNR)	-6dB	0dB	6dB	12dB	18dB
  No Processing	16	56	74	85	95
  Neural preprocessing	38	92	96	96	98
  Difference	22	36	22	11	3

• In similar work, Sorensen designed a system for compensating F-16 jet noise based on a mapping in a cepstral domain [29]. Impressive results were reported for an ASR experiment (510 word test set for multiple speakers), with the recognition rates summarized below:

  F16 Noise (SNR)	0dB	3dB	9dB	15dB	21dB	$\infty$dB
  No Preprocessing	14.1	17.8	33.1	59.4	83.3	99.4
  Neural preprocessing	79.4	82.8	86.9	91.6	93.5	-
  Difference	65.3	65.0	53.8	32.2	10.2	-

• Use of a time-delay neural network (TDNN) for Mel-scaled spectral estimation was reported by Dawson [30] (though performance is hard to evaluate).

• Moon et al. report mapping LPC coefficients with an Elman net used in conjunction with a TDNN speech recognizer [31]. They also proposed ``coordinated'' training in which the recognition error was used (through backpropagation) to directly modify the preprocessor enhancement network.

• Finally, Wan et al. [32] reported on experiments in which feedforward neural networks were used to modify RASTA processing [33]. In this work, as in spectral subtraction, a Fourier transform is used, but neural networks form nonlinear filters to map the trajectory of noisy magnitude spectral components, over several analysis windows, to an estimate of the clean spectrum. While noticeable improvement in noise reduction was achieved, often greater signal distortion and more musical noise was noted.

Figure: Switching model based on signal classification