JP5852550B2 - Acoustic model generation apparatus, method and program thereof - Google Patents

Description

  The present invention relates to an acoustic model generation apparatus, method and program for generating an acoustic model by unsupervised adaptation.

  In speech recognition, generally, learning (adaptation) of an acoustic model is performed using a speech file and correct text representing the utterance content of the speech file as learning data. Note that “acoustic model learning (adaptation)” means a process of optimizing the parameters of the acoustic model so that as many examples as possible in the learning data can be correctly recognized by the learning process. In addition, the adaptation of the acoustic model is based on supervised adaptation using the correct text created as a learning data by the person writing up the reading corresponding to the speech file, and unsupervised adaptation using the speech recognition result as the learning data. It is roughly divided into

  Here, when the acoustic model is adapted by unsupervised adaptation, it is necessary to use a speech recognition result with high recognition accuracy as the correct text. However, when a speech recognition result with low recognition accuracy is used as a correct text, there is a possibility that the accuracy of the acoustic model is lowered due to incorrect adaptation of the acoustic model.

  To solve this problem, a method of assigning a confidence level to a speech recognition result, selecting a speech recognition result according to the reliability level, and adapting an acoustic model using the selected speech recognition result (for example, Patent Documents) 1) is known. The technique is based on the idea of using an utterance sequence whose reliability exceeds a reference value as learning data.

JP 2007-248730 A

  The conventional method of using an utterance sequence whose reliability exceeds a reference value as learning data has a problem that the learning efficiency is not high. That is, the fact that the reliability indicates a value equal to or greater than a certain value is nothing other than that the acoustic model has already been adapted to the acoustic feature amount. Therefore, even if such an utterance sequence is used, the accuracy of the acoustic model is not reduced, but the progress of acoustic model learning is slow and not efficient.

  The present invention has been made in view of this problem, and an object of the present invention is to provide an acoustic model generation apparatus, method and program capable of efficiently performing acoustic model learning.

  The acoustic model generation device of the present invention includes a label generation speech recognition unit, a data selection unit, and an acoustic model learning unit. The label generation speech recognition unit recognizes the input speech signal by referring to the language model and the base acoustic model, assigns a label to the speech recognition result, and outputs its reliability and acoustic likelihood. The data selection unit receives the speech signal output from the label recognition speech recognition unit, its label, and the reliability, and receives the speech signal having the reliability greater than the reliability threshold and the acoustic likelihood smaller than the likelihood threshold. select. The acoustic model learning unit generates a learned acoustic model by causing the audio signal selected by the data selection unit to learn the base acoustic model.

  According to the acoustic model generation device of the present invention, a speech signal having a speech recognition result having a reliability greater than or equal to a predetermined value and an acoustic likelihood smaller than the predetermined value can be used for learning the acoustic model. In other words, the learning efficiency of the acoustic model can be improved by using a speech signal that is not sufficiently advanced but is linguistically correct.

The figure which shows the function structural example of the acoustic model production | generation apparatus 100 of this invention. The figure which shows the operation | movement flow of the acoustic model production | generation apparatus 100. FIG. The figure explaining the relationship between acoustic likelihood and reliability. The figure which shows the function structural example of the acoustic model production | generation apparatus 200 of this invention. The figure which shows the operation | movement flow of the acoustic model production | generation apparatus 200. FIG. The figure explaining grammatical speech recognition. The figure which shows the function structural example of the acoustic model production | generation apparatus 300 of this invention. The figure which shows the function structural example of the acoustic model production | generation apparatus 400 of this invention. The figure which shows the function structural example of the acoustic model production | generation apparatus 500 of this invention. The figure which shows the function structural example of the acoustic model production | generation apparatus 600 of this invention. The figure which shows the function structural example of the acoustic model production | generation apparatus 700 of this invention. The figure which shows the function structural example of the acoustic model production | generation apparatus 800 of this invention.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.

  FIG. 1 shows an example of the functional configuration of an acoustic model generation apparatus 100 according to the present invention. The operation flow is shown in FIG. The acoustic model generation device 100 includes a label generation speech recognition unit 10, a language model 20, a base acoustic model 30, a data selection unit 40, an acoustic model learning unit 50, and a control unit 60. The acoustic model generation apparatus 100 is realized by a predetermined program being read into a computer including, for example, a ROM, a RAM, a CPU, and the like, and the CPU executing the program.

  The label generation speech recognition unit 10 receives a speech signal as an input, gives a label to the speech signal with reference to the language model and the base acoustic model, and outputs its reliability and acoustic likelihood (step S10). The input audio signal is converted into a discrete digital signal, for example, at a sampling frequency of 16 kHz, and converted into an acoustic feature amount for each frame where a predetermined number (eg, 320) of discrete audio signals is one frame. The In FIG. 1, the notation of the A / D conversion unit and the feature amount calculation unit is omitted. The acoustic feature amount is calculated by, for example, Mel frequency cepstrum coefficient (MFCC) analysis.

  The speech recognition unit for label generation 10 refers to the acoustic likelihood in the base acoustic model 30 of the acoustic feature amount calculated for each frame and the speech recognition result candidate having the highest sum of the language likelihoods with reference to the language model 20. Output as voice recognition result. At this time, for each voice file output as a speech recognition result, a label based on the reading used for learning the acoustic model, an acoustic likelihood, and a reliability are added to the speech recognition result composed of kanji kana magic morpheme and reading. Is done. An audio file is, for example, one sentence unit to be recognized.

  FIG. 3 shows an example of acoustic likelihood, language likelihood, and reliability of an audio file. When the voice recognition result of the voice file 1 is “Thank you for calling”, the label is “Thank you, odenwa, thank you,” for example, the acoustic likelihood is 8000, the language likelihood is -800, The reliability is 0.80. The voice recognition result of the voice file 2 is “I will ask you for your business.”, Its label is “, Goyoken, wa ga suru, wa”, for example, its acoustic likelihood is 4500, language likelihood is -900, The reliability is 0.80.

  As described above, the label, the acoustic likelihood, the language likelihood, and the reliability are given to the voice signal output from the label generating voice recognition unit 10 and the voice recognition result. The label generating voice recognition unit 10 is realized by a known voice recognition technique.

  The data selection unit 40 receives the audio signal output from the label generation speech recognition unit 10, the label and the reliability, and inputs the audio having the reliability higher than the reliability threshold and the acoustic likelihood lower than the likelihood threshold. Audio data including signals and the like is selected (step S40). That is, the data selection unit 40 selects voice data having a certain degree of reliability of the voice recognition result but having a low acoustic likelihood. In the example of FIG. 3, for example, “Ask for business” of the audio file 2 is selected. The selected speech data may be temporarily stored as unsupervised learning speech in the unsupervised learning speech DB 70 as indicated by a broken line in FIG.

  The data selection unit 40 selects, for example, voice data having a reliability of 0.80 or more and an acoustic likelihood of 4500, for example, an acoustic likelihood lower than the average value. Such voice data is voice data that has been correctly recognized but has not yet been trained in an acoustic model for the voice data.

  The acoustic model learning unit 50 learns the base acoustic model 30 from the audio signal included in the audio data selected by the data selection unit 40 to generate a learned acoustic model (step S50). The learning of the acoustic model is performed by learning the base acoustic model 30 using the audio signal selected by the data selection unit 40 and generating a learned acoustic model. When the amount of learning data is large, learning is repeated. The adaptation method of the acoustic model is not limited. For example, ML (Maximum Likelihood) learning based on maximum likelihood estimation using a Baum-Weltch algorithm or identification learning may be used. The learned acoustic model output by the acoustic model learning unit 50 is stored in an external learned acoustic model 80. The control unit 60 controls the time-series operation of each unit described above until the acoustic model generation device 100 finishes the operation (step S60).

  As described above, according to the acoustic model generation device 100 of the present invention, the voice data whose reliability of the voice recognition result is equal to or higher than the predetermined value and whose acoustic likelihood is smaller than the predetermined value, that is, the voice recognition is correctly performed. However, since the sound data that has not yet been learned for the acoustic model is used for the learning of the acoustic model, the learning efficiency of the acoustic model can be improved.

  FIG. 4 shows a functional configuration example of the acoustic model generation apparatus 200 of the present invention. The operation flow is shown in FIG. The acoustic model generation device 200 is different from the above acoustic model generation device 100 only in that a label conversion re-recognition unit 210 is provided.

  The label conversion re-recognition unit 210 performs grammatical speech recognition with reference to the base acoustic model 30 using the label output from the label generation speech recognition unit 10, re-assigns acoustic likelihood, and the data selection unit 40 (step S210). Here, the grammatical speech recognition is a method of recognizing speech according to a morphological arrangement, that is, grammar, for determination of reading or determination of silence insertion position.

  Since the reliability included in the speech recognition result is also limited by the language model, it may be different from the case where the silence insertion position is determined only by the acoustic model. Therefore, re-recognition is performed by grammatical speech recognition using only the acoustic model, not the language model, for the determination of reading or the insertion position of silence.

  FIG. 6 shows an example of a grammar corresponding to one sentence of an audio file “It is sunny today”. A route (path) having the maximum cumulative likelihood of state transition from the start to the end for this sentence is taken as a speech recognition result. At this time, the length of silence and the reading of a word having a plurality of readings are also determined.

  Since the data selection unit 40 of the acoustic model generation device 200 selects speech data in which the length of silence and the reading of words are determined based on the grammar, it is possible to improve the accuracy of silence and cope with multiple readings. The learning accuracy of the acoustic model in the learning unit 50 can be further improved.

  FIG. 7 shows a functional configuration example of the acoustic model generation apparatus 300 of the present invention. The acoustic model generation apparatus 300 is different from the above-described acoustic model generation apparatus 100 only in that the label generation voice recognition unit 10 is replaced with a label generation voice recognition unit 310.

  The label generation speech recognition unit 310 receives speech signals, performs speech recognition of the speech signals with reference to the language model 20 and the base acoustic model 30, assigns a label only to speech signals having a reliability higher than a predetermined value, and outputs them. To do. That is, it is an idea that voice data with low reliability is discarded and voice data with high reliability is used as learning data for the acoustic model. By doing so, audio data with low reliability is excluded before the processing of the data selection unit 40.

  The predetermined value of the reliability is a threshold value for determining whether or not to believe the voice recognition result, and is set to 0.8, for example. If the predetermined value of the reliability is set to a high value, for example, the amount of data selected by the data selection unit 40 decreases, so that the processing time of the acoustic model generation device 300 for the same amount of input speech signals is shortened. Thus, the processing speed of the acoustic model generation device 300 can be shortened compared to the acoustic model generation device 100.

  FIG. 8 shows a functional configuration example of the acoustic model generation apparatus 400 of the present invention. The acoustic model generation device 400 differs from the acoustic model generation device 100 described above only in that the data selection unit 40 is replaced with the data selection unit 440.

  The data selection unit 440 receives the speech signal output from the label generation speech recognition unit 10, the label, and the reliability, and the reliability is greater than the reliability threshold (for example, reliability> 0.8) and the acoustic likelihood is high. Voice data that is smaller than the threshold value and larger than the second likelihood threshold value is selected. The likelihood threshold is, for example, the average value μ of the acoustic likelihood distribution, and the second likelihood threshold is, for example, μ−σ. In such a case, those whose acoustic likelihood is 1σ or more smaller than the average can be excluded from learning.

  As a result, it is possible to exclude voice data having an acoustic likelihood that is too small from the learning target of the acoustic model, and expect the effect of improving the learning accuracy of the acoustic model.

  FIG. 9 shows a functional configuration example of the acoustic model generation apparatus 500 of the present invention. The acoustic model generation device 500 uses the points provided with the acoustic likelihood threshold value determination unit 510 and the threshold value determined by the acoustic likelihood threshold value determination unit 510 by the data selection unit 40 with respect to the acoustic model generation device 100 described above. The difference is that the data selection unit 540 performs selection.

  The acoustic likelihood threshold determination unit 510 automatically generates a likelihood threshold corresponding to the acoustic likelihood distribution output by the label conversion re-recognition unit 10. The acoustic likelihood is a logarithmic likelihood value unlike a probability value (value of 0 to 1), so it is difficult to determine a predetermined range. Therefore, the acoustic likelihood threshold determination unit 510 obtains, for example, the average μ and the standard deviation σ from the acoustic likelihood distribution output by the label generating speech recognition unit 10, and sets the threshold, for example, the average μ, μ-σ, or Automatically determined to a value such as μ + σ.

  Alternatively, a histogram of acoustic likelihood may be created, and a threshold value may be set in a frequency range from the histogram. For example, the frequency of the histogram is averaged, and a range higher than the frequency average is set as a frequency-high range. The data selection unit 540 selects audio data based on the threshold value determined by the acoustic likelihood threshold value determination unit 510. According to the acoustic model generation device 500, an effect of saving the trouble of setting the likelihood threshold of the data selection unit 540 can be obtained.

  FIG. 10 shows a functional configuration example of the acoustic model generation apparatus 600 of the present invention. The acoustic model generation device 600 includes an unsupervised learning speech database 670 (hereinafter referred to as DB), an acoustic model adaptation unit 650, and a threshold evaluation unit 660, in addition to the acoustic model generation device 100 described above, and data The difference is that the selection unit 40 is replaced with a multiple data selection unit 640.

  The multiple data selection unit 640 selects speech likelihood data having a plurality of reliability levels. For example, audio data having a reliability of 0.9 or higher, a reliability of 0.8 or higher, and a reliability of 0.75 or higher is selected. The voice data selected by each reliability value is stored in the unsupervised learning voice DB 670.

  The acoustic model adaptation unit 650 creates an adaptive acoustic model that only needs to be learned once, for example, by MAP adaptation (maximum posterior probability estimation). The acoustic model adaptation unit 650 adapts the acoustic model of the base acoustic model 30 for each reliability value stored in the unsupervised learning speech DB 670.

  The threshold evaluation unit 660 evaluates the acoustic model adapted to MAP for each reliability value using the development data set. A development data set is audio data with a transcript. The threshold evaluation unit 660 selects speech data using the reliability value with the highest speech recognition accuracy or acoustic likelihood for the development data set, and the acoustic model learning unit 50 unsupervises the speech signal included in the speech data. ML (likelihood maximization) learning and identification learning are repeatedly performed as learning data.

  According to the acoustic model generation apparatus 600, an optimum reliability threshold value is obtained by evaluating a plurality of reliability threshold values with an acoustic model obtained with a small amount of calculation such as MAP adaptation, and the acoustic model is obtained using the optimum reliability threshold value. Therefore, it is possible to reduce the time required for generating the acoustic model, compared to the case where the learning is repeatedly performed for all of the plurality of reliability threshold values.

  FIG. 11 shows a functional configuration example of the acoustic model generation apparatus 700 of the present invention. The acoustic model generation apparatus 700 is different from the above-described acoustic model generation apparatus 100 in that an existing speech DB 710 is provided and the acoustic model learning unit 50 is replaced with an acoustic model learning unit 750.

  The existing voice DB 710 is a database that stores voice data used to create the base acoustic model 30. The acoustic model learning unit 750 adaptively learns the base acoustic model 30 with reference to the voice data selected by the data selection unit 40 and the voice data of the existing voice DB 710.

  According to the acoustic model generation apparatus 700, the acoustic model is learned by combining the existing speech DB 710 and the generated unsupervised learning speech, so that an effect of improving the accuracy of the acoustic model can be expected. That is, there is a possibility that the unsupervised learning speech includes an error, whereas the speech of the existing speech DB 710 has no error, and the acoustic model is trained with the unsupervised learning speech by using the speech data without the error. Can be corrected. In short, the accuracy of the acoustic model can be improved by using the speech data without error together with the acoustic model learning, rather than learning the acoustic model only with unsupervised learning speech.

  FIG. 12 shows a functional configuration example of the acoustic model generation apparatus 800 of the present invention. The acoustic model generation apparatus 800 is different from the acoustic model purification apparatus 700 in that a pseudo non-recognition target signal DB 820 is provided.

  The pseudo non-recognition target signal DB 820 records a pseudo non-recognition target signal. The pseudo non-recognition target signal is a signal whose volume level is reduced by multiplying the interference signal by a gain of 1 or less. The interference signal is, for example, an audio signal in which a human voice is superimposed on a background noise of a crowd on a platform of a station. For example, a non-stationary human voice is superimposed on a stationary background noise and collected. Audio signal. There is no need for background noise. It may be a human voice with clean voice. That is, the point of the interference signal is that it is an unsteady audio signal.

  The pseudo non-recognition target signal is reduced from the recognition target voice by multiplying the interference signal by a gain of 1 or less to reduce the volume level, thereby reducing the possibility of learning the recognition target voice as non-speech. This increases the noise resistance of the non-voice model. The pseudo non-recognition target signal can be generated by the gain adjustment unit 810 that receives the interference signal.

  The acoustic model learning unit 850 adaptively learns the base acoustic model 30 with reference to the voice data selected by the data selection unit 40 and the voice data of the existing voice DB 710, and bases the pseudo non-recognition target signal as a non-speech signal. A non-voice model of the acoustic model 20 is learned. This non-speech model can be a model that reduces the occurrence of erroneous recognition results due to non-stationary interference signals, that is, background noise, by adapting to pseudo non-recognition target signals.

  Thus, in addition to the effects of the acoustic model generation device 700, the acoustic model generation device 800 can improve the noise resistance of the non-voice model.

  As described above, according to the acoustic model generation device of the present invention, it is possible to use speech data whose speech recognition result reliability is equal to or higher than a predetermined value and whose acoustic likelihood is smaller than the predetermined value for learning an acoustic model. . As a result, the learning efficiency of the acoustic model can be improved.

  The above-described acoustic model generation apparatus 100 may be combined with the idea of using the grammatical speech recognition of the acoustic model generation apparatus 200. Further, the acoustic model generation apparatuses 100 and 200 may be combined with the idea of discarding audio data with low reliability of the acoustic model generation apparatus 300. Further, the acoustic model generation apparatuses 100, 200, and 300 may be combined with the idea of the acoustic model generation apparatus 400 that performs unsupervised adaptation by removing voice data whose acoustic likelihood is too low. Further, the acoustic model generation apparatuses 100, 200, 300, and 400 may be combined with the idea of the acoustic model generation apparatus 500 that automatically sets a threshold value of acoustic likelihood. In addition, the acoustic model generation apparatuses 100, 200, 300, 400, and 500 evaluate a plurality of reliability thresholds with an acoustic model obtained with a small amount of calculation such as MAP adaptation to obtain an optimal reliability threshold, You may combine the idea of the acoustic model production | generation apparatus 500 of the idea which uses a reliability threshold value. Furthermore, you may combine the idea of the acoustic model production | generation apparatus 700 which uses the existing audio | voice DB710 in combination. Further, the idea of the acoustic model generation apparatus 800 that improves the noise tolerance of the non-voice model may be combined. As described above, the embodiments can be configured by combining them, and the respective effects can be obtained.

  When the processing means in the above apparatus is realized by a computer, the processing contents of the functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only) Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording media, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  This program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (6)

A speech recognition unit for label generation that recognizes speech by referring to an input speech signal with reference to a language model and a base acoustic model, gives a label to the speech recognition result, and outputs its reliability and acoustic likelihood;
The speech signal output from the label generating speech recognition unit, its label, reliability, and acoustic likelihood, and the reliability is greater than the reliability threshold and the acoustic likelihood is smaller than the likelihood threshold. A data selection section for selecting
An acoustic model learning unit for generating the learned acoustic model by learning the base acoustic model to the audio signal selected by the data selection unit;
Obtain one of average, standard deviation, and histogram from the distribution of acoustic likelihood output by the label generating speech recognition unit, and use at least one of the average, standard deviation, and histogram An acoustic model generation apparatus comprising an acoustic likelihood threshold determination unit that automatically generates the likelihood threshold . The acoustic model generation device according to claim 1,
Furthermore,
A label conversion re-recognition unit that performs grammatical speech recognition with reference to the acoustic model and re-assigns acoustic likelihood and outputs to the data selection unit using the label output by the label generation speech recognition unit ,
An acoustic model generation apparatus comprising the acoustic model generation apparatus. In the acoustic model generation device according to claim 1 or 2,
The label generating voice recognition unit
The sound is characterized in that the speech signal is recognized by referring to the language model and the base acoustic model with the speech signal as an input, and only a speech signal having a reliability of a predetermined value or higher is given a label and output. Model generator. The acoustic model generation device according to any one of claims 1 to 3,
The data selection part
With the speech signal output from the label generating speech recognition unit, its label and reliability as input, the reliability is greater than the reliability threshold, the acoustic likelihood is less than the likelihood threshold, and is greater than the second likelihood threshold. An acoustic model generation apparatus characterized by selecting a large audio signal. A speech recognition process for label generation that recognizes speech by referring to a speech model and a base acoustic model, gives a label to the speech recognition result, and outputs its reliability and acoustic likelihood;
An input audio signal and the reliability thereof label the label generation speech recognition process outputs, the reliability is large at and the acoustic likelihood than confidence threshold to select a small audio signal than the likelihood threshold The data selection process,
To the audio signal which the data selection process selected, the acoustic model training process for generating the learned acoustic model by learning the base acoustic model,
Obtain one of average, standard deviation, and histogram from the distribution of acoustic likelihood output by the label generation speech recognition process, and use at least one of the average, standard deviation, and histogram An acoustic model generation method comprising an acoustic likelihood threshold determination process for automatically generating the likelihood threshold . A program for causing a computer to function as an acoustic model generating apparatus according to claim 1乃optimum 4.

Images