JPH06131320A - Signal processor - Google Patents

Abstract

PURPOSE:To attain a high-speed response by decreasing the number of times of required measurement in the case of calculating an average value as an output value for dealing with the fluctuation of pulse density in a pulse density type neural network. CONSTITUTION:This signal processor for hierarchically forming a neural network 1 from an input layer to an output layer by coupling plural signal processing units 2, 3 and 4 for outputting a signal sequence composed of binarized information units as an output signal by receiving a signal sequence composed of binarized information units as an input signal and performing prescribed operation processing is provided with a counting means 5 for counting the number of specified information units in binary values of the output signal of the present unit 4 corresponding to one or all the signal processing units 4, and adding means 6 for adding output values from the plural counting means 5 so as to calculate the multiple additive average value.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a parallel distributed information processing apparatus aiming at artificially realizing the information processing function of a neural network, including character / image recognition, voice recognition, robot control, and image processing. The present invention relates to a signal processing device called a so-called neurocomputer, which is used for processing, natural language processing, associative memory, and the like.

[0002]

2. Description of the Related Art Recently, many attempts have been made to realize a neural network by hardware. Such hardware implementation is roughly classified into an analog type and a digital type. Among the digital type, there is a pulse density type neural network that expresses a signal by a pulse train. Such a pulse density type neural network is based on Japanese Patent Application Laid-Open No. 4-549 as a forward process, and Japanese Patent Application Laid-Open No. 4-1 as a learning process.
Various improved products have been proposed by the present applicant based on 11185. A pulse density type neural network other than the one by the present applicant is disclosed in, for example, Japanese Patent Application Laid-Open No. 1-244567 or US Pat.
Some of them are shown in the specification of 893255,
The method of signal processing differs depending on each type.

Here, signal processing in the pulse density type neural network proposed by the present applicant will be described. First, a bit train is actually used as the pulse train. That is, a train of bits (binaryized information unit) that takes "0" or "1" as a value represents a pulse train, a bit having a value of "1" represents one pulse, and a value of "0" represents a pulse. A bit indicates that there is no pulse.

The value of the signal is represented by the pulse density in the pulse train, that is, the density of bits having the value "1" in the bit train. For example, assuming that the bit string has a length of 10 bits, when a signal having a value of 0.6 is input to this neural network, 6 out of 10 bits are input to this neural network.
Bit strings of which the value is “1” and the remaining four are of the value “0” are input. Since the density never exceeds 1.0, the maximum value of the signal is 1.0.

Each neuron unit in the neural network receives as an input signal a bit string sent from outside the neural network or from some other neuron unit in the neural network. The coupling coefficient representing the strength of coupling between the neuron unit and another neuron unit is also represented by a bit string, and weighting by the coupling coefficient of the input signal is performed by taking the logical product of the input bit string and the coupling coefficient bit string. The neuron unit divides the weighted input signal into several groups, and individually calculates the logical sum of these weighted input signals for each group. Then, a predetermined logical operation is performed on the bit string of the logical sum result obtained for each group to generate a new bit string, and the new bit string is output to the outside of the neuron unit as an output signal. The output signal from such a neuron unit is sent to some other neuron unit in the neural network to be an input signal to that neuron unit, or is output outside the neural network and output from the neural network itself. Become a signal. FIG. 5 shows an image diagram of one neuron unit configuration.

Now consider the output value of such a pulse density type neural network. First, even if signals have the same pulse density, there are various pulse arrangements. Considering, for example, a signal having a value of 0.5 when the pulse length is 10, Example 1) 1010101010 Example 2) It can be different, such as 1110001001. Since the pulse arrangements are different in this way, for example, even when considering the logical product of a signal having a value of 0.5 and a signal having a value of 0.3, "1010101010"
And the logical product of "1110000000" is "10100"
00000 ", that is, the value becomes 0.2, while" 111 "
The calculation results are different so that the logical product of 001001 ”and“ 1110000000 ”becomes“ 1110000000 ”, that is, the value 0.3.

The output value of the neural network is measured by counting the number of pulses in the output signal with a counter. However, as described above, the number of pulses in the output signal varies depending on the input signal and the pulse arrangement of the coupling coefficient. The magnitude of this variation depends on the input signal, the teacher signal, and the value of the coupling coefficient. If these values are 1.0 or 0.0, all the bits in the pulse train are "1" or "0", and therefore the pulse arrangement is determined in one way and does not change. However, when learning is performed so that the output has an intermediate value between 0.0 and 1.0, for example, a value of 0.5, the fluctuation amount of the number of pulses in the output signal becomes large. In such a case, in order to know the output value of the neural network, it is necessary to measure the output value many times for the same input signal and take the average of the output pulse numbers. That is, the output value before taking the average changes greatly every time, so if the output value of one time is directly used as the output value of the neural network without taking the average value, a reliable output value cannot be obtained.

[0008]

As described above, in the case of the pulse density type neural network of the proposed example system,
In order to know the average value of the number of output pulses, it is necessary to measure the output value many times for the same input signal, which requires processing time. For example, assuming the case where the output of 100 times is averaged, assuming that the pulse train length of each signal is 127 bits and it takes 1/1000 milliseconds for the neural network to process 1 bit, the output of 100 times is measured. It will take 12.7 milliseconds to do. In this case, it cannot be applied to applications requiring high-speed response.

[0009]

A plurality of signal sequences each of which receives a signal sequence of binarized information units as an input signal, performs a predetermined arithmetic processing, and outputs a signal sequence of the binarized information units as an output signal. In a signal processing device in which a signal processing unit is combined to form a hierarchical neural network from an input layer to an output layer, in the output signal of its own unit with respect to some or all of the signal processing units. The counting means for counting the number of specific information units of the two values is provided, and the adding means for adding the output values from the plurality of counting means is provided.

[0010]

In order to know the average value of the number of output pulses of the neural network, the outputs of the neural network may be multiplexed and the multiple average thereof may be taken. That is, as the output value of the plurality of signal processing units in the output layer, the counting means counts a specific information unit of the two values, and the adding means knows the added values, so that a multiple average can be obtained. This reduces the number of measurements required to obtain high-speed response.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described with reference to FIG. First, the neural network 1 according to this embodiment is configured as a four-layer hierarchical neural network including an input layer, a first intermediate layer, a second intermediate layer, and an output layer. Here, for example, the number of neurons (signal processing unit) 2 in the input layer is 1, the number of neurons (signal processing unit) 3 in the first and second intermediate layers is 10 each, and the neurons (signal processing unit) in the output layer are further provided. ) 4 is also set to 10. The neuron 2 in the input layer is connected to the neuron 3 in the first intermediate layer. Further, the connection states of the neurons 3 in the first and second intermediate layers are omitted, but they are connected to each other in a mesh shape. Although the connection state between the neurons 3 of the second intermediate layer and the neurons 4 of the output layer is also omitted, they are connected to each other in a mesh shape.

Therefore, a counter (counting means) 5 is individually provided on the output side of each neuron 4 in the output layer. These counters 5 count the number of pulses (a specific information unit represented by a binarized representation, eg, the number of “1”) in the output signal output from each neuron 4. The outputs of these counters 5 are input to an adder circuit (adding means) 6 and used for total calculation.

In the neural network 1 having such a configuration, a value of 0.9 is given as an input signal and a value of 0.5 is given as the same teacher signal to all the neurons 4 in the output layer, and learning is performed 150 times. Length of each signal (pulse train length)
Is 127 bits. After such learning, only the forward process is set to 1 by giving the value 0.9 as the input signal.
A value obtained by dividing the output from the adder circuit 6 that executes 00 times and adds the value counted by the counter 5 for each time by the pulse train length 127 and the number 10 of the neurons 4 in the output layer (multiple average value)
Was calculated and recorded, and the standard deviation of 100 data was 0.0137.

By the way, as a comparative example, instead of the neural network 1 having the configuration shown in FIG. 1, as shown in FIG. 2, there is one neuron 4a in the output layer, and one counter 5a corresponding to this is provided. The provided neural network 1a is formed, and the number of pulses output from the neuron 4a in the output layer is counted by the counter 5a. In such a configuration, similarly to the above case, the value 0.9 is given to the neural network 1a as an input signal, and the value 0.5 is given to the neuron 4a in the output layer as a teacher signal, and the learning is performed 150 times. . Again, each signal is 12
It has a 7-bit length. After learning, the value 0.
9 is given and only the forward process is executed 10 times, and the output for each time is added by the integrated count of the counter 5a, then divided by the number of execution times 10 and the pulse train length 127 to obtain 1
The average value of the output signal values of 0 times was calculated. The operation of calculating the average value by executing the forward process 10 times was repeated 100 times, and the value of each time was recorded. The standard deviation of the 100 data thus obtained is 0.0140.

Therefore, when this embodiment is compared with this comparative example, it is found that the standard deviation is almost equal to the standard deviation of the comparative example. Therefore, by providing 10 neurons 4 in the output layer and providing the corresponding counters 5, each time the forward process is executed, the neurons 4a in the output layer 4a are executed.
When the number is one, the output of the neural network 1 can be obtained with an accuracy equivalent to the execution of the forward process 10 times, and the number of measurements required for averaging the number of pulses is reduced. It can also be applied to fields where a response is required.

In other words, in general terms, if the outputs of the neural network are multiplexed and the average between the multiple outputs is taken, it can be said that the average value can be obtained at high speed with reliable accuracy. . That is, it is necessary to take an average by increasing the number of neurons in the output layer and using the average value of the outputs of the neurons in these two or more output layers instead of the output of one neuron. The number of measurements required can be reduced. For example, instead of letting a neural network with one neuron in the output layer learn to output an output signal of a certain value, the number of neurons in the output layer is increased to 10 and all of these 10 neurons are increased. Learning is performed so that output signals of the same value are output, and the average value of the output signal values of these 10 neurons is used as the output of the neural network. By doing this, for example, 1
Instead of taking the average of the output of 100 neurons, the average of the output of 10 neurons should be taken 10 times. Therefore, the time required to measure the output value is the output layer. Ratio of the number of neurons of, ie, 1/1
It becomes 0.

Next, a second embodiment of the present invention will be described with reference to FIG. The same parts as those shown in the above-mentioned embodiment are designated by the same reference numerals. Basically, this is the same as the above-described embodiment, but in this embodiment, 10 neurons 4 in the output layer are divided into 5 groups A and B, and each group is divided into 5 groups. The difference is that two adder circuits 6A and 6B are provided. That is, the counters 5 corresponding to the groups A and B are connected to the adder circuits 6A and 6B, and are configured to add only the count values belonging to the respective groups A and B.

In such a configuration, the neural network 1 is given a value of 0.9 as an input signal, and the neurons 4 of the output layer are given a value of 0.7 as a teacher signal for those belonging to the group A, while Group B
A value of 0.3 was given as a teacher signal to the ones belonging to and learning was performed 150 times. Also in this case, each signal length is 12
7 bits. After such learning is completed, the value 0.9 is given as an input signal and only the forward process is performed by 100.
Each time, the output from the adder circuits 6A and 6B was divided by the pulse train length 127 and the number of neurons 5 in each group of the output layer, and a value was calculated and recorded for the two adder circuits 6A and 6B. The standard deviation of 100 pieces of data on the neurons 4 of the group A thus obtained is 0.0144, and the standard deviation of 100 pieces of data on the neurons 4 of the group B is 0.0148.

Incidentally, as a comparative example to this embodiment, instead of the neural network 1 having the configuration shown in FIG. 3, as shown in FIG. 4, two neurons 4 are provided in the output layer.
a, 4b are provided and divided into two groups, and a neural network 1a provided with two counters 5a, 5b corresponding to this is formed, and neurons 4a, 4b in the output layer are formed.
The number of pulses output from each of the counters 5a and 5b is separately counted. In such a configuration, similarly to the above case, the value 0.9 is given to the neural network 1a as an input signal, and the value 0.5 is given to the neuron 4a in the output layer as a teacher signal, while the neuron 4a
A value of 0.3 was given as a teacher signal to b, and learning was performed 150 times. Also in this case, each signal has a length of 127 bits.
After learning, the value 0.9 is given as an input signal, only the forward process is executed 5 times, and the output for each time is added by the integrated counts of the counters 5a and 5b, and then divided by the execution count 5 and the pulse train length 127. And each neuron 4
The average value of the output signal values of a and 4b five times was calculated. The operation of calculating the average value by executing the forward process 5 times was repeated 100 times, and the value of each time was recorded. The standard deviation of 100 data thus obtained is 0.0152 for the neuron 4a and 0.0153 for the neuron 4b.

Therefore, in the case of the present embodiment as well, when compared with this comparative example, it is found that the standard deviation is almost equal to the standard deviation of the comparative example. Therefore, the number of neurons in the output layer is 5
By doubling, the number of neurons in the output layer is 5 for each execution of the forward process.
The output of the neural network 1 can be obtained with the accuracy equivalent to the execution of the forward process once, and the number of measurements required for averaging the number of pulses is reduced, which is suitable for fields requiring high-speed response. Will also be applicable.

[0021]

As described above, according to the present invention, in order to know the average value of the number of output pulses of the neural network, it is necessary to take the outputs of the neural network as multiple outputs and take the multiple averages thereof. For some or all of the units, counting means for counting the number of specific information units of the binary in the output signal of its own unit is provided, and the output values from the plurality of counting means Since the addition means for adding is added and the average value is calculated by the multiple addition average value method, the number of measurements required to calculate the average value is reduced, and it can be applied to fields requiring high-speed response. Becomes

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a block diagram showing a comparative example.

FIG. 3 is a block diagram showing a second embodiment of the present invention.

FIG. 4 is a block diagram showing a comparative example.

FIG. 5 is an explanatory diagram conceptually showing a pulse density type neuron element configuration.

[Explanation of symbols]

 1 Neural network 2-4 Signal processing unit 5 Counting means 6, 6A, 6B Addition means

Claims (1)

[Claims] 1. A plurality of signal processing units for receiving a signal string of binarized information units as an input signal, performing a predetermined arithmetic processing, and outputting a signal string of binarized information units as an output signal. In a signal processing device in which a neural network is formed in a hierarchical manner from an input layer to an output layer by combining, a part of all of the signal processing units or all of the units have a binary value in an output signal of its own unit. Is provided with counting means for counting the number of specific information units of
A signal processing device comprising an adding means for adding output values from a plurality of the counting means.