JP6532334B2 - Parallel computing device, image processing device and parallel computing method - Google Patents

Description

  The present invention relates to a technique for performing a plurality of arithmetic processes in parallel.

  Convolution (filtering) processing is commonly known as processing commonly used in image processing, and typical processing is Gaussian filtering for smoothing, Sobel filtering for edge extraction, etc. . Another process often used in image processing is matrix multiplication. Known processes involving matrix multiplication include dimension reduction using principal component analysis (PCA), affine transformation, perceptron, support vector machines, and the like. Such filter operations and matrix product operations require a large amount of processing time since many product-sum operations need to be performed. In order to reduce the processing time using a general-purpose processor, it is necessary to prepare a plurality of processors, and there is a problem that the circuit scale becomes large.

  Therefore, from the viewpoint of circuit size and processing time, techniques for performing product-sum operations in parallel using dedicated hardware have been studied. Patent Document 1 discloses a technique for performing various types of operations by changing the connection relationship between computing units.

JP, 2009-38758, A

  However, in the technique disclosed in Patent Document 1, there is a path in which computing units are arranged in series between data input and output, and the output of one computing unit is input to another computing unit. That is, there is a problem that it is difficult to process a plurality of product-sum operations at high speed because it is necessary to go through a plurality of arithmetic units to obtain a final output. Therefore, an object of the present invention is to enable processing of a plurality of product-sum operations at high speed.

  In order to solve the above problems, the present invention provides a plurality of arithmetic processing means for performing arithmetic operations in parallel based on first data and second data, and supplying the first data to the plurality of arithmetic processing means. First supplying means for performing the processing, second supplying means for supplying the second data to the plurality of arithmetic processing means, and the first contents having different contents at the same timing for each of the plurality of arithmetic processing means Supply control for controlling the first supply unit to supply data, and controlling the second supply unit to supply the second data having the same content at the same timing to each of the plurality of arithmetic processing units A first supply mode for supplying the first data so that the first data having the same content is shared at different timings among the plurality of arithmetic processing means. , To the plurality of arithmetic processing means, a plurality of timings, and executes the second supply mode to supply the first data content are different, the.

  According to the above configuration, the present invention can process a plurality of product-sum operations at high speed.

FIG. 2 is a block diagram of a parallel operation device according to the first embodiment. FIG. 2 is a view showing an example of the arrangement of a first data supply unit 105 according to the first embodiment; 6 is a flowchart when performing a filter operation in the first embodiment. FIG. 3 is a diagram for explaining a filter kernel in the first embodiment. FIG. 5 is a schematic view of first and second data storage units in the filter operation of the first embodiment; 6 is a flowchart when performing matrix product operation in the first embodiment. FIG. 5 is a schematic view of first and second data storage units in matrix product operation of the first embodiment. A figure explaining an operation example of Deep Learning in a 2nd embodiment. The figure which shows the example of the convolution filter in filter processing. FIG. 7 is a block diagram of a parallel operation device according to a second embodiment. FIG. 7 is a block diagram of an image processing apparatus according to a second embodiment.

First Embodiment
The outline of the first embodiment of the present invention will be described first. The parallel operation device of this embodiment executes processing of a plurality of different product-sum operations. As the types of product-sum operations to be processed by this embodiment, there are product-sum operations and matrix-product operations in filter processing as described above.

  Here, the product-sum operation in the filter process will be described. FIG. 9 shows an example of a convolution filter. FIG. 6A shows an example of the case where the filter operation is performed on the image data 11 of the processing target image using a 3 × 3 filter kernel 10 with a kernel size of 3 × 3. In this example, the result of the filter operation is calculated by the product-sum operation process shown in the following Equation 1.

ここで、「ｄ_ｉ，ｊ」は座標（ｉ，ｊ）での処理対象画像画素値を示し、「ｆ_ｉ，ｊ」は座標（ｉ，ｊ）でのフィルタ演算結果を示す。また、「ｗ_ｓ，ｔ」は座標（ｉ＋ｓ−１，ｊ＋ｔ−１）に適用するフィルタカーネルの値（フィルタ係数）を示し、「ｃｏｌｕｍｎＳｉｚｅ」および「ｒｏｗＳｉｚｅ」はフィルタカーネルサイズを示す。

Here, “d _{i, j} ” indicates the processing target image pixel value at coordinate (i, j), and “f _{i, j} ” indicates the filter calculation result at coordinate (i, j). Further, “w _{s, t} ” indicates the value (filter coefficient) of the filter kernel to be applied to the coordinates (i + s−1, j + t−1), and “columnSize” and “rowSize” indicate the filter kernel size.

  In matrix multiplication operation, an m × p matrix C calculated as a result of matrix multiplication of the m × n matrix A and the n × p matrix B is expressed by Equation 2 below.

このとき行列Ｃの要素「ｃ_ｉ，ｊ」は、次の数式３で算出される。

At this time, the element “c _{i, j} ” of the matrix C is calculated by the following Equation 3.

このように、一般的に行われる画像処理では、数式１や数式３の形で表わされる積和演算がよく使用される。ここで、フィルタ演算も行列積演算もどちらも積和演算（乗算結果を順次加算する演算）ではあるが、フィルタ演算では、同一のフィルタカーネルに対して、フィルタされる側のデータ（スキャンウインドウ内の画像データ）が部分的に重複する場合がある。つまり、フィルタ演算では、一部共通するデータを使用することがある。

As described above, in image processing generally performed, a product-sum operation represented in the form of Formula 1 or Formula 3 is often used. Here, although both the filter operation and the matrix product operation are product-sum operations (operations in which the multiplication results are sequentially added), in the filter operation, the data on the filter side of the same filter kernel (in the scan window Image data) may partially overlap. That is, in the filter operation, some common data may be used.

  FIGS. 9B to 9D show that data in the filter calculation partially overlap. FIGS. 9B to 9D are schematic views when the filter kernel 10 performs filter processing on the image data 11 to be processed. When the filter kernel 10 is at the position shown in FIG. 9B and at the position shown in FIG. 9C, the image data of the hatched portion in FIG. 9C overlap. Similarly, when the filter kernel 10 is in the position shown in FIG. 9C and in the position shown in FIG. 9D, the image data of the portion shown by oblique lines in FIG. 9D overlap. . Thus, in the filter operation, data (image data) to be filtered may partially overlap data of the same filter kernel.

  On the other hand, in matrix product operation, for one row vector of one matrix (for example, row vector of matrix A of equation 2), column vector of the other matrix (for example, column vector of matrix B of equation 2) There is no partial overlap in, and there is no data common to multiple column vectors. That is, even when the product-sum operation is performed with a row vector of one matrix and a plurality of column vectors of the other matrix, there is no data that overlaps the plurality of column vectors.

  As described above, when one of the operand data of the product-sum operation is fixed, there is a difference between the other operand data whether or not there is an overlapping portion between the filter operation and the matrix product operation. In this embodiment, since different types of product-sum operations are performed using the same parallel operation device, the data supply mode (supply order) is provided when data to be operated on is supplied to operation processing units existing in parallel. A plurality of product-sum operations can be processed by switching. In this embodiment, as an example of the product-sum operation, an example in which both the filter operation and the matrix product operation are processed by a single parallel operation device will be described.

  Hereinafter, the details of the present embodiment will be described with reference to the drawings. FIG. 1 is a block diagram of a parallel processing device according to the present embodiment. Operation target data to be subjected to product-sum operation is input to the parallel operation device 101. When the filter operation is performed, image data and filter kernel data are input as operation target data. When performing a matrix multiplication operation, two matrix data to be multiplied are input as operation target data. The parallel computing device 101 executes a product-sum operation on these operation target data, and outputs the operation result.

  When operation target data of a product-sum operation to be performed from now on is input to the parallel operation device 101, the operation target data is stored in the first data storage unit 102 and the second data storage unit 103. When the filter operation is performed, the image data is stored in the first data storage unit 102, and the filter kernel data is stored in the second data storage unit 103. When matrix multiplication operation is performed, one of the matrix data to be multiplied (for example, the matrix A of Formula 2) is stored in the first data storage unit 102, and the other of the matrix data to be multiplied (for example, the matrix B of Formula 2) is 2 stored in the data storage unit 103. In the present embodiment, the first data storage unit 102 and the second data storage unit 103 are configured by a RAM (Random Access Memory), but are configured by other means such as a register file (a collection of a plurality of registers). It may be

  The data supply control unit 104 supplies the first data (data output from the first data storage unit 102) having different contents to the plurality of arithmetic processing units 107 at the same timing. At the same time, the second data (data output from the second data storage unit 103) having the same content is supplied to each of the plurality of arithmetic processing units 107 at the same timing. The data supply control unit 104 performs all operations on the same second data as the first data supply unit 105 that supplies each of the plurality of first data to the corresponding operation processing unit 107 in order to realize such a function. A second data supply unit 106 that supplies the processing unit 107 is included. The first data supply unit 105 temporarily holds the first data output from the first data storage unit 102 and supplies the first data to the arithmetic processing unit 107. The first data supply unit 105 of the present embodiment is configured of a shift register capable of loading data.

  FIG. 2 is a diagram showing a configuration example of the first data supply unit 105, and shows four stages of shift registers. The shift register is provided with four multi-bit registers 801a to 801d, which latch data of a predetermined bit in synchronization with the CLOCK signal. An enable signal (Enable signal) is given to the registers 801a to 801d, and when the Enable signal is 1, the registers 801a to 801d latch data at the rising edge of the CLOCK signal. On the other hand, when the Enable signal is 0, the data latched by the previous clock is held as it is. Therefore, when the Enable signal is 0, no transition occurs in the state of data latched by the registers 801a to 801d.

  In addition, three selectors 802a to 802c are provided, which select the signal OUTx (x: 0 to 2) when the selection signal (Load signal) is 0, and the selection signal (Load signal) is 1 Select the signal INx (x: 1 to 3). That is, the selectors 802a to 802c select the shift operation or the load operation according to the Load signal.

  The bit width of the multi-bit registers 801 a to 801 d may be the same as the bit width of the image data or matrix data stored in the first data storage unit 102 (for example, 8 bits). In addition, the number of stages of the shift register may be “the number of operation processing units 107 (the number in parallel)” + “the horizontal size of the filter kernel” −1 ”. For example, when the number of arithmetic processing units 107 is four and the horizontal size of the filter kernel is three pixels, 4 + 3-1 = 6 stages may be set. However, when it is assumed that filter operations are performed with filter kernels of various sizes, it is desirable to configure the number of stages with respect to the horizontal size of the maximum filter kernel that is assumed.

  In addition, an operation type signal from the operation type switching unit 110 is input to the first data supply unit 105. The first data supply unit 105 can supply data in a plurality of modes, and switches the mode of data supply according to the input operation type signal. The data supply mode referred to here corresponds to what order and timing the data loaded from the first data storage unit 102 is to be supplied to the arithmetic processing unit 107.

  For example, when an instruction to perform a filter operation is input to the first data supply unit 105 as an operation type signal, the first data supply unit 105 may change the timing when the same data is different for each of the plurality of operation processing units 107. Operates as supplied by That is, it operates in a data supply mode in which data supplied to the arithmetic processing unit 107 at a certain timing is supplied to another arithmetic processing unit 107 at the next timing. In this case, the plurality of arithmetic processing units 107 share the same supplied data at different timings.

  On the other hand, when the instruction to perform matrix multiplication operation is input to the first data supply unit 105 as the operation type signal, the first data supply unit 105 does not supply the same data to the plurality of operation processing units 107. To work. That is, it operates in a data supply mode in which data supplied to the arithmetic processing unit 107 which is a certain timing is not supplied to another arithmetic processing unit 107 at the same timing or another timing. In this case, sharing of data supplied between the plurality of arithmetic processing units 107 is not performed.

  The second data supply unit 106 temporarily holds the second data output from the second data storage unit 103 and supplies the second data to the arithmetic processing unit 107. The second data supply unit 106 of the present embodiment is configured of a register. The bit width of this register may be the same as the bit width of the filter kernel and matrix data stored in the second data storage unit 103 (for example, 8 bits).

  The arithmetic processing unit 107 performs product-sum operation using the first data supplied from the first data supply unit 105 and the second data supplied from the second data supply unit 106, and outputs the product-sum operation result. Do. The parallel operation device 101 includes a plurality of operation processing units 107 in order to perform product-sum operations in parallel. Different first data are supplied at the same timing from the first data supply unit 105 to each of the plurality of arithmetic processing units 107, and at the same time, the same second data is supplied from the second data supply unit 106 at the same timing. Be done. Then, the arithmetic processing unit 107 includes a multiplier 108 and a cumulative adder 109 in order to execute a product-sum operation on the supplied first data and second data.

  The operation type switching unit 110 outputs an operation type signal based on the operation type setting set from the outside. For example, when the parallel processing device is incorporated in the image processing device, the operation type according to the image processing to be performed by the image processing device is set. If the operation to be performed from now on is a filter operation, information to that effect is set in the operation type switching unit 110, and the operation type switching unit 110 transmits the first data supply unit 105 and the read control unit 111 to the filter operation. Give instructions to operate in the operating mode. Further, when the operation to be performed from now on is matrix multiplication operation, information to that effect is set in the operation type switching unit 110, and the operation type switching unit 110 causes the first data supply unit 105 and the read control unit 111 to perform matrix multiplication. It instructs to operate in the operation mode for calculation.

  When the first data supply unit 105 reads data from the first data storage unit 102, the read control unit 111 controls the read mode. The read mode here corresponds to the order in which the data stored in the first data storage unit 102 is read by the first data supply unit 105.

  In addition, an operation type signal is input to the read control unit 111 from the operation type switching unit 110. When an operation type signal instructing the filter operation to the read control unit 111 is input, the read control unit 111 causes the first data storage unit 102 to transmit a part of the data read out for the product-sum operation in the past for the filter operation. Set the mode to repeat and read again. That is, data including a part of data read from the first data storage unit 102 for product-sum operation performed in the past in the process of a series of filter operations is read for another product-sum operation. Set the read mode as above.

  On the other hand, when an operation type signal indicating that matrix product operation is instructed to the read control unit 111 is input, the read control unit 111 causes the data read for the product-sum operation in the past for the matrix product operation to overlap again. Set to a mode that can not be read out. That is, the read control unit 111 sets an operation mode such as reading new data or reading the same data as the previous calculation.

FIG. 3 is a flowchart showing the processing procedure when performing the filter operation by the parallel operation device of the present embodiment. First, in step S301, the setting of the operation type of the operation type switching unit 110 is performed. Here, filter operation is set as the operation type of the operation type switching unit 110. Subsequently, in step S302, the filter kernel data is stored in the second data storage unit 103. FIG. 4 is a diagram for explaining a filter kernel in the present embodiment. The figure shows a filter kernel 10 and image data 11 to be calculated. The data of each pixel of the filter kernel 10 is represented by w _{i, j} and the data of each pixel of the image data 11 is represented by d _{i, j} (i, j is an index indicating a coordinate position). The filter kernel 10 and the image data 11 are held, for example, outside the parallel operation device of the present embodiment (such as storage means of an image processing device incorporating the parallel operation device) before the start of operation.
Here, processing in which the filter kernel data is stored in the second data storage unit 103 in step S302 described above will be described. FIG. 5 is a diagram showing how data is stored in the first data storage unit 102 and the second data storage unit 103 in the filter operation, and FIG. 5A shows data storage in the second data storage unit 103. It shows the situation of The second data storage unit 103 of this embodiment is configured by a RAM, and in step S302, filter kernel data is stored in the area of each address as shown in FIG. 5A.

Subsequently, in step S303, target image data of the arithmetic processing is stored in the first data storage unit 102. FIG. 5B shows how data is stored in the first data storage unit 102. The first data storage unit 102 of this embodiment is configured by a RAM, and in step S303, the image data 11 is stored in the RAM in raster order as shown in FIG. 5B. That is, the data of d _1,1 to d _1,4 are stored in the area of the address ₁ , and the data of d _1,5 to d _1,8 are stored in the area of the address 2. Also, data of d _2,1 to d _2,4 is stored in the area of address p, data of d _3,1 to d _3,4 is stored in the area of address q, d 4,1 in the area of address r _. Data of _{1 to} d ₄ , ₄ are stored.

  Here, the shape (width) of the RAM constituting the first data storage unit 102 will be described. Generally, even if the capacity of the RAM is the same, if the width is wide and the depth is narrow, the circuit scale becomes larger than the width is narrow and the depth is deep. It is desirable from the viewpoint of processing speed that the first data storage unit of the present embodiment has a width such that the number of data equal to the number of processing units 107 (degree of parallel processing) can be read at one time. For example, if the degree of parallelism is 4, it is preferable that the RAM has a width sufficient to store four pieces of data at the same address as shown in FIG. 5 (B).

  Next, in step S304, data is output from the first data storage unit 102 to the first data supply unit 105. The first data supply unit 105 according to the present embodiment is formed of a shift register, and temporarily holds data output from the first data storage unit 102. Specifically, first, data at address 1 and address 2 are output from the first data storage unit 102 and held by the first data supply unit 105. The read control of data from the first data storage unit 102 is performed by the read control unit 111. Here, FIG. 5C shows a schematic view of a shift register which is the first data supply unit 105. As shown in FIG. In the same figure, since the number of stages of the shift register (here, six) is wider than the width of the first data storage unit 102 (for four data in this embodiment), the data (two words) read from two addresses is used. Loading is done.

  Subsequently, in step S305, data is output from the second data storage unit 103 to the second data supply unit 106. The second data supply unit 106 of the present embodiment is configured by a register, and temporarily holds the data output from the second data storage unit 103. FIG. 5D is a schematic view of a shift register which is the second data supply unit 105. As shown in the figure, the data at address 1 is output from the second data storage unit 103 and held by the second data supply unit 106. In the subsequent step S306, product-sum operations are performed in parallel in each operation processing unit 107.

  Then, the product-sum operation is continued by repeatedly performing operations in steps S307, S308, and S309. That is, by simultaneously outputting the next kernel data from the second data storage unit 103 while shifting the shift register of the first data supply unit 105, product-sum operations are performed in parallel in each operation processing unit 107. (Horizontal multiply-accumulate filter kernel).

  Similarly, the product-sum operation is continued by repeatedly performing operations in steps S310 and S311, and the product-sum operation in the horizontal direction of the filter kernel is repeated a number of times corresponding to the size in the vertical direction to obtain the final result.

  The filter operation of this embodiment is executed by the above processing flow. In addition, what is necessary is just to carry out from step S304, when performing the next horizontal filter calculation. The same applies to the case of performing the filter operation at other positions. In particular, when performing the next horizontal filter operation, in step S304, the read control unit 111 causes the first data storage unit 102 to use part of the data read out for the product-sum operation in the past of the filter operation. Control to read the data again so as to overlap. For example, in the previous product-sum operation, it is assumed that the data stored in address 1, address 2, address p, address p + 1, address q, and address q + 1 is read. On the other hand, in the present product-sum operation, the data stored in the address p, the address p + 1, the address q, the address q + 1, the address r, and the address r + 1 is read out. In this case, part (4 words) of the previously read data (6 words) is redundantly included. Such duplication of partial data also occurs when the next horizontal filter operation is performed. Thus, when the filter operation is designated as the operation type, the read control unit 111 operates as follows. That is, for a certain product-sum operation performed in the process of a series of filter operations, another product-sum operation can be performed on data that includes a portion of the data read from the first data storage unit 102 in an overlapping manner. To operate in a read mode such as

  In addition, the first data supply unit 105 supplies the held data to the arithmetic processing unit 107 while shifting. That is, when the filter operation is instructed as the operation type, the first data supply unit 105 operates to supply the same data to different operation processing units 107 at different timings.

  Subsequently, a process flow when performing matrix multiplication operation by the parallel operation device 101 according to the present embodiment will be described. Here, the case where the product-sum operation of Equation 2 described above is performed will be described. FIG. 6 is a flowchart showing a processing procedure when performing a matrix multiplication operation by the parallel operation device of this embodiment.

  First, in step S601, the setting of the operation type is performed on the operation type switching unit 110. Here, matrix product operation is set as the operation type. Next, in step S602, the matrix B is stored in the second data storage unit 103. FIG. 7 is a diagram showing how data is stored in the first data storage unit 102 and the second data storage unit 103 in matrix multiplication operation, and FIG. 7A shows data in the second data storage unit 103. It shows the state of storage. The second data storage unit 103 of the present embodiment is configured by a RAM, and as shown in FIG. 7A, each element data of the matrix B is stored in the area of each address.

  Subsequently, in step S603, the matrix A is stored in the first data storage unit 102. FIG. 7B shows how data is stored in the first data storage unit 102. The first data storage unit 102 of the present embodiment is configured by a RAM, and as shown in FIG. 7B, data of the matrix A is stored in the RAM.

  Subsequently, in step S604, data is output from the first data storage unit 102 to the first data supply unit 105. First, data at address 1 is output from the first data storage unit 102 and held by the first data supply unit 105. The read control of data from the first data storage unit 102 is performed by the read control unit 111. FIG. 7C is a schematic view of a shift register corresponding to the first data supply unit 105. As shown in FIG. Although the number of stages of the shift register (here, six stages) is wider than the width (for four data) of the first data storage unit 102, sharing of data is not performed by different operation processing units 107 in matrix product operation. That is, since the shift operation is not performed in the shift register, it is only necessary to store data for the number of the arithmetic processing units 107 (four data in this embodiment) in the shift register.

  Subsequently, in step S605, data is output from the second data storage unit 103 to the second data supply unit 106. The second data supply unit 106 of the present embodiment is configured by a register, and temporarily holds the data output from the second data storage unit 103. FIG. 7D is a schematic view of a register corresponding to the second data supply unit 106. First, the data at address 1 is output from the first data storage unit 103, and the output data is held by the second data supply unit 106.

  Subsequently, in step S606, each arithmetic processing unit 107 performs product-sum operation in parallel. Then, the product-sum operation is continued by repeatedly performing operations in steps S 607, S 608, and S 609. That is, when the next data is output from the first data storage unit 102, the next kernel data is output from the second data storage unit 103, whereby the product-sum operations are performed in parallel in each operation processing unit 107. . A final result is obtained by performing a product-sum operation for a predetermined number of times (n times, which is the number of column dimensions of the matrix A in this embodiment).

  The matrix product operation according to the present embodiment is executed by the above process flow. In addition, what is necessary is just to re-execute from step S604, when performing another matrix product calculation of water parallel of the matrix A. The same applies to the case of performing matrix multiplication at other positions. When performing another horizontal parallel matrix product operation of the matrix A, in step S604, the read control unit 111 causes the first data storage unit 102 to read the data read for the product-sum operation in the past of the matrix product operation. It operates to read out the data following. That is, while the data stored in address 1, address 2, ..., address n was read in the previous product-sum operation, in the current product-sum operation, address n + 1, address n + 2, ..., address 2n Read out the data stored in. As described above, when data in the horizontal row having the matrix A is read, there is no overlap between the previously read data and the currently read data.

  Also, when performing matrix multiplication operation of another vertical column of the matrix B, it may be executed again from step S604. In this case, in step S604, the read control unit 111 operates to read the same data as the read data for the product-sum operation in the past of the matrix product operation to the first data storage unit 102. However, the data read from the second data storage unit in step S605 is different from the data used for the product-sum operation in the past. For example, it is assumed that the data stored in address 1, address 2, ..., address n is read from the first data storage unit 102 and the second data storage unit 103 in the past product-sum operation. However, although the first data storage unit 102 reads the same data this time, the second data storage unit 103 reads the data stored in the address n + 1, the address n + 2,..., The address 2n. As described above, when the matrix multiplication operation is designated as the operation type, the read control unit 111 operates as follows. That is, for a certain product-sum operation performed in the past in the process of a series of matrix product operations, data different from the data read from the first data storage unit 102 or the same data is subjected to another product-sum operation. To operate in a read mode such as

  As described in the process flow of matrix product operation described above, the first data supply unit 105 supplies data to the operation processing unit 107 without performing shift processing on the held data. Therefore, when matrix product operation is instructed as the operation type, data sharing is not performed between the plurality of operation processing units 107. Further, when the data read from the second data storage unit 103 is the same as the data in the past product-sum operation, the read control unit 111 operates as follows. That is, it operates in a read mode in which data different from a certain product-sum operation executed in the past in the process of a series of matrix product operations is read out for another product-sum operation.

  As described above, in the filter operation, the parallel operation device according to the present embodiment partially shares the image data of the operation target among the plurality of operation processing units 107 with respect to the same filter kernel. On the other hand, in matrix product operation, the same column vector of matrix B does not share the data of the row vector of matrix A. According to the difference of such arithmetic processing, a plurality of modes of data supply from the first data supply unit 105 to the arithmetic processing unit 107 are prepared, and the first data storage unit 102 to the first data supply unit 105. A plurality of data read modes are also prepared. And, by switching these modes, it is possible to execute different types of product-sum operations. As a result, according to the present embodiment, it is possible to process a plurality of product-sum operations at high speed without having to go through a plurality of arithmetic units to obtain a final output.

Second Embodiment
Next, a second embodiment of the present invention will be described. In the present embodiment, in addition to the functions of the parallel processing device of the first embodiment described above, a non-linear conversion processing unit is added. The components already described in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  In image recognition processing, non-linear transformation processing is often performed on filter operation results and product-sum operation results. For example, in CNN (Convolutional Neural Networks), it is common to perform sigmoid conversion (or hyperbolic tangent conversion) on filter operation (convolution operation). It is also common to perform multiclass classification of input data by applying a softmax function to the result of perceptron which can be expressed by matrix product. Therefore, by providing the non-linear conversion processing unit in the parallel computing device, it becomes possible to cope with more flexible processing, and the usefulness as a parallel computing device is increased.

  The present embodiment will be described by way of an example in which a parallel computing device performs Deep Learning processing. Deep Learning is a technical field in which research and development are actively conducted. Generally, hierarchical processing (the processing result of a certain hierarchy is given to the upper hierarchy of input data (for example, image data)). It refers to the one that performs the process). Here, as a typical Deep Learning, a configuration in which CNN is used for feature quantity extraction processing from an image, and matrix multiplication such as that represented by a perceptron is used for identification processing using the extracted feature quantity An example in which the operation is performed by the parallel operation device will be described. This feature value extraction process is often a multi-layer process in which CNN is repeated many times, and a multi-layer perceptron of all combinations may be used as the identification process.

  Here, an operation example of Deep Learning will be described using FIG. FIG. 8 shows processing for performing feature extraction on the input layer (input image) 801 by CNN, acquiring the feature amount 807, performing identification processing based on the feature amount, and obtaining an identification result 814. . CNN is repeated many times (here, three times) until the feature amount 807 is acquired from the input image 801. In addition, perceptron processing of all bonds is performed on the feature amount 807, and a final identification result 814 is obtained.

  First, the CNN process in the first half will be described. In FIG. 8, an input layer 801 indicates raster-scanned image data of a predetermined size when performing CNN operation on image data. The feature planes 803a to 803c show the feature planes of the first tier 808. The feature surface is a data surface indicating the detection result of a predetermined feature extraction filter (convolution filter operation and non-linear processing). Since the detection result is for the raster scanned image data, the detection result is also represented by a plane. The feature planes 803 a to 803 c are generated by convolution filter operation and non-linear processing on the input layer 801. For example, the feature plane 803a is obtained by convolution filter operation using the filter kernel 8021a and nonlinear conversion of the operation result. The filter kernels 8021 b and 8021 c in FIG. 8 are filter kernels used when generating the feature planes 803 b and 803 c, respectively.

  Next, an operation for generating the feature plane 805a of the second tier 809 will be described. The feature surface 805a is coupled to the three feature surfaces 803a to 803c of the previous layer 808. Therefore, when calculating data of the feature plane 805a, a convolution filter operation using a kernel indicated by a filter kernel 8041a is performed on the feature plane 803a, and the result is held. Similarly, the convolution filter operation of the filter kernels 8042a and 8043a is performed on the feature planes 803b and 803c, respectively, and these results are held. After completion of these three types of filter operations, the respective results are added to perform non-linear conversion processing. The feature plane 805a is generated by processing the above processing for the entire image.

  Similarly, when generating the feature plane 805b, three convolution filter operations are performed by the filter kernels 8041b, 8042b, and 8043b with respect to the feature planes 803a to 803c of the previous stage layer 808. When generating the feature plane 807 of the third tier 810, two convolution filter operations are performed on the feature planes 805a to 805b of the previous tier 809 by the filter kernels 8061 and 8062.

  Next, the second half perceptron processing will be described. FIG. 8 shows a two-level perceptron. The perceptron is a non-linear transformation of the weighted sum for each element of the input feature quantity. Therefore, it is possible to obtain a result 813 by performing matrix multiplication on the feature amount 807 and performing non-linear transformation on the result. Furthermore, the final identification result 814 can be obtained by repeating the same process.

  FIG. 10 is a block diagram of the parallel processing device of this embodiment. The parallel processing device 101 includes an operation result storage unit 1001 and a non-linear conversion processing unit 1002 in addition to the configuration described in the first embodiment. Note that the operation type switching unit 110 according to the present embodiment outputs a non-linear conversion switching signal, and whether or not non-linear conversion processing is to be executed by the non-linear conversion switching signal. , And instructs the non-linear conversion processing unit 1002. When instructing to perform nonlinear conversion by the nonlinear conversion switching signal, the parallel processing device 101 outputs the product-sum operation result subjected to nonlinear conversion as a final operation result. If the processing of non-linear transformation is set in the operation type switching unit 110, the parallel computing device 101 can output a result other than the non-linear transformation, and outputs the product-sum operation result as in the first embodiment, for example. be able to.

  The operation result storage unit 1001 has a function of temporarily storing a plurality of product-sum operation results output from the plurality of operation processing units 107 and sequentially outputting the result to the non-linear conversion processing unit 1002. The calculation result storage unit 1001 is configured of a loadable shift register or the like.

  The non-linear conversion processing unit 1002 performs non-linear conversion processing such as sigmoid conversion on the input product-sum operation result. The non-linear transformation is realized by using a look-up table stored in the storage means of the parallel computing device. A nonlinear conversion switching signal is input to the nonlinear conversion processing unit 1002 from the operation type switching unit 110, and various nonlinear conversion processes are executed by rewriting the look-up table based on the input nonlinear conversion switching signal. When an instruction not to perform non-linear conversion is input by the non-linear conversion switching signal, the non-linear conversion processing by the non-linear conversion processing unit 1002 is skipped.

  In the parallel computing device 101 according to the present embodiment, the product-sum operation result calculated by the operation processing unit 107 is once stored in the operation result storage unit 1001, and then input to the non-linear conversion processing unit 1002. This is because, in general, the non-linear conversion processing unit 1002 has a large circuit size, and it is often difficult to prepare a plurality. In order to sequentially process a plurality of product-sum operation results in one non-linear conversion processing unit 1002, a configuration is provided in which the parallel operation device 101 is provided with the operation result storage unit 1001. However, as long as the non-linear transformation processing unit 1002 can be prepared in the same number as the operation processing unit 107, the operation result storage unit 1001 is not necessarily required.

  Subsequently, an image processing apparatus according to the present embodiment provided with the above-described parallel processing apparatus 101 will be described. FIG. 11 is a block diagram of the image processing apparatus 200 of the present embodiment. The image processing apparatus 200 includes an image input unit 20, a parallel computing device 101, a bridge 24, a preprocessing unit 25, a direct memory access controller (DMAC) 26, and a RAM 21. Furthermore, a CPU (Central Processing Unit) 27, a ROM 28, and a RAM 29 are provided. The image input unit 20, the parallel arithmetic device 101, the preprocessing unit 25 and the DMAC 26 are connected to one another via the image bus 23, and the CPU 27, the ROM 28 and the RAM 29 are connected to one another via the CPU bus 30. The bridge 24 enables data transfer between the image bus 23 and the CPU bus 30.

  The image input unit 20 is configured of an optical system, a photoelectric conversion device such as a CCD (Charge-Coupled Devices) or a Complementary Metal Oxide Semiconductor (CMOS) sensor, or the like. Furthermore, a driver circuit that controls the sensor, an AD converter, a signal processing circuit that manages various image corrections, a frame buffer, and the like are also provided. The image input unit 20 may process image data input from an apparatus other than a camera or a medium, or image data stored in advance in the image processing apparatus 200 as a target image. The parallel computing device 101 executes hierarchical CNN operations and matrix product operations. The RAM 21 is used as an operation work buffer of the parallel operation device 101.

  The pre-processing unit 25 performs various pre-processing for effectively performing the image recognition processing. For example, image data conversion processing such as color conversion processing and contrast correction processing is processed by hardware. The DMAC 26 manages data transfer between the image input unit 20 on the image bus 23, the parallel arithmetic device 101 and the preprocessing unit 25, and the CPU bus 30. A ROM (Read Only Memory) 28 stores instructions and parameter data that define the operation of the CPU 27. The CPU 27 controls the overall operation of the image processing apparatus 200 while reading these. At this time, the RAM 29 is used as a work area of the CPU 27. The CPU 27 can also access the RAM 21 on the image bus 23 through the bridge 24.

  As described above, Deep Learning is a process of extracting feature quantities by repeating CNN several layers, performing identification processing based on the extracted feature quantities, and obtaining a final result. When executing Deep Learning in the image processing apparatus 200 of the present embodiment, first, CNN is set as the calculation type setting for the parallel calculation apparatus 101. CNN is an operation that applies nonlinear processing (here, sigmoid conversion) to the result of the convolution filter operation. Therefore, the operation type switching unit 110 outputs information called filter operation as an operation type signal and outputs information called sigmoid conversion as a non-linear switching signal.

  The details of the process of the parallel computing device 101 when the information of the filter operation is output as the operation type signal have already been described in the first embodiment, and thus are omitted here. The filter operation result calculated as in the first embodiment is output to the operation result storage unit 1001, subjected to non-linear transformation, and the result is output. By repeating such filter operation with non-linear processing while changing the filter kernel, extraction of feature quantities (vector data) by Deep Learning is performed.

  Subsequently, identification processing is performed on the output feature amount (vector data). In this case, identification processing is set to the parallel computing device 101 as computation type setting. Since the identification process performed in the present embodiment is an operation that performs non-linear processing (here, soft max conversion) on the result of matrix product operation using feature amounts, the operation type signal from operation type switching unit 110 Information called matrix multiplication operation is output as Further, information called soft max conversion is output as the non-linear switching signal.

  The details of the processing of the parallel arithmetic device 101 when the information of matrix product operation is output as the operation type signal have already been described in the first embodiment, and thus are omitted here. The result of the matrix product operation calculated as in the first embodiment is output to the operation result storage unit 1001, and the identification result is output by performing nonlinear conversion on it. The identification result is used to obtain the final result (for example, the category of the object present in the input image).

  As described above, the image processing apparatus 200 according to the present embodiment includes the parallel processing apparatus 101, and can perform Deep Learning processing configured by various calculations with a single parallel processing apparatus.

Other Embodiments
In the above description, although the example in which the first data supply unit 105 is configured by the shift register has been described, the first data supply unit 105 of the present invention is not limited to the shift register. If it is a means that can supply the same data to different arithmetic processing units 107 at different timings, and can switch permission / prohibition that the same data is supplied to a plurality of arithmetic processing units 107 according to the operation type signal, The configuration can be adopted as the configuration of the first data supply unit 105. For example, the outputs of a plurality of registers can be selected by a selector. In this case, by sequentially switching control signals of the selector, the same operation as the shift register can be performed. Further, by fixing the signal of the selector, control can be performed so that the same data is not supplied to the plurality of arithmetic processing units 107.

  Further, the present invention may be applied to a system constituted by a plurality of devices or to an apparatus comprising a single device. The present invention is not limited to the above embodiments, and various modifications (including organic combinations of the respective embodiments) are possible based on the spirit of the present invention, which are excluded from the scope of the present invention is not. That is, the configuration in which each of the above-described embodiments and their modifications are combined is also included in the present invention.

101 Parallel Arithmetic Unit 102 First Data Storage Unit 103 Second Data Storage Unit 104 Data Supply Control Unit 105 First Data Supply Unit 106 Second Data Supply Unit 107 Arithmetic Processing Unit 108 Multiplier 109 Cumulative Adder 110 Calculation Type Switching Unit 111 Read control unit

Claims (12)

A plurality of arithmetic processing means for performing arithmetic operations in parallel based on the first data and the second data;
First supply means for supplying the first data to the plurality of arithmetic processing means;
Second supply means for supplying the second data to the plurality of arithmetic processing means;
The first supply means is controlled to supply the first data having different contents at the same timing to the plurality of arithmetic processing means, and the contents are the same at the same timing for the plurality of arithmetic processing means Supply control means for controlling the second supply means to supply the second data of
Have
The supply control means is
A first supply mode for supplying the first data so that the first data having the same content is shared at different timings among the plurality of arithmetic processing means;
A second supply mode in which the plurality of arithmetic processing means are supplied with the first data having different contents at a plurality of timings;
A parallel operation device characterized by performing. First storage means for storing the first data;
And a read control unit that controls reading of the first data from the first storage unit to the first supply unit;
The read control means
A first read mode in which a portion of the first data read from the first storage means in the past calculation is read again in duplicate;
A second read mode for reading the first data read from the first storage unit in the past calculation without duplication;
The parallel operation device according to claim 1, characterized in that It further comprises a second storage unit for storing the second data,
The supply control means is
The second supply unit is controlled to receive the second data read from the second storage unit and to supply the second data having the same content at the same timing to each of the plurality of arithmetic processing units. The parallel operation device according to claim 2, characterized in that: The first supply means comprises a shift register,
The first supply means is
In the first supply mode, the first data read from the first storage unit is loaded, and the loaded first data is supplied to the plurality of arithmetic processing units while being shifted a predetermined number of times,
In the second supply mode, the first data read from the first storage unit is loaded, and the loaded first data is supplied to the plurality of arithmetic processing units without shifting. The parallel operation device according to item 3. The apparatus further comprises operation type switching means for controlling the supply control means and the read control means according to the type of operation.
The operation type switching means is
When the first operation is performed, the supply control means is caused to execute the first supply mode, and the read control means is caused to execute the first read mode.
When the second operation is performed, the supply control means is caused to execute the second supply mode, and the read control means is caused to execute the second read mode.
The parallel computing device according to any one of claims 2 to 4, characterized in that:   The parallel operation device according to claim 5, wherein the first operation is a filter operation.   7. The parallel operation device according to claim 5, wherein the second operation is a matrix product operation. It further comprises conversion processing means for performing non-linear conversion on the calculation results of each of the plurality of calculation processing means,
The parallel operation device according to any one of claims 5 to 7, wherein the operation type switching unit instructs the conversion processing unit whether or not to perform non-linear conversion according to the type of the operation.   9. The parallel operation device according to claim 8, wherein the operation type switching unit instructs a type of the non-linear conversion when the conversion processing unit performs the non-linear conversion.   10. The parallel operation device according to any one of claims 1 to 9, wherein the operation processing means includes a multiplier and a cumulative adder. A parallel operation device according to any one of claims 1 to 10,
An image processing apparatus characterized by performing image processing to be processed using the parallel operation device. A plurality of operation processing means performing an operation in parallel based on the first data and the second data;
Supplying the first data from the first supply means to the plurality of arithmetic processing means;
Supplying the second data from the second supply means to the plurality of arithmetic processing means;
The first supply means is controlled to supply the first data having different contents at the same timing to the plurality of arithmetic processing means, and the contents are the same at the same timing for the plurality of arithmetic processing means Controlling the second supply means to supply the second data of
A first supply mode for supplying the first data so that the first data having the same content is shared at different timings among the plurality of arithmetic processing means;
A second supply mode in which the plurality of arithmetic processing means are supplied with the first data having different contents at a plurality of timings;
A parallel operation method characterized by performing.

Images