KR102143034B1 - Method and system for tracking object in video through prediction of future motion of object - Google Patents

Abstract

Disclosed is a method and system for tracking an object in a moving picture through prediction of an object's future motion. The object tracking method includes: extracting a feature of an object included in the still image and surrounding information of the object from a still image corresponding to one frame of a moving picture; Predicting a future motion of the object from the extracted features; And performing tracking of the object by generating a search area in a next frame based on the prediction result of the future motion.

Description

Method and system for object tracking in video through prediction of future motion of object {METHOD AND SYSTEM FOR TRACKING OBJECT IN VIDEO THROUGH PREDICTION OF FUTURE MOTION OF OBJECT}

The description below relates to a technology for tracking objects in a video.

Study on understanding and expression of motion for objects in many visual and visual problems such as optical flow, action recognition, future frame prediction, and video compression. Has become.

For example, Korean Patent Application Publication No. 10-2012-0106279 (published on September 26, 2012) discloses a technology for predicting motion in an image signal based on a block matching technology.

Most of the existing technologies model motion by finding a region that minimizes the difference in pixel values in adjacent frames. This is a model that is approximated by a brightness consistency constraint that most of the image acquisition equipment is a frame-based camera, and the pixel values of the same object are similar, so it does not reach human cognitive ability.

On the other hand, humans quickly and accurately predict the future motion of objects despite being still images. It can be said that this is because humans comprehensively judge the information of objects and their peripheries in the process of recognizing motion.

On the other hand, most of the existing tracking technologies determine that the object has moved to the position with the highest matching score through an exhaustive search of the surrounding area centering on the object detection result.

Since the existing tracking technology analyzes adjacent frames to predict the movement of objects, the tracking performance (matching accuracy, speed aspect, etc.) is variable according to the frame rate of the input image, and the input image suddenly occurs due to communication delay. In the event of a frame drop, stable tracking results cannot be guaranteed.

It provides a method and system for stably tracking an object by predicting the future motion of an object from a single still image.

Provides a method and system that can prevent deterioration of tracking performance and ensure stable tracking when a video or frame drop with a significantly low frame rate occurs.

Provides a method and system for efficiently reducing an object search area by predicting future motion of an object in one frame, which is a single still image, and searching for an object in the next frame based on the prediction result.

An object tracking method performed in a computer system, the method comprising: extracting an object included in the still image and a feature of surrounding information of the object from a still image corresponding to one frame of a moving picture; Predicting a future motion of the object from the extracted features; And generating a search area in a next frame based on the prediction result of the future motion to perform tracking of the object.

According to one aspect, the predicting comprises predicting an instance-level future motion in which at least one of a direction, a velocity, and an action of the object as each class, and performing the May perform tracking of the object by first searching for a region having a high movement probability in the next frame in response to at least one of a direction, a speed, and an action predicted as a future movement of the object.

According to another aspect, the performing may include reducing the search area according to a speed predicted by a future motion of the object.

According to another aspect, the performing may include determining a sector-shaped search area corresponding to a direction predicted by the future movement of the object.

According to another aspect, in the performing step, tracking of the object may be performed after correcting a background movement around the object.

According to another aspect, the predicting may include predicting a group direction of each cluster by clustering a direction predicted by a future motion of each object included in the still image.

According to another aspect, in the extracting, the feature may be extracted through a learning model learned by integrating the object included in the training image and surrounding information of the object.

According to another aspect, in the extracting, the feature is extracted through a learning model obtained by learning images to which at least one of a direction, a speed, and an action representing a future movement is assigned as correct answer data for an object instance. I can.

According to another aspect, the extracting includes an object feature for an object region in the still image through a plurality of region of interest (RoI) pooling layers constituting a learning model, and a peripheral region including the object. A local feature for, and a global feature for the entire area of the image may be extracted.

According to another aspect, the predicting may include receiving the output of the RoI pooling layer through a fully connected layer and a softmax layer constituting the learning model to determine the future motion of the object. You can predict at least one of the direction, speed, and behavior you represent.

It provides a computer-readable recording medium, characterized in that a program for executing the object tracking method on a computer is recorded.

A computer system comprising at least one processor embodied to execute a command readable by a computer, wherein the at least one processor includes an object included in the still image in a still image corresponding to one frame of a moving picture and the object A feature extraction unit for extracting a feature of the surrounding information; A motion prediction unit predicting a future motion of the object from the extracted features; And an object tracking unit for tracking the object by generating a search area in a next frame based on a prediction result of the future motion.

According to embodiments of the present invention, an object can be stably tracked by predicting a future motion of an object in a single still image.

According to embodiments of the present invention, since future motion of an object can be predicted from a single still image, even when a video or frame drop having a significantly low frame rate occurs, it is possible to prevent a decrease in tracking performance and ensure stable tracking.

According to embodiments of the present invention, an object search area can be efficiently reduced by predicting a future motion of an object in one frame, which is a single still image, and searching for an object in a next frame based on a prediction result.

1 is a block diagram illustrating an example of an internal configuration of a computer system according to an embodiment of the present invention.
2 is a diagram showing an example of components that may be included in a processor of a computer system according to an embodiment of the present invention.
3 is a flow chart illustrating an example of an object tracking method that can be performed by a computer system according to an embodiment of the present invention.
4 is a view for explaining an example of an image that can be used as a training image in an embodiment of the present invention.
5 illustrates an example of a speed class and an action class for movement in an embodiment of the present invention.
6 illustrates a DNN structure for future motion prediction according to an embodiment of the present invention.
7 shows a table for explaining the classification performance of a DNN including an MCRoI pooling layer.
FIG. 8 is an exemplary diagram for explaining a process of generating a search area for object tracking by using a future motion prediction result according to an embodiment of the present invention.
9 illustrates an example of a crowd analysis result using a future motion prediction result in an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention relate to a technology for tracking an object in a moving picture through a method of predicting a future motion of an object in a single still image.

Embodiments including those specifically disclosed in this specification can stably track an object even if a video or frame drop with a significantly low frame rate occurs, and through this, significant in terms of efficiency, accuracy, speed, cost reduction, etc. Achieve the advantages.

1 is a block diagram illustrating an example of an internal configuration of a computer system according to an embodiment of the present invention. For example, an object tracking system according to embodiments of the present invention may be implemented through the computer system 100 of FIG. 1. As shown in FIG. 1, the computer system 100 is a component for executing an object tracking method, and includes a processor 110, a memory 120, a permanent storage device 130, a bus 140, and an input/output interface 150. ) And a network interface 160.

The processor 110 may include or be part of any device capable of processing a sequence of instructions as a component for tracking an object in a video. The processor 110 may include, for example, a computer processor, a processor in a mobile device or other electronic device, and/or a digital processor. The processor 110 may be included in, for example, a server computing device, a server computer, a series of server computers, a server farm, a cloud computer, a content platform, and the like. The processor 110 may be connected to the memory 120 through the bus 140.

The memory 120 may include volatile memory, permanent, virtual, or other memory for storing information used or output by the computer system 100. The memory 120 may include, for example, random access memory (RAM) and/or dynamic RAM (DRAM). The memory 120 can be used to store arbitrary information, such as status information of the computer system 100. Memory 120 may also be used to store instructions of computer system 100, including instructions for object tracking, for example. Computer system 100 may include one or more processors 110 as needed or appropriate.

The bus 140 may include a communication infrastructure that enables interaction between various components of the computer system 100. The bus 140 may carry data, for example, between components of the computer system 100, for example between the processor 110 and the memory 120. The bus 140 may include wireless and/or wired communication media between components of the computer system 100, and may include parallel, serial or other topology arrangements.

Persistent storage device 130 is a component such as memory or other permanent storage device used by computer system 100 to store data for a predetermined extended period of time (eg, compared to memory 120). It may include. The permanent storage device 130 may include non-volatile main memory as used by the processor 110 in the computer system 100. The permanent storage device 130 may include, for example, a flash memory, hard disk, optical disk, or other computer readable medium.

The input/output interface 150 may include interfaces to a keyboard, mouse, voice command input, display, or other input or output device. Input for configuration commands and/or object tracking may be received through the input/output interface 150.

The network interface 160 may include one or more interfaces to networks such as a local area network or the Internet. Network interface 160 may include interfaces for wired or wireless connections. Input for configuration commands and/or object tracking may be received through the network interface 160.

Also, in other embodiments, the computer system 100 may include more components than those in FIG. 1. However, there is no need to clearly show most prior art components. For example, the computer system 100 is implemented to include at least some of the input/output devices connected to the input/output interface 150 described above, or a transceiver, a global positioning system (GPS) module, a camera, various sensors, Other components, such as a database, may also be included.

The present invention proposes a learning model (for example, a deep neural network (DNN)) that integrates various levels of semantic information in order to simulate the human-level motion prediction capability, and through this, it is possible to predict the future motion of the instance level in a single image. have.

In the present specification, an embodiment of predicting a future movement of a corresponding object by using a pedestrian as a representative object is described, but the limitation may be applied to other types of objects such as automobiles and animals.

FIG. 2 is a diagram showing an example of components that can be included in a processor of a computer system according to an embodiment of the present invention, and FIG. 3 is an object tracking that can be performed by a computer system according to an embodiment of the present invention. It is a flowchart showing an example of a prediction method.

As shown in FIG. 2, the processor 110 may include a feature extraction unit 210, a motion prediction unit 220, and an object tracking unit 230. The components of the processor 110 may be expressions of different functions performed by the processor 110 according to a control command provided by at least one program code. For example, the feature extraction unit 210 may be used as a functional expression operated to control the computer system 100 so that the processor 110 extracts a feature of an object from a still image. The processor 110 and components of the processor 110 may perform steps S310 to S340 included in the object tracking method of FIG. 3. For example, the processor 110 and components of the processor 110 may be implemented to execute instructions of the operating system code included in the memory 120 and at least one program code described above. Here, at least one program code may correspond to a code of a program implemented to process an object tracking method.

The object tracking method may not occur in the illustrated order, and some of the steps may be omitted or an additional process may be further included.

In step S310, the processor 110 may load a program code stored in a program file for an object tracking method into the memory 120. For example, a program file for an object tracking method may be stored in the permanent storage device 130 described with reference to FIG. 1, and the processor 110 may use a program from a program file stored in the persistent storage device 130 through a bus. Computer system 110 can be controlled so that code is loaded into memory 120. In this case, each of the feature extraction unit 210, the motion prediction unit 220, and the object tracking unit 230 included in the processor 110 and the processor 110 is a corresponding part of the program code loaded into the memory 120 It may be different functional expressions of the processor 110 for executing the subsequent steps S320 to S340 by executing the command of. For the execution of steps S320 to S340, the processor 110 and components of the processor 110 may process an operation according to a direct control command or control the computer system 100.

The object tracking method according to the present invention includes the steps of extracting an object region from a single still image, which is one frame of an input video, through object detection (S320); Predicting a next direction, speed, and behavior of the corresponding object in a predetermined class through prediction of a future motion of the object region (S330); And adaptively generating a search area in a next frame based on a result of predicting a future motion of one frame and performing tracking of an object in the corresponding area (S340).

In step S320, the feature extraction unit 210 may receive a single still image corresponding to one frame of the moving picture as an input image and extract a feature related to an object from the input image. In particular, the feature extraction unit 210 extracts features of blocks of various sizes from the input image, that is, an object area, a local context including an object, and a global context. You can combine all of the features of.

In step S330, the motion prediction unit 220 may predict the future motion of the corresponding object from the features extracted in step S320. The motion prediction unit 220 may predict a future motion of an object by dividing it into three types of motions, namely, direction, velocity, and action. The motion prediction unit 220 may predict a future motion at an instance level in which a direction, a speed, and a behavior are as classes in a single image. For example, the direction can be divided into eight directions (N, E, S, W, NE, SE, SW, NW), which can be based on coordinates in the image. In addition, the speed can be divided into a plurality of stages, for example, three stages: stop, slow, and fast. Behavior is variable depending on the situation, but when the autonomous driving system recognizes a pedestrian as an object, the behavior of the pedestrian can be classified into, for example, sidewalk, crosswalk, and jaywalk. .

In step S340, the object tracking unit 230 may generate a search area for the object based on the future movement of the object predicted in step S330, and perform tracking within the corresponding area. The object tracking unit 230 may preferentially search for a region having a high probability of moving a corresponding object in a next frame based on a result predicted as a future motion of an object in a single still image that is a previous frame. The object tracking unit 230 may stably perform object tracking without deteriorating speed performance by changing the shape of a search point for an object according to a result of predicting a future motion of the object.

The future motion of an object can be predicted from a single still image, so it can reliably track an object even in a video with a frame rate of 1 fps or less, and an abrupt frame drop occurs while autonomous vehicles or robots recognize the surrounding environment. Even so, it is possible to prevent deterioration of object tracking performance and predict the behavior of surrounding objects in advance.

In the present invention, it is possible to predict the future motion of an instance with the direction, speed, and behavior of an object as each class in a single image, and proposes a DNN model that integrates various levels of semantic information in order to predict the future motion.

Referring to FIG. 4, a human can predict the next movement of the object 401 included in the image 400 to some extent when looking at a still image 400. The present invention intends to use DNN to implement human-level motion prediction capability for future motion.

Hereinafter, a specific embodiment will be described using a pedestrian instance, which is the object of interest in many vision applications, as a representative example.

It is possible to collect a future motion dataset by selecting an image containing a pedestrian as a training image on an existing dataset or an image database system on the Internet. At this time, a bounding box annotation is applied to the pedestrian area included in the training image, and then three properties of the future movement (i.e., direction, speed, and behavior) are manually assigned to each pedestrian instance as correct answer data. .

The direction class may be defined in eight directions (N, E, S, W, NE, SE, SW, NW). And, referring to FIG. 5, the speed class 510 may be defined as stop, slow, or fast, and the action class 520 may be defined as general walking, crosswalk walking, and stepless walking. This is only an example, and direction, speed, and behavior class may be defined differently depending on the case. For each training image, each class of direction, speed, and behavior of the pedestrian instance is manually labeled.

In general, when humans see a moving object, they simultaneously perceive visual information about the object and its surroundings. Similarly, the present invention proposes a multi-context region of interest (MCRoI) pooling layer integrating objects, local and global features in order to utilize a scene context for future motion prediction. By integrating the MCRoI pooling layer into a densely connected convolutional network (DenseNet), we learn a unified model to predict the future direction, velocity, and behavior of instances.

In the present invention, a DNN is constructed to predict future movements of pedestrians by performing three classification tasks for direction, speed, and behavior.

6 shows a DNN structure for future motion prediction.

Referring to FIG. 6, the DNN 600 processes an input image patch, in which the input image patch is located in the center of a pedestrian and generates three classification results.

The DNN 600 is included in the processor 110 of the computer system 100 and may include a feature extraction unit 210 and a motion prediction unit 220.

It is assumed that from the training image in which a bounding box of height h is annotated with respect to the pedestrian area 61, a patch cut to a size of 2h×2h based on the bounding box of the pedestrian area 61 is used as the input image 60.

The feature extractor 210 may include a convolutional layer and an MCRoI pooling layer. In this case, the convolution layer may be composed of DenseNet.

The feature extraction unit 210 generates features of the pedestrian area 61 corresponding to the object area, the peripheral area 62 including the object, and the entire image area 63 based on DenseNet. That is, the feature extraction unit 210 extracts an object feature, a local context feature, and a global context feature as an output of DenseNet.

The feature extraction unit 210 includes an MCRoI pooling layer using four RoI pooling layers.

The bounding box annotated for the pedestrian area 61 corresponds to the RoI of the object feature pooled at a spatial resolution of 7×7. At this time, the pedestrian area 61 corresponding to the object area includes appearance information of the pedestrian with a minimum background.

The peripheral region 62 is a patch cut to a size of 1.3h×1.3h based on the bounding box, and corresponds to RoI for two local context features. The patches are pooled at resolutions 9×9 and 11×11 by two RoI pooling layers, respectively. Local context features include background information around the pedestrian and the pedestrian appearance.

A 2h×2h patch corresponding to the entire image area 63 corresponds to an RoI for a global context feature and is pooled at a resolution of 5×5. The global context feature is important in predicting future movements because it provides overall semantic information about the scene.

The output sizes of the four RoI pooling layers are determined empirically.

The motion prediction unit 220 may be composed of a fully connected layer and a softmax layer. In particular, the motion prediction unit 220 receives the output of the MCRoI pooling layer, processes it into three stages of a fully connected layer, and then processes it into three softmax layers to classify the direction, speed, and behavior of pedestrians, respectively.

The learning process of DNN is as follows.

The MCRoI pooling layer uses 4 pooling layers to connect the convolutional layer and the fully connected layer.

The DNN 600 of FIG. 6 may apply an input patch of any size, but includes a pedestrian area 61 having a size of 200 pixels by standardizing the size of the input patch to 400 x 400 for effective learning and inference. .

The total loss function is defined as the sum of the three losses as shown in Equation 1.

[Equation 1]

Here, L _Dir is a loss for direction classification, L _Vel is a loss for speed classification, and L _Act is a loss for behavior classification.

Cross entropy is applied for each classification loss.

The direction classification loss is defined as in Equation 2.

[Equation 2]

는 8방향에 대한 소프트맥스 확률 벡터이고,

는 실측 결과(ground-truth) 바이너리 벡터를 의미한다.here,

Is the softmax probability vector for 8 directions,

Denotes a ground-truth binary vector.

L _Vel and L _{Act are} also defined similarly to Equation 2.

In this embodiment, the DNN 600 of FIG. 6 is trained through stochastic gradient descent in which a batch size is 4 for 0.9 momentum and 14 epochs. As an initial parameter, the DenseNet model learned in advance in ImageNet is used.

Therefore, in the present invention, it is possible to implement a network model that integrates and learns the object and surrounding information through the MCRoI pooling layer, and through this, the future movement (direction, speed, and behavior) of the object at the instance level with only a single stationary image. Can be predicted.

7 shows a table for explaining the classification performance of a DNN including an MCRoI pooling layer.

The table in FIG. 7 shows the performance of the'Proposed' network, which is a network structure including the MCRoI pooling layer, in an existing network (a'Object' network using object features for classification, a'Local context' network using local context features for classification, This is a comparison with the'Global context' network, which uses global context features for classification, and the'Object+Local' network, which uses object features and local context features for classification).

In FIG. 7,'Direction+' considers ambiguity when it is difficult to quantify the actual direction because even a human sees one image and judges it as a different class.If the prediction direction is the same as the actual measurement result or the direction adjacent to it is regarded as accurate It means the accuracy when it becomes.

The'Proposed' network including the MCRoI pooling layer predicts future motion by combining all three types of features: object features, local context features, and global context features, so it can be seen that overall performance is better than existing networks. have.

Instance-level motion prediction using direction, speed, and behavior as classes has the following advantages over pixel-level motion prediction in utilizing deep learning: (1) Pixel-based annotations require a lot of time and cost, so DNN There is a side in which it is difficult to secure enough data to learn. (2) Object-level motion modeling is more similar to the process of human recognition and understanding of motion, and it is easy to integrate with various computer vision technologies such as detection and tracking.

In an embodiment of the present invention, using a DNN learned by integrating various levels of semantic information, it is possible to accurately and efficiently predict the future motion of an object in a single still image and provide reliable prediction performance for future motion. .

The above-described future motion prediction algorithm can make object tracking more efficient.

For multiple objects as well as a single object between consecutive frames, the search area for tracking the corresponding object can be reduced by using the future motion prediction result for each object.

If the search area of the target object is reduced using the future motion prediction result, the object can be tracked more quickly and efficiently.

FIG. 8 is a comparison of (a) a search area 81 of an existing tracking method and (b) a search area 801 of a tracking method using a future motion prediction result.

In the past, a reference line is sampled from a Gaussian distribution to select a search target within the search area 81 corresponding to a rectangular window, whereas in the tracking method according to the present invention, the search area 801 is based on a future motion prediction result. Can be reduced. In this case, the search area 801 may be determined according to a direction, speed, and behavior predicted as future motion. For example, as the speed predicted by the future motion increases, the size of the search area 801 may increase, and as the speed decreases, the size of the search area 801 may decrease. For example, the object tracking unit 230 may apply a search area 801 that is four times smaller than the existing search area 81 or a search area having a predetermined size when the speed of the future movement is'stop'. As another example, the object tracking unit 230 may determine a sector-shaped search area 801 corresponding to a direction of a future movement.

In the present invention, since a search area for object tracking can be reduced, a tracking speed can be improved by searching a smaller area than before.

In order to track multiple objects in a video with a low frame rate, sophisticated tracking can be performed using a tracking-by-detection method. For example, for multi-object tracking, a search area may be reduced based on a result of predicting a future motion of each object and a sampling density may be increased.

Since the future motion of the object is predicted as the relative motion of the object in relation to the background information around the object, if the background moves, the reduction of the search area using the future motion may be invalid. Accordingly, in the present invention, object tracking may be performed after first correcting the background movement using an algorithm such as key point matching and affine transformation.

In addition to object tracking, it is also possible to apply a future motion prediction algorithm to crowd analysis of a single image.

For example, the motion prediction unit 220 may predict the group direction of each cluster after dividing the crowd into multiple clusters using the predicted directions of each pedestrian in the crowd. At this time, the K-mean algorithm is used for clustering. To calculate the distance between two instances, the weighted distance in Equation 3 can be used.

[Equation 3]

는 인스턴스들 사이의 유클리드 거리이고,

은 방향적 지수의 주기적인 차이를 의미한다. 예를 들어, 인접한 방향 사이의

은 1이고 반대 방향에서 최대

은 4이다. 또한,

은 가중 파라미터를 의미한다.here,

Is the Euclidean distance between instances,

Means the periodic difference of the directional index. For example, between adjacent directions

Is 1 and maximum in the opposite direction

Is 4. Also,

Means weighting parameter.

After clustering, the group direction of each cluster is acquired, and for this, the sum of the direction probabilities of instances in the cluster may be calculated for each direction. In this case, the probability vector may be generated by the motion prediction unit 220 of the DNN 600 of FIG. 6, and a direction in which the sum of probabilities is the maximum may be determined as the group direction of the corresponding cluster.

FIG. 9A shows a result of predicting the movement direction of each pedestrian, and FIG. 9B shows a result of predicting the movement direction of a clustered group through clustering. As shown in the image shown in (b) of FIG. 9, the direction information of pedestrians may be compressed based on data clustering and provided as a result of crowd analysis.

As described above, according to embodiments of the present invention, it is possible to predict the future motion of an object in a single still image, so that even when a video or frame drop having a significantly low frame rate occurs, it is possible to prevent a decrease in tracking performance and ensure stable tracking. Further, according to embodiments of the present invention, an object search area can be efficiently reduced by predicting a future motion of an object in one frame, which is a single still image, and searching for an object in a next frame based on a prediction result.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments are a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable gate array (PLU). It may be implemented using one or more general purpose computers or special purpose computers, such as a logic unit), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may perform an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the art, the processing device may include a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include. For example, the processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and/or data may be embodied in any type of machine, component, physical device, computer storage medium, or device in order to be interpreted by the processing device or to provide instructions or data to the processing device. have. The software may be distributed on networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may continuously store a program executable on a computer or may be temporarily stored for execution or download. Further, the medium may be a variety of recording means or storage means in a form in which a single or several pieces of hardware are combined, but is not limited to a medium directly connected to a computer system, but may be distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks, And program instructions including ROM, RAM, flash memory, and the like. Also, examples of other media include an application store for distributing applications, a site for distributing or distributing various software, and a recording medium or storage medium managed by a server.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description by those of ordinary skill in the art. For example, the described techniques are performed in a different order than the described method, and/or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the claims to be described later.

Claims (12)

In the object tracking method performed in a computer system,
Extracting a feature of an object included in the still image and surrounding information of the object from a still image corresponding to one frame of a moving picture;
Predicting a future motion of the object from the extracted features; And
Performing tracking of the object by generating a search area in the next frame based on the prediction result of the future motion
Including,
The extracting step,
Extracting an object feature for an object region, a local feature for a surrounding region including the object, and a global feature for the entire image region from the still image
Object tracking method, characterized in that. The method of claim 1,
The predicting step,
Predicting the future motion of the instance level with at least one of the direction, velocity, and action of the object as each class,
The performing step,
Performing tracking of the object by first searching for a region with a high probability of movement in the next frame in response to at least one of a direction, speed, and behavior predicted by the future movement of the object
Object tracking method, characterized in that. The method of claim 1,
The performing step,
Reducing the search area according to the speed predicted by the future movement of the object
Object tracking method comprising a. The method of claim 1,
The performing step,
Determining a sector-shaped search area corresponding to a direction predicted by the future movement of the object
Object tracking method comprising a. The method of claim 1,
The performing step,
Compensating for background movement around an object and then tracking the object
Object tracking method, characterized in that. The method of claim 1,
The predicting step,
Predicting a group direction of each cluster by clustering a direction predicted by a future motion of each object included in the still image
Object tracking method comprising a. The method of claim 1,
The extracting step,
Extracting the feature through a learned learning model by integrating the object included in the training image and the surrounding information of the object
Object tracking method, characterized in that. The method of claim 1,
The extracting step,
Extracting the feature through a learning model that learns images to which at least one of a direction, a velocity, and an action representing a future motion is assigned as correct answer data for an object instance
Object tracking method, characterized in that. The method of claim 1,
The extracting step,
Extracting the object feature, the local feature, and the global feature through a plurality of RoI (region of interest) pooling layers constituting a learning model
Object tracking method, characterized in that. The method of claim 9,
The predicting step,
Predicting at least one of a direction, speed, and behavior representing the future movement of the object by receiving the output of the RoI pooling layer through a fully connected layer and a softmax layer constituting the learning model. that
Object tracking method, characterized in that. A computer-readable recording medium having a program recorded thereon for executing the object tracking method of any one of claims 1 to 10 on a computer. In the computer system,
At least one processor implemented to execute computer-readable instructions
Including,
The at least one processor,
A feature extraction unit for extracting features of an object included in the still image and surrounding information of the object from a still image corresponding to one frame of a moving picture;
A motion prediction unit predicting a future motion of the object from the extracted features; And
An object tracking unit for tracking the object by generating a search area in the next frame based on the prediction result of the future motion
Including,
The feature extraction unit,
Extracting an object feature for an object region, a local feature for a surrounding region including the object, and a global feature for the entire image region from the still image
Computer system characterized in that.

Images