KR101772820B1 - Method for generating a thermal image and electronic device performing the same - Google Patents

Abstract

The present invention relates to a thermal image generating method and a device for performing the same. The thermal image generating method according to an embodiment of the present invention comprises the steps of: acquiring a high-resolution visible light image and a low-resolution thermal image corresponding to each other; detecting a boundary line of a specific object in the high-resolution visible light image; and generating a high-resolution thermal image using the low-resolution thermal image and a result of the boundary detection. The steps of generating the high-resolution thermal image include: determining boundary pixels of the low-resolution thermal image corresponding to the boundary line; and determining a boundary pixel group of the high-resolution thermal image corresponding to the boundary pixels. Temperature values of the rest pixels of the high-resolution thermal image which excludes the boundary pixel group are determined based on the temperature values of pixels of the low-resolution thermal image corresponding to the rest pixels Temperature values of the inside pixels of the specific object which includes pixels corresponding to the boundary line in the boundary pixel group are determined based on temperature values of the pixels inside the specific object, which are adjacent to the boundary pixels of the pixels of the low-resolution thermal image. Temperature values of outside pixels of the specific object in the boundary pixel group are determined based on temperature values of the pixels outside the specific object, which are adjacent to the boundary pixels of the pixels of the low-resolution thermal image.

Description

[0001] METHOD FOR GENERATING A THERMAL IMAGE AND ELECTRONIC DEVICE PERFORMING THE SAME [0002]

The present invention relates to a thermal image generating method and an electronic apparatus performing the same, and more particularly, to a thermal image generating method for generating a low-resolution thermal image as a high-resolution thermal image with a clear borderline by using a high- .

A thermal image is an image of the temperature of a target object. In general, a thermal image can be obtained by imaging an object of interest to be recognized and analyzed by using an infrared sensor or the like that detects infrared rays emitted by an object having a temperature.

Thermal images are widely used in various industrial fields that are difficult to apply visible light images due to problems such as illumination. In particular, thermal imaging technology has been used to determine overheating and gas leaks that can cause failures and accidents in various electrical and mechanical equipment. It has been used to identify poor insulation in buildings and is used in automotive, aircraft, It has been used to assisting visual capabilities by showing objects that are difficult to visually judge during the nighttime operation of various transportation means.

However, the infrared sensor, which is the core of the thermal image technology, has a lower resolution than the conventional visible image and the high-resolution infrared image sensor is very expensive and difficult to realize a high resolution thermal image. Recently, a low cost infrared sensor There is a growing demand for technologies that can produce high quality thermal images.

An aspect of the present invention is to provide a thermal image generating method for generating a high-resolution thermal image from a low-resolution thermal image by using a visible light image, and a device for performing the same.

It is to be understood that the present invention is not limited to the above-described embodiments and that various changes and modifications may be made without departing from the spirit and scope of the present invention as defined by the following claims .

According to an aspect of the present invention, there is provided a method of manufacturing a liquid crystal display, comprising: obtaining a high-resolution visible light image and a low-resolution thermal image corresponding to each other; Detecting a boundary line of a specific object from the high-resolution visible light image; And generating a high resolution thermal image using the low resolution thermal image and the boundary detection result, wherein the step of generating the high resolution thermal image includes: determining a boundary pixel of the low resolution thermal image corresponding to the boundary line ; And determining a boundary pixel group of the high resolution thermal image corresponding to the boundary pixel, wherein the temperature value of the remaining pixels excluding the boundary pixel group among the pixels of the high resolution column image, Wherein a temperature value of an inner pixel of the specific object including a pixel corresponding to the boundary line of the boundary pixel group is determined based on a temperature value of a pixel of the low resolution thermal image, Wherein a temperature value of an outer pixel of the specific one of the boundary pixel groups is determined based on a temperature value of a pixel located inside the adjacent specific object, Which is determined based on the temperature value of the pixel located outside of the thermal image generating room This can be provided.

According to another aspect of the present invention, there is provided a method of generating a visible light image, the method comprising: acquiring a visible light image and a low-resolution thermal image corresponding to the visible light image and having a lower resolution than the visible light image; Detecting a boundary line in the visible light image; And generating a high-resolution thermal image having a higher resolution than the low-resolution thermal image, the low-resolution thermal image being composed of high-resolution pixels obtained by dividing low-resolution pixels of the low-resolution thermal image using the boundary detection result and the temperature data of the low- Wherein the step of generating the temperature data comprises the step of calculating the temperature value of the high resolution pixel belonging to the low resolution pixel corresponding to the pixel representing the boundary line in the visible light image to the temperature data of another low resolution pixel adjacent to the low resolution pixel to which the high resolution pixel belongs A method of generating a thermal image may be provided.

According to another aspect of the present invention, there is provided a visible light sensor module for capturing a high-resolution visible light image including a specific object; A thermal image camera for capturing a low-resolution thermal image including the specific object; And generating a high resolution thermal image by using the low resolution thermal image and the boundary detection result to determine a boundary pixel of the low resolution thermal image corresponding to the boundary line, Resolution thermal image corresponding to the boundary pixel, and the temperature value of the remaining pixels excluding the boundary pixel group among the pixels of the high-resolution column image is determined as the temperature of the pixel of the low resolution column image corresponding to the remaining pixel And the temperature value of the inner pixel of the specific object including the pixel corresponding to the boundary line of the boundary pixel group is positioned on the inner side of the specific object adjacent to the boundary pixel of the low resolution thermal image Based on the temperature value of the pixel High, the temperature values of the pixels outside of the boundary of the pixel group wherein the specific object is a module for determining based on the temperature value of the pixel located at the outer side of the specific object adjacent to the boundary pixel of the pixels of the low resolution image column; And a thermal image processing apparatus according to the present invention.

It is to be understood that the solution of the problem of the present invention is not limited to the above-mentioned solutions, and the solutions which are not mentioned can be clearly understood by those skilled in the art to which the present invention belongs It will be possible.

It is an object of the present invention to obtain a high-resolution thermal image having improved quality from a low-resolution thermal image by using a boundary line extracted from a visible light image.

The effects of the present invention are not limited to the effects described above, and the effects not mentioned can be clearly understood by those skilled in the art from the present specification and the accompanying drawings.

Figure 1 is a block diagram illustrating a thermal image generation system 1000 in accordance with an embodiment of the present invention.
FIG. 2 is a view showing the visible light sensor module 1200 according to an embodiment of the present invention in more detail.
3 is a view showing a visible light image of an object according to an embodiment of the present invention.
4 is a diagram illustrating the infrared sensor module 1200 according to an embodiment of the present invention in more detail.
5 is a view showing low-resolution temperature map data for an object according to an embodiment of the present invention.
6 is a diagram illustrating a low-resolution thermal image for a subject according to an embodiment of the invention.
7 is a diagram illustrating a high resolution thermal image for a subject according to an embodiment of the invention.
8 is a view showing a column image displaying only the V value among H, S, and V for an object according to an embodiment of the present invention.
FIG. 9 is a diagram showing the image processing apparatus 1300 according to an embodiment of the present invention in more detail.
10 is a diagram showing an operation included in a thermal image generating operation (S1000) according to an embodiment.
FIG. 11 is a diagram showing a result of detecting pixels corresponding to a boundary line in map data according to an embodiment of the present invention.
12 is a view showing low-resolution temperature map data and high-resolution temperature map data according to an embodiment of the present invention.
13 is a diagram illustrating a method of setting a temperature value of a pixel by applying extrapolation in temperature map data according to an embodiment of the present invention.
14 is a diagram illustrating a method of setting a temperature value of a pixel by applying interpolation in temperature map data according to an embodiment of the present invention.
15 is a pixel definition diagram illustrating definitions of pixels and / or groups of pixels in low-resolution temperature map data and high-resolution temperature map data according to attributes of pixels and / or groups of pixels according to an embodiment of the present invention.
16 is a diagram showing pixel addresses in low-resolution temperature map data and high-resolution temperature map data according to an embodiment of the present invention.
17 is a diagram showing pixel addresses of a high-resolution thermal image and a low-resolution thermal image according to an embodiment of the present invention.
FIG. 18 is a diagram showing pixels and an object in low-resolution temperature map data corresponding to a boundary line according to an embodiment of the present invention.
FIG. 19 is a diagram showing pixels and an object in high-resolution temperature map data corresponding to a boundary line according to an embodiment of the present invention. FIG.
20 is a diagram illustrating a method of setting a temperature value of a boundary pixel group of high-resolution temperature map data according to an embodiment of the present invention.
21 is a diagram illustrating a method of setting a temperature value by extrapolating boundary pixel groups of high-resolution temperature map data according to an embodiment of the present invention.
22 is a diagram illustrating a method of setting a temperature value of a pixel group of high-resolution temperature map data according to an embodiment of the present invention.
23 is a diagram illustrating a method of setting a temperature value by interpolating boundary pixel groups of high-resolution temperature map data according to an embodiment of the present invention.
24 is a flowchart showing a method of generating temperature map data in an embodiment of the present invention.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to be illustrative of the present invention and not to limit the scope of the invention. Should be interpreted to include modifications or variations that do not depart from the spirit of the invention.

Although the terms used in the present invention have been selected in consideration of the functions of the present invention, they are generally used in general terms. However, the present invention is not limited to the intention of the person skilled in the art to which the present invention belongs . However, if a specific term is defined as an arbitrary meaning, the meaning of the term will be described separately. Accordingly, the terms used herein should be interpreted based on the actual meaning of the term rather than on the name of the term, and on the content throughout the description.

The drawings attached hereto are intended to illustrate the present invention easily, and the shapes shown in the drawings may be exaggerated and displayed as necessary in order to facilitate understanding of the present invention, and thus the present invention is not limited to the drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a detailed description of known configurations or functions related to the present invention will be omitted when it is determined that the gist of the present invention may be obscured.

According to an aspect of the present invention, there is provided a method of manufacturing a liquid crystal display, comprising: obtaining a high-resolution visible light image and a low-resolution thermal image corresponding to each other; Detecting a boundary line of a specific object from the high-resolution visible light image; And generating a high resolution thermal image using the low resolution thermal image and the boundary detection result, wherein the step of generating the high resolution thermal image comprises: determining a boundary pixel of the low resolution thermal image corresponding to the boundary line step; And determining a boundary pixel group of the high resolution thermal image corresponding to the boundary pixel, wherein the temperature value of the remaining pixels excluding the boundary pixel group among the pixels of the high resolution column image, Wherein a temperature value of an inner pixel of the specific object including a pixel corresponding to the boundary line of the boundary pixel group is determined based on a temperature value of a pixel of the low resolution thermal image, Wherein a temperature value of an outer pixel of the specific one of the boundary pixel groups is determined based on a temperature value of a pixel located inside the adjacent specific object, Which is determined based on the temperature value of the pixel located outside of the thermal image generating room This can be provided.

According to another aspect of the present invention, there is provided an image processing method comprising the steps of: obtaining a visible light image and a low-resolution thermal image corresponding to the visible light image but having a lower resolution than the visible light image; Detecting a boundary line in the visible light image; And generating a high-resolution thermal image having a higher resolution than the low-resolution thermal image, the low-resolution thermal image being composed of high-resolution pixels obtained by dividing low-resolution pixels of the low-resolution thermal image using the boundary detection result and the temperature data of the low- Wherein the step of generating the temperature data comprises the step of calculating the temperature value of the high resolution pixel belonging to the low resolution pixel corresponding to the pixel representing the boundary line in the visible light image to the temperature data of another low resolution pixel adjacent to the low resolution pixel to which the high resolution pixel belongs A method of generating a thermal image including the steps of:

Also, in the determining, the adjacent other low-resolution pixels may be located on the same side as the high-resolution pixel with respect to the boundary line.

Further, in the determining step, the adjacent other low-resolution pixels may be located in an area to which the high-resolution pixel defined by the boundary line belongs.

In addition, the boundary line is associated with a specific object, and in the determining step, the high-resolution pixel and the adjacent low-resolution pixel may be located within the specific object.

Further, the temperature value can be calculated by extrapolating using the temperature data of the other low resolution pixels.

The generating may include determining a temperature value of the high resolution pixel belonging to the low resolution pixel that does not correspond to a pixel representing the boundary line in the visible light image using the temperature data of the low resolution pixel to which the high resolution pixel belongs Step < / RTI >

The temperature value of the high-resolution pixel belonging to the low-resolution pixel which does not correspond to the pixel representing the boundary line can be calculated by interpolation using the temperature data of the low-resolution pixel.

The method may further include determining at least one of color elements of a predetermined color space based on the temperature value and displaying the high resolution image.

The color space may include at least one of RGB system, HSV system, CMYK system, CIE system, PMS system, Munsell color system, NCS system, and YIQ system.

The method may further include displaying the detected boundary line on the high resolution image.

The high-resolution thermal image may have the same resolution as the visible light image.

The method may further include displaying at least one of the visible light image, the low resolution image, and the high resolution image.

According to still another aspect of the present invention, there is provided a display device including: a visible light camera for capturing a high-resolution visible light image including a specific object; A thermal image camera for capturing a low-resolution thermal image including the specific object; And generating a high resolution thermal image by using the low resolution thermal image and the boundary detection result to determine a boundary pixel of the low resolution thermal image corresponding to the boundary line, Resolution thermal image corresponding to the boundary pixel, and the temperature value of the remaining pixels excluding the boundary pixel group among the pixels of the high-resolution column image is determined as the temperature of the pixel of the low resolution column image corresponding to the remaining pixel And the temperature value of the inner pixel of the specific object including the pixel corresponding to the boundary line of the boundary pixel group is positioned on the inner side of the specific object adjacent to the boundary pixel of the low resolution thermal image Based on the temperature value of the pixel And a temperature value of an outer pixel of the specific object among the boundary pixel groups is determined based on a temperature value of a pixel of the low resolution thermal image located outside the specific object adjacent to the boundary pixel; And a control unit.

<Thermal image generation system>

Hereinafter, a thermal image generating system according to an embodiment of the present invention will be described.

A thermal image generation system according to an embodiment of the present invention detects infrared rays emitted from an object to generate temperature map data and / or thermal images, and generates low-resolution temperature map data and / or low-resolution temperature map data using map data and / A system for generating temperature map data and / or thermal images with improved quality relative to thermal images.

Typical applications of thermal imaging technology are facility management to detect the gas leaks that are visually difficult to detect by visualizing the gas leaks or overheating of the pipeline, and to safely protect the facility in advance. In addition, thermal imaging technology can be used to transport vehicles such as automobiles, ships, or airplanes, so that operators can view objects that can not be confirmed in low light conditions, to assist operators of transportation means, For monitoring purposes.

The thermal image generation system according to the embodiment of the present invention can generate temperature map data and / or a thermal image comparable to a sensed image using a substantially high resolution infrared sensor while using a relatively inexpensive low resolution infrared sensor .

Figure 1 is a block diagram illustrating a thermal image generation system 1000 in accordance with an embodiment of the present invention.

Referring to FIG. 1, a thermal image generation system according to an embodiment of the present invention may include a visible light camera 1100, an infrared camera 1200, and an image processing apparatus 1300.

In a thermal image generation system according to an embodiment of the present invention, the visible light camera 1100 acquires map data and / or visible light images, the infrared camera 1200 acquires temperature map data and / or thermal images, The processing device 1300 can generate high-resolution temperature map data and / or high-resolution thermal images by performing image processing on acquired temperature map data and / or thermal images and map data and / or visible light images.

The visible light camera 1100 and the infrared camera 1200 may have different resolutions. Here, the temperature map data and / or the thermal image may be mainly a lower resolution image than the map data and / or the visible light image. For example, the map data and / or visible image obtained in the visible light camera 1100 may have a resolution of 640x480, while the temperature map data and / or the thermal image obtained in the infrared camera 1200 have a resolution of 80x60 have. Therefore, hereinafter, the temperature map data and / or the thermal image obtained by the infrared camera 1200 are referred to as the low-resolution temperature map data and / or the low-resolution thermal image, and the low-resolution temperature map data and / The temperature map data and / or the thermal image, which is improved in quality by using the low-resolution thermal image and the map data and / or the visible light image, will be referred to as high-resolution temperature map data and / or high-

In addition, the temperature map data and / or the thermal image acquired through the image processing device 1300 may be higher in resolution than the captured temperature map data and / or the thermal image. For example, the quality-improved temperature map data and / or thermal image may have the same resolution as the map data and / or visible light image.

On the other hand, the devices shown in Fig. 1 are not essential, and the thermal image generation system may have more devices or fewer components.

<Each component of thermal image generation system>

Hereinafter, each component of the thermal image generation system will be described in more detail.

1. Visible light camera 1100

FIG. 2 is a diagram showing a visible light camera 1100 according to an embodiment of the present invention in more detail.

The devices shown in FIG. 2 are not essential, and the visible light camera 1100 may have more or fewer components than those.

Referring to FIG. 2, the visible light camera 1100 includes an image sensor 1110 for detecting visible light emitted from an object to be imaged; A control unit 1120 which processes various data of the visible light camera and controls the operation of the visible light camera as a whole; A storage unit 1130 for storing various data; And a communication unit 1140 that can exchange data with other devices.

The visible light camera 1100 according to an embodiment of the present invention receives visible light emitted from an object by the image sensor 1110 to generate object data and transmits map data composed of the object data to the controller 1120 It is possible to generate a visible light image composed of color data that can be processed and visualized.

The target data may be understood as a pixel-by-pixel data table constituting the map data. That is, the object data may be data assigned for each pixel constituting the map data. Here, the target data may be expressed by numerically calculating the frequency, intensity, wavelength, etc. of the visible light received by the image sensor, or may be represented by the voltage or current value of the electric signal output from the image sensor.

The color data is a data table for each pixel constituting the visible light image. The color data includes a color (H), a saturation (S), a lightness (V) model, a red (R) C), blue (M), yellow (Y), and black (K) models.

The visible light camera 1100 according to an exemplary embodiment of the present invention may be provided in a form fixedly installed at a specific point, or may be provided in a form moving along a track or a form carried by a user.

The visible light camera 1100 may be a stationary type having a fixed angle of view, or a non-stationary shape in which the angle of view is changed according to tilting or panning operations. In the case of the nonvisualized visible light camera 1100, the view angle changing operation may be performed according to the user's input or the object tracking may be performed automatically. When changing the automatic angle of view, the angle of view may be changed in such a manner that a specific object is tracked using the image of the infrared camera 1200 or the image analysis result of the self-visible light image.

The image sensor 1110 receives visible light emitted from a subject to be imaged as a photodiode arranged in a two-dimensional array, receives charges generated in accordance with the photoelectric effect from the photodiode into a charge coupled device (CCD) and / Lt; / RTI &gt;

Charge Coupled Device (CCD) acquires current intensity information through the amount of electrons generated in proportion to the amount of photons, and generates an image by using the amount of electrons. The CMOS generates a voltage by counting the amount of electrons generated in proportion to the amount of photons The image can be generated using the information. At this time, CCD has an advantage of being excellent in image quality, and CMOS may be advantageous in that the process is simple and the processing speed is fast.

In addition, an apparatus for generating charge data generated according to the purpose of use according to the photoelectric effect can be used for all of the above-described CCD and / or CMOS and other methods.

The controller 1120 may perform various operations on the data of the visible light camera 1100 and may control operations of the components of the visible light camera 1100. For example, the control unit 1110 can control the visible light camera 1100 so that the transmitted infrared rays can be focused on the visible light sensor 1120 when the visible light does not fit the focal point of the visible light sensor 1120.

Also, the control unit 1120 may be implemented as a computer or a similar device according to hardware, software, or a combination thereof. The controller 1120 may be provided in the form of an electronic circuit such as a CPU chip that performs a control function by processing an electrical signal and is provided in a form of a program that drives a hardware controller 1120 in software .

The storage unit 1130 may store data temporarily or semi-permanently. For example, the temperature value obtained through the visible light sensor 1110 can be stored, and the camera parameter data such as the field of view (FOV) of the visible light camera 1100 and the focal distance can be stored.

In addition, the storage unit 1130 may store an operating system (OS) for operating the visible light camera 1100, firmware, middleware, various programs assisting the operation, Data received from the device, and the like can be stored.

Examples of the storage unit 1130 include a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a ROM (Read-Only Memory), a RAM Random Access Memory) or Cloud Storage.

In addition, the storage unit 1130 may be provided in a visible light camera 1100 and / or a visible light camera 1100, or may be provided in a detachable form.

The communication unit 1140 can perform communication with an external device. For example, the communication unit 1140 can transmit / receive data to / from an external device. For example, the communication unit 1140 can transmit the visible light image of the visible light camera 1100 to the image processing apparatus 1300. [

The communication unit 1140 may be implemented as a wired module and a wireless module.

The visible light camera 1100 described above can generate data relating to an object by receiving the visible light from the image sensor 1110 and outputting an electric signal based on the characteristics of the received visible light. Such data may be map data, and a visible light image may be generated based on the map data. Hereinafter, a visible light image will be described in detail

3 is a view showing a visible light image of an object according to an embodiment of the present invention.

Referring to FIG. 3, the visible light camera 1100 processes the data generated by the operation of the components to generate a visible high-resolution visible light image for the object, thereby providing an image that is almost the same as the real world .

In addition, such a high-resolution visible light image may or may not include at least one object of interest.

In the above description, the image is generated using the visible light camera in the thermal image generation system, but it is also possible to use other means for obtaining the visible light image.

2. Infrared camera (1200)

4 is a diagram showing the infrared camera 1200 according to an embodiment of the present invention in more detail.

The devices shown in Fig. 4 are not essential, and the infrared camera 1200 may have more or fewer components than those.

4, the infrared camera 1200 includes a lens unit 1210 for transmitting infrared rays emitted from an object to be imaged; An infrared sensor 1220 for detecting transmitted infrared rays and sensing temperature; A controller 1230 which processes various data of the infrared camera 1200 and controls the operation of the infrared camera 1200 as a whole; A storage unit 1240 for storing various data; And a communication unit 1250 capable of mutually transmitting and receiving data with other devices.

The infrared camera 1200 according to an embodiment of the present invention can focus infrared light on the infrared sensor 1200 by transmitting the infrared light emitted from the object through the lens unit 1210 and focus the infrared light on the infrared light using the infrared sensor 1220. [ It is possible to generate the temperature map data having the temperature data for each pixel by extracting the temperature data of the target by grasping the physical characteristics of the temperature map data and the thermal image data having the color data obtained by visualizing the temperature map data for each pixel through the control unit 1230 can do.

The temperature data may be understood as a temperature value table for each pixel constituting the temperature map data as data for making temperature map data. That is, the temperature data may be data on a temperature value assigned to each pixel constituting the temperature map data. Here, the temperature value may be expressed in units of Celsius or Fahrenheit or Kelvin temperature actually used, but it may be expressed in other units or numerical values reflecting the temperature. For example, the temperature value may be expressed by numerically calculating the frequency, intensity, wavelength, etc. of infrared rays received by the infrared sensor, or by expressing the voltage or current value of the electric signal output from the infrared sensor.

Further, the color data can refer to the data that can be visualized as described above.

The infrared camera 1200 according to one embodiment of the present invention may be provided in a fixed form at a specific point, or may be provided in a form moving along a track or in a form carried by a user.

Also, the infrared camera 1200 may be a stationary type having a fixed angle of view, or a non-stationary shape in which the angle of view is changed according to a tilting or panning operation. In the case of the non-fixed infrared camera 1200, the angle of view changing operation may be performed according to the user's input or the object tracking may be performed automatically. When changing the automatic angle of view, the angle of view may be changed by tracking a specific object using an image of a visible light camera 1100 or an image analysis result of a self-infrared image, which will be described later.

The lens unit 1210 transmits infrared rays emitted from an object to be imaged, collects infrared rays, and focuses the infrared rays on the infrared ray sensor 1220.

In addition, the lens portion 1210 may be made of germanium (Ge) as a representative example, and the lens portion 1210 may be coated with a non-reflective coating in order to increase the infrared transmittance of the lens.

The infrared sensor 1220 can detect transmitted infrared rays and can detect temperature of an object by detecting the physical characteristics (wavelength, amplitude, cycle, etc.) of infrared rays and output temperature data.

In addition, as a method of detecting the temperature depending on the purpose of use, quantum infrared detection and thermal infrared detection may be used, or other detection methods may be used.

Generally, the quantum infrared detection can detect the temperature based on the change of the resistance or the current generated by the electrons of the semiconductor, which become the material of the sensor, by the infrared rays emitted from the object to the conduction band, and the infrared sensor detects the infrared If the temperature of the sensor changes, the resistance value of the sensor also changes, so that the temperature of the object can be detected.

The control unit 1230 may perform various calculations and processes of various data of the visible light camera 1200 such as the control unit 1230 of the infrared camera 1200 and may control the operations of the components of the visible light camera 1200 can do. Therefore, redundant description of the apparatus relating to the control unit 1230 is omitted.

The storage unit 1240 may store data temporarily or semi-permanently, such as the storage unit 1130 of the visible light camera 1100 described above. Therefore, a redundant description of the apparatus relating to the storage unit 1240 is omitted.

The communication unit 1250 can transmit and receive data to / from the image processing apparatus and / or other external devices as the communication unit 1140 of the visible light camera 1100 described above. Therefore, redundant descriptions of the modules constituting the communication unit are omitted.

The infrared camera 1200 described above may receive infrared rays emitted from a subject and generate temperature data according to an electrical signal reflecting the temperature of the subject of the infrared sensor 1220 based on the characteristics of the received infrared rays. The temperature map data composed of such temperature data can be created, and the temperature map data can be generated as a thermal image through the control unit 1230 or the like. Hereinafter, the generated temperature map data and the thermal image will be specifically described do.

5 is a view showing low-resolution temperature map data for an object according to an embodiment of the present invention.

Referring to FIG. 5, the infrared camera 1200 can obtain temperature data for each pixel according to the resolution of the sensed temperature map data obtained through the infrared sensor 1220. For example, when the resolution of the low-resolution temperature map data is 5 X 5, the temperature value may be obtained as shown in FIG. 4 for the object shown in FIG.

6 is a diagram illustrating a low-resolution thermal image for a subject according to an embodiment of the invention.

Referring to FIG. 6, the infrared camera 1200 processes the temperature map data of FIG. 5 generated by the operation of the components through the controller 1230 or the like to generate a low-resolution thermal image that is visualized with respect to a target temperature So that an image can be provided so that the object can be viewed thermally. However, since the low-resolution thermal image contains less data than the high-resolution image in the image, it is difficult to recognize the object because the representation of the object to be imaged is difficult and the boundary line of the existing object becomes unclear.

On the other hand, the low-resolution thermal image may or may not include at least one object.

Hereinafter, a high-resolution thermal image generated through the thermal image generating system 1000 will be described.

7 is a diagram illustrating a high resolution thermal image for a subject according to an embodiment of the invention.

Referring to FIG. 7, H, S, and V values can be converted into color data corresponding to the high resolution thermal image to be visualized. In addition, it can be seen that the recognition of the object is easy, for example, the boundary line of the object existing in the image is clear as compared with the low-resolution thermal image of FIG. The thermal image generation system 1000 is intended to generate a thermal image that can be thermally recognized as shown in FIG. 7 by converting a low-resolution low-resolution thermal image into a high-resolution thermal image without using the high-resolution infrared camera 1200.

Hereinafter, a method of displaying a thermal image through the thermal image generating system 1000 will be described.

8 is a view showing a column image displaying only the V value among H, S, and V for an object according to an embodiment of the present invention.

Referring to FIG. 8, a column image can be visualized using only one of the constituent data of color data. For example, a thermal image can display a thermal image with only the brightness value of V among H, S, V color data.

In the above description, a thermal image is generated by using the infrared camera 1200 in the thermal image generation system. However, it is also possible to use other means for obtaining thermal images instead of the infrared camera 1200.

3. Image processing device (1300)

9 is a diagram showing the image processing apparatus 1300 according to an embodiment of the present invention in more detail.

The apparatuses shown in Fig. 9 are not essential, and the image processing apparatus 1300 may have more or fewer components than those.

9, the image processing apparatus includes an input unit 1310 for receiving various inputs from a user; A power supply unit 1320 for supplying power necessary for operation of other devices, an output unit 1330 for visualizing data, a communication unit 1340 for transmitting and receiving data mutually with other devices, A storage unit 1350, and a thermal image generation unit 1360 for generating a converted thermal image. However, the components shown in FIG. 2 are not essential, and an image processing apparatus 1300 having components having more or fewer components may be implemented.

Input unit 1310 may receive user input from a user. The user input may be in various forms including a key input, a touch input, and a voice input. For example, the input unit 1310 may receive a user's image resolution selection input through a key input button.

Typical examples of the input unit 1310 include a touch sensor that detects a touch of a user, a microphone that receives a voice signal, a camera that recognizes a gesture through image recognition, A proximity sensor including an illuminance sensor or an infrared sensor for sensing approach, a motion sensor for recognizing a user's operation through an acceleration sensor or a gyro sensor, and various types of input means for sensing or receiving various types of user inputs It is a comprehensive concept that includes both. Here, the touch sensor may be implemented by a touch panel attached to a display panel or a piezoelectric or electrostatic touch sensor that senses a touch through a touch film, or an optical touch sensor that senses a touch by an optical method.

The power supply unit 1320 may provide power necessary for operation of each component of the thermal image generation system. For example, the power supply unit 1320 can supply power necessary for operation of the visible light camera 1100 and / or the infrared camera 1200. [

Also, the power supply unit 1320 may be implemented as a rechargeable battery.

The output unit 1330 may output the data obtained from the user and / or the processed data. For example, the output unit 1330 may output camera parameters such as image resolution, FOV, focal length, etc., as well as at least one of the thermal image and the visible light image.

The output unit 1330 may include a display for outputting an image, a speaker for outputting sound, a haptic device for generating vibration, and various other types of output means. Hereinafter, the output unit 1330 of the image processing apparatus will mainly describe a display capable of visually transmitting an image. Nevertheless, in the image processing apparatus, the image is not always output to the user through the display, but may be output to the user through all of the other output means described above. The display may be a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a flat panel display (FPD) various types of devices capable of performing image display functions such as display, curved display, flexible display, 3D display, holographic display, projector, Quot; means a wide range of video display devices. Such a display may be in the form of a touch display integrated with the touch sensor of the input unit 1310. The output unit 1330 may be implemented in the form of an output interface (USB port, PS / 2 port, or the like) for connecting an external output device to the image processing apparatus 1300 instead of a device for outputting information to the outside .

The communication unit 1340 may function as the communication unit 1140 of the visible light camera 1100 and the infrared communication unit 1200 communication unit 1250. For example, it can transmit and receive data to and from the infrared camera 1200 and / or the visible light camera 1100 and / or other external devices. Therefore, redundant descriptions of the modules constituting the communication unit 1340 are omitted.

The storage unit 1350 may function as the storage unit 1140 of the visible light camera 1100 and the storage unit 1240 of the infrared camera 1200 described above. For example, the storage unit 1350 may store data generated according to the operation of the image processing apparatus 1300. Therefore, redundant description of the storage unit 1350 is omitted.

The thermal image generating unit 1360 may perform a thermal image generating operation (S1000). For example, using the low-resolution temperature map data and the low-resolution thermal image and / or the map data and the visible light image, high-resolution temperature map data and / or high-resolution thermal images are generated and the temperature value of the pixel and / .

However, in order to facilitate explanation, the thermal image generating operation (S1000) of the thermal image generating unit 1360 to be described below generates the high-resolution temperature map data using the low-resolution temperature map data and the map data .

In the following description, the operation of the image processing apparatus 1300 can be interpreted as being performed by the control of the thermal image generating unit 1360. [

&Lt; Thermal image generating operation (S1000) >

Hereinafter, the thermal image generating operation (S1000) of the thermal image generating unit 1360 will be described in more detail.

10 is a diagram showing an operation included in a thermal image generating operation (S1000) according to an embodiment.

10, a thermal image generating operation S1000 according to an embodiment of the present invention includes a pixel detecting operation (S1010) corresponding to a boundary line from the map data, an operation of generating high-resolution temperature map data (S1020) Operation S1030. Also, the operations shown in FIG. 10 are not essential, and the thermal image generating unit 1360 can perform more operations or less operations.

Hereinafter, each operation of the above-described column image generating operation (S1000) will be described in more detail.

1. Pixel detection operation (S1010) corresponding to a boundary line from map data

The thermal image generator 1360 according to an embodiment of the present invention can detect at least one boundary line from at least one map data. The method of detecting the boundary line may be Edge Detection. For example, if the difference in brightness value between adjacent pixels is equal to or greater than a threshold value, it can be determined as a boundary line. The edge detection is a known technique in the field of image processing, so a detailed description thereof will be omitted.

FIG. 11 is a diagram showing a result of detecting pixels corresponding to a boundary line in map data according to an embodiment of the present invention.

11, the column image generating unit 1360 compares the pixel data of the map data with the data of the adjacent pixels by the edge detection method in order to extract the boundary line of the object existing in the map data, It is determined whether or not the pixel value is greater than or equal to a predetermined threshold value and detected as a pixel corresponding to the boundary line at a threshold value or more.

Hereinafter, a pixel corresponding to the boundary line of the map data will be referred to as a boundary line in order to facilitate description of the thermal image generation operation (S1000).

The method of detecting the boundary line in the visible light image according to an embodiment of the present invention is not limited to the above-described method, but other methods may exist.

2. High resolution temperature map data resolution generation operation (S1020)

The thermal image generator 1360 according to an embodiment of the present invention may generate at least one high-resolution temperature map data.

12 is a view showing low-resolution temperature map data and high-resolution temperature map data according to an embodiment of the present invention.

Referring to FIG. 12, the high-resolution temperature map data generated by the thermal image generating unit 1360 may correspond to the low-resolution temperature map data. The corresponding relationship may be mxn times. For example, one pixel of low-resolution temperature map data may correspond to 2 x 2 pixel groups of high-resolution temperature map data.

Meanwhile, m and n may be natural numbers, but they are not necessarily natural numbers.

Further, the corresponding relationship does not need to correspond to the same ratio in the vertical and horizontal directions. For example, one pixel of the low-resolution temperature map data becomes 2X4 of the high-resolution temperature map data, and 160x240 high-resolution temperature map data corresponding to the low-resolution temperature map data of 80x60 can be generated.

10 is a view showing generated high-resolution temperature map data according to an embodiment of the present invention.

Referring to FIG. 10, the thermal image generator 1360 may generate at least one or more high-resolution temperature map data corresponding to the low-resolution temperature map data according to an embodiment of the present invention. For example, if the resolution of the low-resolution temperature map data is 80X60, the resolution of the high-resolution temperature map data may have various resolutions such as 640X480, 800X600, and 1280X720.

The method of generating the high-resolution temperature map data according to an embodiment of the present invention is not limited to the methods listed above, but other methods may exist.

3. Pixel temperature value setting operation (S1030)

Hereinafter, the operation of setting the temperature of the thermal image generator 1360 (S1030) will be described.

The thermal image generating unit 1360 can generate the high-resolution temperature map data by setting the temperature value of the pixel so that the boundary line is clear and can be easily recognized thermally. For example, the thermal image generating unit 1360 can detect pixels having an attribute in the vicinity of the boundary line in the temperature map data and set the temperature value of the pixel, thereby generating high-resolution temperature map data with a clear boundary line.

Hereinafter, a method of setting a temperature value according to an embodiment of the present invention will be described.

3.1 Applying properties of extrapolation and / or interpolation method when setting temperature value

In accordance with an embodiment of the present invention, the thermal image generator 1360 may set the temperature value of the pixel of the temperature map data. In addition, the method of setting the temperature value of the pixel may be similar to the properties of the extrapolation method and / or interpolation method. However, the method of setting the temperature value is not limited to the extrapolation method and the interpolation method, but there may be other methods.

Hereinafter, how the extrapolation and interpolation attributes can be applied when setting the temperature value of a pixel in the temperature map data according to an embodiment of the present invention will be described.

According to one embodiment of the present invention, extrapolation can refer to a method of obtaining an approximation of a function value or a function value for an arbitrary variable outside those areas based on the function values of at least two variables. In this case, in the temperature value setting operation (S1030), the variable of the extrapolation method may correspond to a pixel of the temperature map data, and may be a pixel other than a pixel between pixels based on an arbitrary variable outside the area. Lt; / RTI &gt;

Hereinafter, a method of setting the temperature value of the pixels in the temperature map data by extrapolation from FIG. 13 will be described.

13 is a diagram illustrating a method of setting a temperature value of a pixel by applying extrapolation in temperature map data according to an embodiment of the present invention.

13, the thermal image generating unit 1360 may set the temperature value of the pixel for which the temperature value in the temperature map data is to be set by extrapolating based on the temperature value of the adjacent pixel. For example, the temperature value of the pixel 1100c of the temperature map data is calculated based on the trend of the two data based on the temperature value 10 of the pixel 1100a corresponding to two or more pixels existing at the adjacent positions and the temperature value 15 of the pixel 1100b The temperature value of 1100c can be set to 20.

The trend of the data may be obtained by comparing the temperature values of the adjacent pixels, and the method of setting the temperature value according to the trend may be changed according to the user and / or thermal image generating unit 1360. For example, the temperature value of the pixel 1100c may be set to 20, but the temperature value of the pixel 1100c may be set to 17 by decreasing the trend of the temperature value to a certain rate.

Hereinafter, temperature value setting of temperature map data by interpolation according to an embodiment of the present invention will be described.

The interpolation according to the embodiment of the present invention can be a method of obtaining approximate values near to function values or function values for arbitrary variables between at least two variables based on the function values thereof. In this case, in the temperature value setting operation (S1030), the variable of the interpolation method may correspond to the pixel of the temperature map data, and may refer to a pixel between pixels based on an arbitrary variable in the area. .

Hereinafter, a method of setting the temperature value of the pixels in the temperature map data by interpolation through Fig. 14 will be described.

14 is a diagram illustrating a method of setting a temperature value of a pixel by applying interpolation in temperature map data according to an embodiment of the present invention.

Referring to FIG. 14, the thermal image generator 1360 may interpolate the temperature values of the pixels for which the temperature values in the temperature map data are to be set, based on the temperature values of the adjacent pixels. For example, the temperature value of the pixel 1200c of the temperature map data is calculated based on the trend of the two data based on the temperature value 0 of the pixel 1200a corresponding to two or more pixels existing at the adjacent positions and the temperature value 30 of the pixel 1200b The temperature value of 1200c can be set to 15.

The trend of the data may be obtained by comparing the temperature values of the adjacent pixels, and the method of setting the temperature value according to the trend may be changed according to the user and / or thermal image generating unit 1360. For example, the temperature value of the pixel 1200c may be set to 15, but the temperature value of the pixel 1200c may be set to 20 by reducing the tendency of decreasing the temperature value to a predetermined ratio.

In the extrapolation method and the interpolation method according to an exemplary embodiment of the present invention, the adjacent pixel refers to a pixel at least one pixel away from a pixel for which a temperature value is to be set. When a temperature value is set, And / or the thermal image generating unit 1360. For example, in the case of FIGS. 11 and 12, when the temperature value of the pixel is set based on the temperature value of the pixel which is one pixel or two pixels apart, Or the like, if there are similarities in the properties of the pixels, such as those present in the pixel.

Extrapolation methods and interpolation methods according to one embodiment of the present invention are not limited to the methods listed above, but other methods may exist.

3.2 Defining Pixels and Pixel Addresses According to Pixel Properties

Hereinafter, in order to facilitate the explanation of the operation of the column image generating unit 1360 in the temperature value setting operation, pixels and / or pixel groups will be defined according to the attributes of pixels and / or groups of pixels.

15 is a pixel definition diagram illustrating definitions of pixels and / or groups of pixels in low-resolution temperature map data and high-resolution temperature map data according to attributes of pixels and / or groups of pixels according to an embodiment of the present invention.

15, the low-resolution pixel 2100 of the low-resolution temperature map data includes a low-resolution boundary pixel 2110 having an attribute in the vicinity of the boundary line in the temperature map data, a low-resolution boundary pixel 2110 in the temperature map data, A first low-resolution pixel 2120 and a low-resolution second pixel 2130 having a low resolution.

The high-resolution pixel 2200 of the high-resolution temperature map data can be divided into the boundary pixel group 2210 having the property near the boundary line in the temperature map data and the pixel group 2220 having the property inside or outside the object, Among the boundary pixel groups 2210 having the property of the neighborhood, the first high-resolution boundary pixel group 2211, the high-resolution second boundary pixel group 2212, and the pixel group having no attribute near the boundary The first pixel group 2221 having a high resolution and the second pixel group 2222 having a high resolution can be classified according to the boundary line of the object.

Hereinafter, the definitions of the pixel and / or pixel group according to the pixel attributes of the low-resolution temperature map data and / or the high-resolution temperature map data will be described in detail.

The low-resolution boundary pixel 2110 is a pixel of the low-resolution temperature map data corresponding to the boundary line of the object in the image.

The low resolution first pixel 2120 is a pixel located on one side with respect to a boundary line among low resolution pixels not corresponding to the low resolution boundary pixel 2110.

The low resolution second pixel 2130 is a pixel existing on the other side of the boundary line among the low resolution pixels not corresponding to the low resolution boundary pixel 2110.

The high-resolution first boundary pixel group 2211 is at least one pixel existing on one side with respect to the pixels and / or the boundary existing in the boundary among the high-resolution boundary pixels existing in the low-resolution boundary pixel 2110.

The second high resolution boundary pixel group 2212 is at least one pixel existing on the other side of the boundary line among the high resolution boundary pixels existing in the low resolution boundary pixel 2110.

The first high-resolution pixel group 2221 is at least one pixel existing on one side with respect to a pixel and / or a boundary existing in a boundary among high-resolution pixels not existing in the low-resolution boundary pixel 2110.

The second high resolution pixel group 2222 is at least one pixel existing on the other side of the boundary line among the high resolution pixels not existing in the low resolution boundary pixel.

16, in order to easily explain the position of the pixel in the operation of the thermal image generator 1360, the low-resolution temperature map data and the pixel address of the high-resolution temperature map data of the pixels existing in the temperature map data are defined I will explain.

16 is a diagram showing pixel addresses in low-resolution temperature map data and high-resolution temperature map data according to an embodiment of the present invention.

16, the pixel address of the low-resolution temperature map data may be set to (m, n), and the pixel addresses of the temperature map data of the resolution of mxn may be respectively (m, n) The pixel address of the temperature map data may be (ma, nb), respectively. For example, when the resolution of the low-resolution temperature map data is 4x4, the address of the left uppermost pixel of the temperature map data is (1,1), and the low- The address of the upper rightmost pixel group in the pixel having the (1,1) pixel address of the map data is (1-2,1-1).

Hereinafter, to easily explain the operation of the column image generating unit 1360, a pixel having a (m, n) pixel address is a (m, n) , mb) will be called the pixel group.

3.3 Temperature value setting of pixel in temperature map data

Hereinafter, by using the pixel definition according to the attribute of the pixel in the temperature map data and the pixel address according to the position attribute of the pixel, the pixel having the property near the boundary line in the temperature map data of the column image generating unit 1360 and the boundary line The process of setting the temperature value of a pixel that does not exist in the target object or has an attribute inside or outside the object will be described.

17 is a diagram showing pixel addresses of a high-resolution thermal image and a low-resolution thermal image according to an embodiment of the present invention.

Hereinafter, the temperature value setting operation of the temperature map data of the thermal image generating unit 1360 will be described with reference to FIG.

On the other hand, the temperature value of the high-resolution temperature map data may or may not have the initial temperature value. For example, the initial temperature value of the high-resolution temperature map data may be set to some constant value, but may be set to zero.

Hereinafter, a temperature value corresponding to a pixel of a low-resolution thermal image will be described. For example, (1-1,1-1), (1-2,1-1), (1-3,1-1), (1-1,1-2), (1 1-2, 1-2, 1-3, 1-2, 1-1, 1-3, 1-2, 1-3, (1, 1) pixel which is the pixel of the corresponding low-resolution thermal image as the initial temperature value.

3.3.1 Near-border property pixel and target internal property Pixel temperature value setting

The thermal image generating unit 1360 according to an embodiment of the present invention can set a temperature value of a pixel having an attribute in the vicinity of the boundary line in the high-resolution temperature map data.

The thermal image generating unit 1350 can set the temperature value of the group of pixels having the attribute near the border in the group of pixels in the high-resolution temperature map data based on the temperature value of the group of pixels and / have. For example, a high-resolution first boundary pixel group and / or a second boundary pixel group may be detected and the temperature value may be divided into at least one low-resolution first pixel and / or a second low-resolution boundary pixel adjacent to the low- Can be detected and extrapolated on the basis of the temperature value.

Also, for example, a method of setting the high-resolution first boundary pixel group and / or the second boundary pixel group temperature value may be performed by using a high-resolution first pixel and / or a second pixel adjacent to the low-resolution first pixel and / The temperature value can be set by an extrapolation method.

Also, the thermal image generating unit 1360 according to an embodiment of the present invention can set the temperature value of the pixel having the attribute inside the object in the high-resolution temperature map data as a natural trend. For example, at least one high-resolution first boundary pixel group, a second boundary pixel group, and / or a first pixel group and a second pixel group among high-resolution temperature map data groups corresponding to a target inner pixel group are detected, The pixel temperature value of the temperature map data can be set naturally by setting the temperature value by interpolation method among at least one or more pixel groups.

3.3.2 How to Detect the Properties of Pixels to Set Temperature Values

Hereinafter, a process of detecting a property of a pixel and / or a pixel group to which a temperature value is to be set in order to set a temperature value of a pixel and / or a pixel group in the temperature map data of the column image generating unit 1360 will be described.

3.3.2.1 Detecting Pixel Attributes in Low-Resolution Temperature Map Data

FIG. 18 is a diagram showing pixels and an object in low-resolution temperature map data corresponding to a boundary line according to an embodiment of the present invention.

Hereinafter, a process of detecting the attribute of a pixel for setting a temperature value in the low-resolution temperature map data of the thermal image generating unit 1360 will be described with reference to FIG.

The thermal image generating unit 1360 may detect a pixel having an attribute in the vicinity of at least one boundary line in the temperature map data. For example, it is possible to detect pixels in the vicinity of at least one boundary where the temperature value should be set with reference to the boundary line detected by the boundary line detecting operation (S1010) in the low-resolution temperature map data. (2,1), (3,1), (1,2), (2,2), (1,3), (3,3), (4,3), 2, 4) and (3, 4) pixels contain the boundary of the map data, so that it can be determined as a low-resolution boundary pixel.

On the other hand, the thermal image generating unit 1360 may not regard the low resolution boundary pixel as a low resolution boundary pixel even if there are few pixels corresponding to the boundary of the map data in the pixels of the low resolution temperature map data. For example, when the resolution difference between the low-resolution temperature map data and the map data is 4x4, and pixels corresponding to the boundary line of the map data exist in the pixels of the low-resolution temperature map data, pixels of the low- It may not be viewed as a pixel. For example, the (2,1), (1,2), and (3,3) pixels of the low-resolution temperature map data may not be judged as a boundary pixel because they include few pixels corresponding to the boundary line.

In addition, the thermal image generating unit 1360 can detect a pixel having an attribute inside or outside the object, which is not an attribute near the boundary line in the temperature map data. For example, it is possible to determine a pixel having a pixel address and / or a pixel address existing at one side of a boundary of a pixel address and / or a boundary of a low-resolution temperature map data that does not correspond to the low- And a pixel having a pixel address existing on the other side opposite to the one side can be determined to detect the low resolution second pixel. (4,1), (3,2), (4,2), (2,3), (1,4), and The pixel can be said to be a low resolution first pixel and the (1,1) and (4,4) pixels can be said to be a low resolution second pixel.

In addition, the thermal image generating unit 1360 can determine the pixel of the low-resolution temperature map data having the attribute in the vicinity of the boundary line but not the boundary pixel as a pixel having an attribute inside or outside the object. For example, the pixel may be determined as the first pixel and / or the second pixel according to the user and / or thermal image generating unit 1360. Also, for example, if it is determined that the first pixel is classified into a low resolution second pixel when it is close to the first resolution pixel and the first pixel is classified into a low resolution first pixel when the resolution is close to the second pixel, (1,2) pixels can be classified as a low-resolution second pixel because the pixel corresponding to the boundary line in the pixel having the (1,2) pixel address is close to the first pixel, ) Pixel is close to the second pixel, it can be the first low-resolution pixel.

3.3.2.2 Detecting pixel attributes in high resolution temperature map data

FIG. 19 is a diagram showing pixels and an object in high-resolution temperature map data corresponding to a boundary line according to an embodiment of the present invention. FIG.

Hereinafter, a method of detecting pixels in the high-resolution temperature map data of the thermal image generating unit will be described with reference to FIG.

The thermal image generating unit 1360 can detect a group of pixels having a property in the vicinity of the boundary line among the pixel groups in the high-resolution temperature map data. For example, A boundary pixel group, and a second boundary pixel group existing on the other side opposite to the one side. Also, for example, if one side is referred to as the object inward direction, the boundary pixel groups (2-1, 2-1), (2-1, 2-2) of the high-resolution temperature map data existing within the (2,2) (2-2, 2-2), (2-2, 2-2), (2-2, 2-2), and (2-1, 2-1), (2-1, 2-2), and (2-2, 2-1) can be determined as the second boundary pixel group.

In addition, the thermal image generating unit 1360 can detect a pixel group having an attribute inside or outside the object in the high-resolution temperature map data. For example, a group of pixels not corresponding to the high-resolution boundary pixel group may be determined to detect the first high-resolution pixel group, and a second high-resolution pixel group existing on the opposite side of the one side may be detected. Also, for example, the thermal image generating unit 1360 generates (3-1, 2-1), (3-1), , 2-2), (3-2, 2-1), and (3-2, 2-2) are all located at one side of the boundary line.

The process of detecting the attribute of the pixel in the temperature map data may include a process other than the listed process.

3.3.3 Detailed description for setting the temperature value of the pixels in the temperature map data

Hereinafter, a method of setting the temperature value of the pixels in the temperature map data by the column image generating unit 1360 in Table 3.3 will be described specifically using, for example, pixels and / or pixel group definitions and addresses.

3.3.3.1 Setting the temperature value near the boundary line in the high-resolution temperature map data

20 is a diagram illustrating a method of setting a temperature value of a boundary pixel group of high-resolution temperature map data according to an embodiment of the present invention.

20, the thermal image generating unit 1360 generates high resolution first boundary pixel groups (3-3,1-2), (3-1,1-3), (3-2,1-3), and (3-3,1-3) is determined based on the temperature value of the (3, 2) pixel among the at least one low-resolution first pixel adjacent to the (3,1) pixel which is the corresponding low- Method can be set.

(3-3,1-2), (3-1,1-3), (3-2,1-3), (3-3,1-3) pixel groups of high resolution and high resolution, May be set based on the temperature value of the (3-a, 2-b) pixel group which is at least one high-resolution first pixel group adjacent to the (3,1) pixel which is the corresponding low resolution boundary pixel.

Hereinafter, a method of setting the temperature value by the method of extrapolation through Fig. 19 will be described in detail by way of example.

21 is a diagram illustrating a method of setting a temperature value by extrapolating boundary pixel groups of high-resolution temperature map data according to an embodiment of the present invention.

Referring to FIG. 21, high resolution first boundary pixel groups (3-3,1-2), (3-1,1-3), (3-2,1-3), (3-3,1-3 ) Pixel group can be set as the temperature value 50 of the (3, 2) pixel which is the first resolution pixel of the first resolution, or (3-1, 1-3) (3-2,1-3) The temperature value of the (3-3,1-3) pixel group can be set to 48, and the temperature value of the (3-3,1-2) pixel group can be set to 46. [

In addition, the degree of the trend may be determined by the judgment of the input and / or thermal image generating unit 1360 of the user.

Meanwhile, the thermal image generation unit 1360 according to an exemplary embodiment of the present invention generates a thermal image based on the first pixel and the second pixel based on the temperature values of the pixel group and / or the second boundary pixel group of the high- Or by taking an average of the temperature values calculated by extrapolating two times based on the case where the number of second pixels is two or more, for example.

3.3.3.2 Set the target internal temperature value in the high-resolution temperature map data

22 is a diagram illustrating a method of setting a temperature value of a pixel group of high-resolution temperature map data according to an embodiment of the present invention.

Referring to FIG. 22, the thermal image generator 1360 generates high resolution first boundary pixel groups (3-3,1-2), (3-1,1-3), (3-2,1-3), 3-3, 1-3) The temperature value of the pixel group is extrapolated based on the temperature value of the (3, 2) pixel among the at least one low-resolution first pixel adjacent to the (3,1) (3-3,1-2), (3-1,1-3), (3-2,1-3), (3-3,1-3), and 1-3) pixels and high resolution first pixels (3-1, 2-1), (3-2, 2-1), (3-3, 2-1), ), (3-2,2-2), (3-3,2-2), (3-1,2-3), (3-2,2-3), (3-3,2-3 ) You can set the temperature value by interpolation between pixels.

Hereinafter, a method of setting the temperature value by the extrapolation method will be described in detail with reference to FIG.

23 is a diagram illustrating a method of setting a temperature value by interpolating boundary pixel groups of high-resolution temperature map data according to an embodiment of the present invention.

Referring to FIG. 23, the thermal image generator 1360 calculates the temperature of the (3-3,1-2) pixel group, which is the high resolution first boundary pixel group, by 46, (3-1,1-3) , 1-3) When the temperature value of the (3-3,1-3) pixel group is 48, the gradual trend is (3-1, 2-1) (3-2,2-1) (3-3,2 -1) pixel group can be set to 49.

The order of setting the temperature values of the high-resolution temperature map data of the thermal image generating unit 1360 according to the embodiment of the present invention is not limited to the order listed above. For example, extrapolation methods and interpolation methods may not be orderly. After the temperature value of the pixel group of the high-resolution temperature map data is set through the interpolation method, the temperature value of the boundary pixel group of the high-resolution temperature map data can be set by an extrapolation method.

The operation of each component of the listed column image generating unit 1360 according to an embodiment of the present invention can be operated regardless of the order described.

In addition, the operation of setting the temperature value of the pixels in the temperature map data of the listed column image generating unit 1360 according to the embodiment of the present invention is not limited to the above-described operations, The method of setting the temperature value of the temperature map data according to an embodiment of the present invention is not limited to the methods listed above but may be other methods.

<Temperature Map Data Generation Method of Temperature Map Data Generation System>

24 is a flowchart showing a method of generating temperature map data in an embodiment of the present invention.

Referring to FIG. 24, the temperature map data generation method (S2400) acquires low resolution temperature map data and high resolution map data (S2410), detects boundary lines of map data (S2420), generates high resolution temperature map data (S2430) Setting of the temperature value of the map data (S2440), and display of the temperature map data (S2450).

In an embodiment of the present invention, steps S2410 to S2450 may be performed simultaneously, but either step may be performed earlier in time. Steps S2410 to S2450 may all be performed, but steps S2410 to S2450 are not always performed at all, and only at least one of steps S2410 to S2450 may be performed.

Hereinafter, each step will be described in detail.

In step S2410 of obtaining the low-resolution temperature map data and high-resolution map data, the infrared camera 1200 and the visible light camera receive a resolution setting command or resolution setting input from the image processing apparatus, Can be picked up.

If it is difficult to detect the boundary of the map data because the pixel value of the acquired map data is lower than or equal to the predetermined threshold value in step S2420 of detecting the boundary of the map data, .

In the step of generating the high-resolution temperature map data (S2430), at least one of the generated high-resolution temperature map data may be stored in the storage unit 1350 of the image processing apparatus 1300.

In the step of setting the temperature value of the temperature map data (S2440), there are various methods of setting the temperature value of the temperature map data depending on the user's control. The embodiments described in the thermal specification data of the pixels in the temperature map data by the improvement of the software, firmware, etc. of the image processing apparatus 1300 may be applied to those skilled in the art to clarify the idea of the present invention. Therefore, the present invention is not limited to the embodiments described herein, and the scope of the present invention should be construed as including modifications or variations that do not depart from the spirit of the present invention.

Although the terms used in the present invention have been selected in consideration of the functions of the present invention, they are generally used in general terms. However, the present invention is not limited to the intention of the person skilled in the art to which the present invention belongs . However, if a specific term is defined as an arbitrary meaning, the meaning of the term will be described separately. Accordingly, the terms used herein should be interpreted based on the actual meaning of the term rather than on the name of the term, and on the content throughout the description.

The drawings attached hereto are intended to illustrate the present invention easily, and the shapes shown in the drawings may be exaggerated and displayed as necessary in order to facilitate understanding of the present invention, and thus the present invention is not limited to the drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a detailed description of known configurations or functions related to the present invention will be omitted when it is determined that the gist of the present invention may be obscured.

According to an aspect of the present invention, there is provided a method of manufacturing a liquid crystal display, comprising: obtaining a high-resolution visible light image and a low-resolution thermal image corresponding to each other; Detecting a boundary line of a specific object from the high-resolution visible light image; And generating a high resolution thermal image using the low resolution thermal image and the boundary detection result, wherein the step of generating the high resolution thermal image comprises: determining a boundary pixel of the low resolution thermal image corresponding to the boundary line step; And determining a boundary pixel group of the high resolution thermal image corresponding to the boundary pixel, wherein the temperature value of the remaining pixels excluding the boundary pixel group among the pixels of the high resolution column image, Wherein a temperature value of an inner pixel of the specific object including a pixel corresponding to the boundary line of the boundary pixel group is determined based on a temperature value of a pixel of the low resolution thermal image, Wherein a temperature value of an outer pixel of the specific one of the boundary pixel groups is determined based on a temperature value of a pixel located inside the adjacent specific object, Which is determined based on the temperature value of the pixel located outside of the thermal image generating room This can be provided.

According to another aspect of the present invention, there is provided an image processing method comprising the steps of: obtaining a visible light image and a low-resolution thermal image corresponding to the visible light image but having a lower resolution than the visible light image; Detecting a boundary line in the visible light image; And generating a high-resolution thermal image having a higher resolution than the low-resolution thermal image, the low-resolution thermal image being composed of high-resolution pixels obtained by dividing low-resolution pixels of the low-resolution thermal image using the boundary detection result and the temperature data of the low- Wherein the step of generating the temperature data comprises the step of calculating the temperature value of the high resolution pixel belonging to the low resolution pixel corresponding to the pixel representing the boundary line in the visible light image to the temperature data of another low resolution pixel adjacent to the low resolution pixel to which the high resolution pixel belongs A method of generating a thermal image including the steps of:

Also, in the determining, the adjacent other low-resolution pixels may be located on the same side as the high-resolution pixel with respect to the boundary line.

Further, in the determining step, the adjacent other low-resolution pixels may be located in an area to which the high-resolution pixel defined by the boundary line belongs.

In addition, the boundary line is associated with a specific object, and in the determining step, the high-resolution pixel and the adjacent low-resolution pixel may be located within the specific object.

Further, the temperature value can be calculated by extrapolating using the temperature data of the other low resolution pixels.

The generating may include determining a temperature value of the high resolution pixel belonging to the low resolution pixel that does not correspond to a pixel representing the boundary line in the visible light image using the temperature data of the low resolution pixel to which the high resolution pixel belongs Step &lt; / RTI &gt;

The temperature value of the high-resolution pixel belonging to the low-resolution pixel which does not correspond to the pixel representing the boundary line can be calculated by interpolation using the temperature data of the low-resolution pixel.

The method may further include determining at least one of color elements of a predetermined color space based on the temperature value and displaying the high resolution image.

The color space may include at least one of RGB system, HSV system, CMYK system, CIE system, PMS system, Munsell color system, NCS system, and YIQ system.

The method may further include displaying the detected boundary line on the high resolution image.

The high-resolution thermal image may have the same resolution as the visible light image.

The method may further include displaying at least one of the visible light image, the low resolution image, and the high resolution image.

According to still another aspect of the present invention, there is provided a visible light sensor module for capturing a high-resolution visible light image including a specific object; A thermal image camera for capturing a low-resolution thermal image including the specific object; And generating a high resolution thermal image by using the low resolution thermal image and the boundary detection result to determine a boundary pixel of the low resolution thermal image corresponding to the boundary line, Resolution thermal image corresponding to the boundary pixel, and the temperature value of the remaining pixels excluding the boundary pixel group among the pixels of the high-resolution column image is determined as the temperature of the pixel of the low resolution column image corresponding to the remaining pixel And the temperature value of the inner pixel of the specific object including the pixel corresponding to the boundary line of the boundary pixel group is positioned on the inner side of the specific object adjacent to the boundary pixel of the low resolution thermal image Based on the temperature value of the pixel And a temperature value of an outer pixel of the specific object among the boundary pixel groups is determined based on a temperature value of a pixel of the low resolution thermal image located outside the specific object adjacent to the boundary pixel; And a control unit.

<Thermal image generation system>

Hereinafter, a thermal image generating system according to an embodiment of the present invention will be described.

A thermal image generation system according to an embodiment of the present invention detects infrared rays emitted from an object to generate temperature map data and / or thermal images, and generates low-resolution temperature map data and / or low-resolution temperature map data using map data and / A system for generating temperature map data and / or thermal images with improved quality relative to thermal images.

Typical applications of thermal imaging technology are facility management to detect the gas leaks that are visually difficult to detect by visualizing the gas leaks or overheating of the pipeline, and to safely protect the facility in advance. In addition, thermal imaging technology can be used to transport vehicles such as automobiles, ships, or airplanes, so that operators can view objects that can not be confirmed in low light conditions, to assist operators of transportation means, For monitoring purposes.

The thermal image generation system according to the embodiment of the present invention can generate temperature map data and / or a thermal image comparable to a sensed image using a substantially high resolution infrared sensor while using a relatively inexpensive low resolution infrared sensor .

Figure 1 is a block diagram illustrating a thermal image generation system 1000 in accordance with an embodiment of the present invention.

Referring to FIG. 1, a thermal image generation system according to an embodiment of the present invention may include a visible light sensor module 1100, an infrared sensor module 1200, and an image processing device 1300.

In a thermal image generation system according to an embodiment of the present invention, the visible light sensor module 1100 acquires map data and / or visible light images, and the infrared sensor module 1200 acquires temperature map data and / or thermal images , The image processing apparatus 1300 can generate the high-resolution temperature map data and / or the high-resolution thermal image by performing the image processing on the obtained temperature map data and / or the thermal image and the map data and / or the visible light image.

The visible light sensor module 1100 and the infrared sensor module 1200 may have different resolutions. Here, the temperature map data and / or the thermal image may be mainly a lower resolution image than the map data and / or the visible light image. For example, the temperature map data and / or the thermal image obtained in the infrared sensor module 1200 has a resolution of 80x60, while the map data and / or visible light image obtained in the visible light sensor module 1100 has a resolution of 640x480 Lt; / RTI &gt; Hereinafter, the temperature map data and / or the thermal image obtained by the infrared sensor module 1200 will be referred to as the low-resolution temperature map data and / or the low-resolution thermal image, and the low-resolution temperature map data and / Or the thermal map image in which the quality is improved by using the low resolution thermal image and the map data and / or the visible light image is referred to as high resolution temperature map data and / or high resolution thermal image.

In addition, the temperature map data and / or the thermal image acquired through the image processing device 1300 may be higher in resolution than the captured temperature map data and / or the thermal image. For example, the quality-improved temperature map data and / or thermal image may have the same resolution as the map data and / or visible light image.

On the other hand, the devices shown in Fig. 1 are not essential, and the thermal image generation system may have more devices or fewer components.

<Each component of thermal image generation system>

Hereinafter, each component of the thermal image generation system will be described in more detail.

1. The visible light sensor module (1100)

FIG. 2 is a view showing the visible light sensor module 1200 according to an embodiment of the present invention in more detail.

The devices shown in FIG. 2 are not essential, and the visible light sensor module 1100 may have more or fewer components than those.

Referring to FIG. 2, the visible light sensor module 1100 includes an image sensor 1120 that detects visible light emitted from an object to be imaged; A controller 1130 for processing various data of the visible light sensor module and controlling the operation of the visible light sensor module 1100 as a whole; A storage unit 1140 for storing various data; And a communication unit 1150 capable of mutually transmitting and receiving data with other devices.

The visible light sensor module 1100 according to the embodiment of the present invention receives visible light emitted from a subject through the lens unit 1110 to the image sensor 1120 to generate target data, The control unit 1130 can process the data to generate a visible light image composed of color data that can be visualized.

The target data may be understood as a pixel-by-pixel data table constituting the map data. That is, the object data may be data assigned for each pixel constituting the map data. Here, the target data may be expressed by numerically calculating the frequency, intensity, wavelength, etc. of the visible light received by the image sensor, or may be represented by the voltage or current value of the electric signal output from the image sensor.

The color data is a data table for each pixel constituting the visible light image. The color data includes a color (H), a saturation (S), a lightness (V) model, a red (R) C), blue (M), yellow (Y), and black (K) models.

The visible light sensor module 1100 according to an exemplary embodiment of the present invention may be provided in a form fixedly installed at a specific point, or may be provided in a form moving along a track or in a form carried by a user.

In addition, the visible light sensor module 1100 may be a stationary type having a fixed angle of view, or a non-stationary shape in which the angle of view is changed according to a tilting or panning operation. In the case of the nonvisualized visible light sensor module 1100, the view angle changing operation may be performed according to the user's input, or the object viewing may be performed to automatically perform the view angle changing operation. The angle of view may be changed by tracking the specific object using the image of the infrared sensor module 1200 or the image analysis result of the self-visible light image.

The lens unit 1110 transmits the visible light emitted from the object to be imaged, and collects the infrared rays to focus the image sensor 1120.

The image sensor 1120 receives visible light emitted from a subject to be imaged as a photodiode arranged in a two-dimensional array, receives charges generated in accordance with the photoelectric effect from a photodiode into a charge coupled device (CCD) and / Lt; / RTI &gt;

Charge Coupled Device (CCD) acquires current intensity information through the amount of electrons generated in proportion to the amount of photons, and generates an image by using the amount of electrons. The CMOS generates a voltage by counting the amount of electrons generated in proportion to the amount of photons The image can be generated using the information. At this time, CCD has an advantage of being excellent in image quality, and CMOS may be advantageous in that the process is simple and the processing speed is fast.

In addition, an apparatus for generating charge data generated according to the purpose of use according to the photoelectric effect can be used for all of the above-described CCD and / or CMOS and other methods.

The controller 1130 may perform various operations on the data of the visible light sensor module 1100 and may control the operation of the components of the visible light sensor module 1100. For example, the control unit 1120 can control the visible light sensor module 1200 so that infrared rays transmitted through the visible light sensor 1130 can be focused on the visible light sensor 1130 when the visible light does not fit the visible light sensor 1130 .

In addition, the control unit 1130 may be implemented as a computer or a similar device according to hardware, software, or a combination thereof. The control unit 1130 may be provided in the form of an electronic circuit such as a CPU chip that performs a control function by processing an electrical signal and is provided in a form of a program that drives a hardware control unit 1130 in software .

The storage unit 1140 may store data temporarily or semi-permanently. For example, the temperature value acquired through the visible light sensor 1120 can be stored, and the camera parameter data such as the field of view (FOV) of the visible light sensor module 1100 and the focal distance can be stored.

Also, the storage unit 1140 may store an operating system (OS) for operating the visible light sensor module 1200, firmware, middleware, various programs assisting the operation, Data received from an external device, and the like can be stored.

Examples of the storage unit 1140 include a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a ROM (Read-Only Memory), a RAM Random Access Memory) or Cloud Storage.

In addition, the storage unit 1140 may be provided in the visible light sensor module 1100 and / or the visible light sensor module 1100, or may be provided in a detachable form.

The communication unit 1150 can perform communication with an external device. For example, the communication unit 1150 can transmit / receive data to / from an external device. For example, the communication unit 1150 can transmit a visible light image of the visible light sensor module 1100 to the image processing apparatus 1300.

The communication unit 1150 may be implemented as a wired module and a wireless module.

The visible light sensor module 1100 described above can generate data on an object by receiving the visible light from the image sensor 1120 and outputting an electric signal based on the characteristics of the received visible light. Such data may be map data, and a visible light image may be generated based on the map data. Hereinafter, a visible light image will be described in detail

3 is a view showing a visible light image of an object according to an embodiment of the present invention.

Referring to FIG. 3, the visible light sensor module 1100 processes data generated by the operation of the components to generate a visible high-resolution visible light image of a target, have.

In addition, such a high-resolution visible light image may or may not include at least one object of interest.

In the above description, the image is generated using the visible light sensor module in the thermal image generation system, but it is also possible to use other means for obtaining the visible light image.

2. Infrared sensor module (1200)

4 is a diagram illustrating the infrared sensor module 1200 according to an embodiment of the present invention in more detail.

The devices shown in Fig. 4 are not essential, and the infrared sensor module 1200 may have more or fewer components than those.

Referring to FIG. 4, the infrared sensor module 1200 includes a lens unit 1210 for transmitting infrared rays emitted from an object to be imaged; An infrared sensor 1220 for detecting transmitted infrared rays and sensing temperature; A controller 1230 for processing various data of the infrared sensor module 1200 and controlling the operation of the infrared sensor module 1200 as a whole; A storage unit 1240 for storing various data; And a communication unit 1250 capable of mutually transmitting and receiving data with other devices.

The infrared sensor module 1200 according to an embodiment of the present invention can focus the infrared ray emitted from the object through the lens portion 1210 to focus on the infrared ray sensor 1200 and use the infrared ray sensor 1220 It is possible to generate the temperature map data having the temperature data for each pixel by extracting the temperature data of the target by grasping the physical characteristics of the infrared rays and the thermal image having the color data obtained by visualizing the temperature map data for each pixel through the control unit 1230 Can be generated.

The temperature data may be understood as a temperature value table for each pixel constituting the temperature map data as data for making temperature map data. That is, the temperature data may be data on a temperature value assigned to each pixel constituting the temperature map data. Here, the temperature value may be expressed in units of Celsius or Fahrenheit or Kelvin temperature actually used, but it may be expressed in other units or numerical values reflecting the temperature. For example, the temperature value may be expressed by numerically calculating the frequency, intensity, wavelength, etc. of infrared rays received by the infrared sensor, or by expressing the voltage or current value of the electric signal output from the infrared sensor.

Further, the color data can refer to the data that can be visualized as described above.

The infrared sensor module 1200 according to an exemplary embodiment of the present invention may be provided in a form fixedly installed at a specific point, or may be provided in a form moving along a track or in a form carried by a user.

Also, the infrared sensor module 1200 may be a stationary type having a fixed angle of view, or a non-stationary type in which the angle of view is changed according to a tilting or panning operation. In the case of the non-fixed infrared sensor module 1200, the angle of view changing operation may be performed according to the user's input or the object tracking may be performed automatically. When changing the automatic angle of view, the angle of view may be changed by tracking a specific object using an image of the visible light sensor module 1100 or an image analysis result of the self infrared ray image, which will be described later.

The lens unit 1210 transmits infrared rays emitted from an object to be imaged, collects infrared rays, and focuses the infrared rays on the infrared ray sensor 1220.

In addition, the lens portion 1210 may be made of germanium (Ge) as a representative example, and the lens portion 1210 may be coated with a non-reflective coating in order to increase the infrared transmittance of the lens.

The infrared sensor 1220 can detect transmitted infrared rays and can detect temperature of an object by detecting the physical characteristics (wavelength, amplitude, cycle, etc.) of infrared rays and output temperature data.

In addition, as a method of detecting the temperature depending on the purpose of use, quantum infrared detection and thermal infrared detection may be used, or other detection methods may be used.

Generally, the quantum infrared detection can detect the temperature based on the change of the resistance or the current generated by the electrons of the semiconductor, which become the material of the sensor, by the infrared rays emitted from the object to the conduction band, and the infrared sensor detects the infrared If the temperature of the sensor changes, the resistance value of the sensor also changes, so that the temperature of the object can be detected.

The control unit 1230 may perform various operations on the data of the visible light sensor module 1200 such as the control unit 1230 of the infrared light sensor module 1200 and may calculate and process various data of the visible light sensor module 1200 The operation can be controlled. Therefore, redundant description of the apparatus relating to the control unit 1230 is omitted.

The storage unit 1240 may store data temporarily or semi-permanently, such as the storage unit 1140 of the visible light sensor module 1100 described above. Therefore, a redundant description of the apparatus relating to the storage unit 1240 is omitted.

The communication unit 1250 can transmit and receive data to / from the image processing apparatus and / or other external devices as the communication unit 1150 of the visible light sensor module 1100 described above. Therefore, redundant descriptions of the modules constituting the communication unit are omitted.

The infrared sensor module 1200 described above may receive infrared rays emitted from a subject and generate temperature data according to an electrical signal reflecting the temperature of the object of the infrared sensor 1220 based on the characteristics of the received infrared rays. The temperature map data composed of such temperature data can be created, and the temperature map data can be generated as a thermal image through the control unit 1230 or the like. Hereinafter, the generated temperature map data and the thermal image will be specifically described do.

5 is a view showing low-resolution temperature map data for an object according to an embodiment of the present invention.

Referring to FIG. 5, the temperature map data obtained by the infrared sensor module 1200 through the infrared sensor 1220 may have temperature data for each pixel according to the sensed resolution. For example, when the resolution of the low-resolution temperature map data is 5 X 5, the temperature value may be obtained as shown in FIG. 4 for the object shown in FIG.

6 is a diagram illustrating a low-resolution thermal image for a subject according to an embodiment of the invention.

Referring to FIG. 6, the infrared sensor module 1200 processes the temperature map data of FIG. 5 generated by the operation of the components through the controller 1230 or the like to obtain a low-resolution thermal image, And provide an image so that the object can be viewed thermally. However, since the low-resolution thermal image contains less data than the high-resolution image in the image, it is difficult to recognize the object because the representation of the object to be imaged is difficult and the boundary line of the existing object becomes unclear.

On the other hand, the low-resolution thermal image may or may not include at least one object.

Hereinafter, a high-resolution thermal image generated through the thermal image generating system 1000 will be described.

7 is a diagram illustrating a high resolution thermal image for a subject according to an embodiment of the invention.

Referring to FIG. 7, H, S, and V values can be converted into color data corresponding to the high resolution thermal image to be visualized. In addition, it can be seen that the recognition of the object is easy, for example, the boundary line of the object existing in the image is clear as compared with the low-resolution thermal image of FIG. The thermal image generation system 1000 is intended to generate a thermal image that can be thermally recognized as shown in FIG. 7 by converting a low-resolution low-resolution thermal image into a high-resolution thermal image without using the high-resolution infrared sensor module 1200.

Hereinafter, a method of displaying a thermal image through the thermal image generating system 1000 will be described.

8 is a view showing a column image displaying only the V value among H, S, and V for an object according to an embodiment of the present invention.

Referring to FIG. 8, a column image can be visualized using only one of the constituent data of color data. For example, a thermal image can display a thermal image with only the brightness value of V among H, S, V color data.

In the above description, a thermal image is generated by using the infrared sensor module 1200 in the thermal image generation system. However, it is also possible to use other means for obtaining a thermal image instead of the infrared sensor module 1200.

3. Image processing device (1300)

FIG. 9 is a diagram showing the image processing apparatus 1300 according to an embodiment of the present invention in more detail.

The apparatuses shown in FIG. 9 are not essential, and the image processing apparatus 1300 may have more or fewer components than those.

9, the image processing apparatus includes an input unit 1310 for receiving various inputs from a user; A power supply unit 1320 for supplying power necessary for operation of other devices, an output unit 1330 for visualizing data, a communication unit 1340 for transmitting and receiving data mutually with other devices, A storage unit 1350, and a thermal image generation unit 1360 for generating a converted thermal image. However, the components shown in FIG. 2 are not essential, and the image processing apparatus 1300 having components having more or fewer components may be implemented.

Input unit 1310 may receive user input from a user. The user input may be in various forms including a key input, a touch input, and a voice input. For example, the input unit 1310 may receive a user's image resolution selection input through a key input button.

Typical examples of the input unit 1310 include a touch sensor that detects a touch of a user, a microphone that receives a voice signal, a camera that recognizes a gesture through image recognition, A proximity sensor including an illuminance sensor or an infrared sensor for sensing approach, a motion sensor for recognizing a user's operation through an acceleration sensor or a gyro sensor, and various types of input means for sensing or receiving various types of user inputs It is a comprehensive concept that includes both. Here, the touch sensor may be implemented by a touch panel attached to a display panel or a piezoelectric or electrostatic touch sensor that senses a touch through a touch film, or an optical touch sensor that senses a touch by an optical method.

The power supply unit 1320 may provide power necessary for operation of each component of the thermal image generation system. For example, the power supply unit 1320 can supply power necessary for the operation of the visible light sensor module 1100 and / or the infrared sensor module 1200.

Also, the power supply unit 1320 may be implemented as a rechargeable battery.

The output unit 1330 may output the data obtained from the user and / or the processed data. For example, the output unit 1330 may output camera parameters such as image resolution, FOV, focal length, etc., as well as at least one of the thermal image and the visible light image.

The output unit 1330 may include a display for outputting an image, a speaker for outputting sound, a haptic device for generating vibration, and various other types of output means. Hereinafter, the output unit 1330 of the image processing apparatus will mainly describe a display capable of visually transmitting an image. Nevertheless, in the image processing apparatus, the image is not always output to the user through the display, but may be output to the user through all of the other output means described above. The display may be a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a flat panel display (FPD) various types of devices capable of performing image display functions such as display, curved display, flexible display, 3D display, holographic display, projector, Quot; means a wide range of video display devices. Such a display may be in the form of a touch display integrated with the touch sensor of the input unit 1310. The output unit 1330 may be implemented in the form of an output interface (USB port, PS / 2 port, or the like) for connecting an external output device to the image processing apparatus 1300 instead of a device for outputting information to the outside .

The communication unit 1340 may perform the same function as the communication unit 1150 and the infrared sensor module 1200 communication unit 1250 of the visible light sensor module 1100 described above. For example, it can transmit and receive data to / from the infrared sensor module 1200 and / or the visible light sensor module 1100 and / or other external devices. Therefore, redundant descriptions of the modules constituting the communication unit 1340 are omitted.

The storage unit 1350 may function as the storage unit 1150 of the visible light sensor module 1100 and the storage unit 1240 of the infrared sensor module 1200. [ For example, the storage unit 1350 may store data generated according to the operation of the image processing apparatus 1300. Therefore, redundant description of the storage unit 1350 is omitted.

The thermal image generating unit 1360 may perform a thermal image generating operation (S1000). For example, using the low-resolution temperature map data and the low-resolution thermal image and / or the map data and the visible light image, high-resolution temperature map data and / or high-resolution thermal images are generated and the temperature value of the pixel and / .

However, in order to facilitate explanation, the thermal image generating operation (S1000) of the thermal image generating unit 1360 to be described below generates the high-resolution temperature map data using the low-resolution temperature map data and the map data .

In the following description, the operation of the image processing apparatus 1300 can be interpreted as being performed by the control of the thermal image generating unit 1360. [

&Lt; Thermal image generating operation (S1000) >

Hereinafter, the thermal image generating operation (S1000) of the thermal image generating unit 1360 will be described in more detail.

10 is a diagram showing an operation included in a thermal image generating operation (S1000) according to an embodiment.

10, a thermal image generating operation S1000 according to an embodiment of the present invention includes a pixel detecting operation (S1010) corresponding to a boundary line from the map data, an operation of generating high-resolution temperature map data (S1020) Operation S1030. Also, the operations shown in FIG. 10 are not essential, and the thermal image generating unit 1360 can perform more operations or less operations.

Hereinafter, each operation of the above-described column image generating operation (S1000) will be described in more detail.

1. Pixel detection operation (S1010) corresponding to a boundary line from map data

The thermal image generator 1360 according to an embodiment of the present invention can detect at least one boundary line from at least one map data. The method of detecting the boundary line may be Edge Detection. For example, if the difference in brightness value between adjacent pixels is equal to or greater than a threshold value, it can be determined as a boundary line. The edge detection is a known technique in the field of image processing, so a detailed description thereof will be omitted.

FIG. 11 is a diagram showing a result of detecting pixels corresponding to a boundary line in map data according to an embodiment of the present invention.

11, the column image generating unit 1360 compares the pixel data of the map data with the data of the adjacent pixels by the edge detection method in order to extract the boundary line of the object existing in the map data, It is determined whether or not the pixel value is greater than or equal to a predetermined threshold value and detected as a pixel corresponding to the boundary line at a threshold value or more.

Hereinafter, a pixel corresponding to the boundary line of the map data will be referred to as a boundary line in order to facilitate description of the thermal image generation operation (S1000).

The method of detecting the boundary line in the visible light image according to an embodiment of the present invention is not limited to the above-described method, but other methods may exist.

2. High resolution temperature map data resolution generation operation (S1020)

The thermal image generator 1360 according to an embodiment of the present invention may generate at least one high-resolution temperature map data.

12 is a view showing low-resolution temperature map data and high-resolution temperature map data according to an embodiment of the present invention.

Referring to FIG. 12, the high-resolution temperature map data generated by the thermal image generating unit 1360 may correspond to the low-resolution temperature map data. The corresponding relationship may be mxn times. For example, one pixel of low-resolution temperature map data may correspond to 2 x 2 pixel groups of high-resolution temperature map data.

Meanwhile, m and n may be natural numbers, but they are not necessarily natural numbers.

Further, the corresponding relationship does not need to correspond to the same ratio in the vertical and horizontal directions. For example, one pixel of the low-resolution temperature map data becomes 2X4 of the high-resolution temperature map data, and 160x240 high-resolution temperature map data corresponding to the low-resolution temperature map data of 80x60 can be generated.

10 is a view showing generated high-resolution temperature map data according to an embodiment of the present invention.

Referring to FIG. 10, the thermal image generator 1360 may generate at least one or more high-resolution temperature map data corresponding to the low-resolution temperature map data according to an embodiment of the present invention. For example, if the resolution of the low-resolution temperature map data is 80X60, the resolution of the high-resolution temperature map data may have various resolutions such as 640X480, 800X600, and 1280X720.

The method of generating the high-resolution temperature map data according to an embodiment of the present invention is not limited to the methods listed above, but other methods may exist.

3. Pixel temperature value setting operation (S1030)

Hereinafter, the operation of setting the temperature of the thermal image generator 1360 (S1030) will be described.

The thermal image generating unit 1360 can generate the high-resolution temperature map data by setting the temperature value of the pixel so that the boundary line is clear and can be easily recognized thermally. For example, the thermal image generating unit 1360 can detect pixels having an attribute in the vicinity of the boundary line in the temperature map data and set the temperature value of the pixel, thereby generating high-resolution temperature map data with a clear boundary line.

Hereinafter, a method of setting a temperature value according to an embodiment of the present invention will be described.

3.1 Applying properties of extrapolation and / or interpolation method when setting temperature value

In accordance with an embodiment of the present invention, the thermal image generator 1360 may set the temperature value of the pixel of the temperature map data. In addition, the method of setting the temperature value of the pixel may be similar to the properties of the extrapolation method and / or interpolation method. However, the method of setting the temperature value is not limited to the extrapolation method and the interpolation method, but there may be other methods.

Hereinafter, how the extrapolation and interpolation attributes can be applied when setting the temperature value of a pixel in the temperature map data according to an embodiment of the present invention will be described.

According to one embodiment of the present invention, extrapolation can refer to a method of obtaining an approximation of a function value or a function value for an arbitrary variable outside those areas based on the function values of at least two variables. In this case, in the temperature value setting operation (S1030), the variable of the extrapolation method may correspond to a pixel of the temperature map data, and may be a pixel other than a pixel between pixels based on an arbitrary variable outside the area. Lt; / RTI &gt;

Hereinafter, a method of setting the temperature value of the pixels in the temperature map data by extrapolation from FIG. 13 will be described.

13 is a diagram illustrating a method of setting a temperature value of a pixel by applying extrapolation in temperature map data according to an embodiment of the present invention.

13, the thermal image generating unit 1360 may set the temperature value of the pixel for which the temperature value in the temperature map data is to be set by extrapolating based on the temperature value of the adjacent pixel. For example, the temperature value of the pixel 1100c of the temperature map data is calculated based on the trend of the two data based on the temperature value 10 of the pixel 1100a corresponding to two or more pixels existing at the adjacent positions and the temperature value 15 of the pixel 1100b The temperature value of 1100c can be set to 20.

The trend of the data may be obtained by comparing the temperature values of the adjacent pixels, and the method of setting the temperature value according to the trend may be changed according to the user and / or thermal image generating unit 1360. For example, the temperature value of the pixel 1100c may be set to 20, but the temperature value of the pixel 1100c may be set to 17 by decreasing the trend of the temperature value to a certain rate.

Hereinafter, temperature value setting of temperature map data by interpolation according to an embodiment of the present invention will be described.

The interpolation according to the embodiment of the present invention can be a method of obtaining approximate values near to function values or function values for arbitrary variables between at least two variables based on the function values thereof. In this case, in the temperature value setting operation (S1030), the variable of the interpolation method may correspond to the pixel of the temperature map data, and may refer to a pixel between pixels based on an arbitrary variable in the area. .

Hereinafter, a method of setting the temperature value of the pixels in the temperature map data by interpolation through Fig. 14 will be described.

14 is a diagram illustrating a method of setting a temperature value of a pixel by applying interpolation in temperature map data according to an embodiment of the present invention.

Referring to FIG. 14, the thermal image generator 1360 may interpolate the temperature values of the pixels for which the temperature values in the temperature map data are to be set, based on the temperature values of the adjacent pixels. For example, the temperature value of the pixel 1200c of the temperature map data is calculated based on the trend of the two data based on the temperature value 0 of the pixel 1200a corresponding to two or more pixels existing at the adjacent positions and the temperature value 30 of the pixel 1200b The temperature value of 1200c can be set to 15.

The trend of the data may be obtained by comparing the temperature values of the adjacent pixels, and the method of setting the temperature value according to the trend may be changed according to the user and / or thermal image generating unit 1360. For example, the temperature value of the pixel 1200c may be set to 15, but the temperature value of the pixel 1200c may be set to 20 by reducing the tendency of decreasing the temperature value to a predetermined ratio.

In the extrapolation method and the interpolation method according to an exemplary embodiment of the present invention, the adjacent pixel refers to a pixel at least one pixel away from a pixel for which a temperature value is to be set. When a temperature value is set, And / or the thermal image generating unit 1360. For example, in the case of FIGS. 11 and 12, when the temperature value of the pixel is set based on the temperature value of the pixel which is one pixel or two pixels apart, Or the like, if there are similarities in the properties of the pixels, such as those present in the pixel.

Extrapolation methods and interpolation methods according to one embodiment of the present invention are not limited to the methods listed above, but other methods may exist.

3.2 Defining Pixels and Pixel Addresses According to Pixel Properties

Hereinafter, in order to facilitate the explanation of the operation of the column image generating unit 1360 in the temperature value setting operation, pixels and / or pixel groups will be defined according to the attributes of pixels and / or groups of pixels.

15 is a pixel definition diagram illustrating definitions of pixels and / or groups of pixels in low-resolution temperature map data and high-resolution temperature map data according to attributes of pixels and / or groups of pixels according to an embodiment of the present invention.

15, the low-resolution pixel 2100 of the low-resolution temperature map data includes a low-resolution boundary pixel 2110 having an attribute in the vicinity of the boundary line in the temperature map data, a low-resolution boundary pixel 2110 in the temperature map data, A first low-resolution pixel 2120 and a low-resolution second pixel 2130 having a low resolution.

The high-resolution pixel 2200 of the high-resolution temperature map data can be divided into the boundary pixel group 2210 having the property near the boundary line in the temperature map data and the pixel group 2220 having the property inside or outside the object, Among the boundary pixel groups 2210 having the property of the neighborhood, the first high-resolution boundary pixel group 2211, the high-resolution second boundary pixel group 2212, and the pixel group having no attribute near the boundary The first pixel group 2221 having a high resolution and the second pixel group 2222 having a high resolution can be classified according to the boundary line of the object.

Hereinafter, the definitions of the pixel and / or pixel group according to the pixel attributes of the low-resolution temperature map data and / or the high-resolution temperature map data will be described in detail.

The low-resolution boundary pixel 2110 is a pixel of the low-resolution temperature map data corresponding to the boundary line of the object in the image.

The low resolution first pixel 2120 is a pixel located on one side with respect to a boundary line among low resolution pixels not corresponding to the low resolution boundary pixel 2110.

The low resolution second pixel 2130 is a pixel existing on the other side of the boundary line among the low resolution pixels not corresponding to the low resolution boundary pixel 2110.

The high-resolution first boundary pixel group 2211 is at least one pixel existing on one side with respect to the pixels and / or the boundary existing in the boundary among the high-resolution boundary pixels existing in the low-resolution boundary pixel 2110.

The second high resolution boundary pixel group 2212 is at least one pixel existing on the other side of the boundary line among the high resolution boundary pixels existing in the low resolution boundary pixel 2110.

The first high-resolution pixel group 2221 is at least one pixel existing on one side with respect to a pixel and / or a boundary existing in a boundary among high-resolution pixels not existing in the low-resolution boundary pixel 2110.

The second high resolution pixel group 2222 is at least one pixel existing on the other side of the boundary line among the high resolution pixels not existing in the low resolution boundary pixel.

16, in order to easily explain the position of the pixel in the operation of the thermal image generator 1360, the low-resolution temperature map data and the pixel address of the high-resolution temperature map data of the pixels existing in the temperature map data are defined I will explain.

16 is a diagram showing pixel addresses in low-resolution temperature map data and high-resolution temperature map data according to an embodiment of the present invention.

16, the pixel address of the low-resolution temperature map data may be set to (m, n), and the pixel addresses of the temperature map data of the resolution of mxn may be respectively (m, n) The pixel address of the temperature map data may be (ma, nb), respectively. For example, when the resolution of the low-resolution temperature map data is 4x4, the address of the left uppermost pixel of the temperature map data is (1,1), and the low- The address of the upper rightmost pixel group in the pixel having the (1,1) pixel address of the map data is (1-2,1-1).

Hereinafter, to easily explain the operation of the column image generating unit 1360, a pixel having a (m, n) pixel address is a (m, n) , mb) will be called the pixel group.

3.3 Temperature value setting of pixel in temperature map data

Hereinafter, by using the pixel definition according to the attribute of the pixel in the temperature map data and the pixel address according to the position attribute of the pixel, the pixel having the property near the boundary line in the temperature map data of the column image generating unit 1360 and the boundary line The process of setting the temperature value of a pixel that does not exist in the target object or has an attribute inside or outside the object will be described.

17 is a diagram showing pixel addresses of a high-resolution thermal image and a low-resolution thermal image according to an embodiment of the present invention.

Hereinafter, the temperature value setting operation of the temperature map data of the thermal image generating unit 1360 will be described with reference to FIG.

On the other hand, the temperature value of the high-resolution temperature map data may or may not have the initial temperature value. For example, the initial temperature value of the high-resolution temperature map data may be set to some constant value, but may be set to zero.

Hereinafter, a temperature value corresponding to a pixel of a low-resolution thermal image will be described. For example, (1-1,1-1), (1-2,1-1), (1-3,1-1), (1-1,1-2), (1 1-2, 1-2, 1-3, 1-2, 1-1, 1-3, 1-2, 1-3, (1, 1) pixel which is the pixel of the corresponding low-resolution thermal image as the initial temperature value.

3.3.1 Near-border property pixel and target internal property Pixel temperature value setting

The thermal image generating unit 1360 according to an embodiment of the present invention can set a temperature value of a pixel having an attribute in the vicinity of the boundary line in the high-resolution temperature map data.

The thermal image generating unit 1350 can set the temperature value of the group of pixels having the attribute near the border in the group of pixels in the high-resolution temperature map data based on the temperature value of the group of pixels and / have. For example, a high-resolution first boundary pixel group and / or a second boundary pixel group may be detected and the temperature value may be divided into at least one low-resolution first pixel and / or a second low-resolution boundary pixel adjacent to the low- Can be detected and extrapolated on the basis of the temperature value.

Also, for example, a method of setting the high-resolution first boundary pixel group and / or the second boundary pixel group temperature value may be performed by using a high-resolution first pixel and / or a second pixel adjacent to the low-resolution first pixel and / The temperature value can be set by an extrapolation method.

Also, the thermal image generating unit 1360 according to an embodiment of the present invention can set the temperature value of the pixel having the attribute inside the object in the high-resolution temperature map data as a natural trend. For example, at least one high-resolution first boundary pixel group, a second boundary pixel group, and / or a first pixel group and a second pixel group among high-resolution temperature map data groups corresponding to a target inner pixel group are detected, The pixel temperature value of the temperature map data can be set naturally by setting the temperature value by interpolation method among at least one or more pixel groups.

3.3.2 How to Detect the Properties of Pixels to Set Temperature Values

Hereinafter, a process of detecting a property of a pixel and / or a pixel group to which a temperature value is to be set in order to set a temperature value of a pixel and / or a pixel group in the temperature map data of the column image generating unit 1360 will be described.

3.3.2.1 Detecting Pixel Attributes in Low-Resolution Temperature Map Data

FIG. 18 is a diagram showing pixels and an object in low-resolution temperature map data corresponding to a boundary line according to an embodiment of the present invention.

Hereinafter, a process of detecting the attribute of a pixel for setting a temperature value in the low-resolution temperature map data of the thermal image generating unit 1360 will be described with reference to FIG.

The thermal image generating unit 1360 may detect a pixel having an attribute in the vicinity of at least one boundary line in the temperature map data. For example, it is possible to detect pixels in the vicinity of at least one boundary where the temperature value should be set with reference to the boundary line detected by the boundary line detecting operation (S1010) in the low-resolution temperature map data. (2,1), (3,1), (1,2), (2,2), (1,3), (3,3), (4,3), 2, 4) and (3, 4) pixels contain the boundary of the map data, so that it can be determined as a low-resolution boundary pixel.

On the other hand, the thermal image generating unit 1360 may not regard the low resolution boundary pixel as a low resolution boundary pixel even if there are few pixels corresponding to the boundary of the map data in the pixels of the low resolution temperature map data. For example, when the resolution difference between the low-resolution temperature map data and the map data is 4x4, and pixels corresponding to the boundary line of the map data exist in the pixels of the low-resolution temperature map data, pixels of the low- It may not be viewed as a pixel. For example, the (2,1), (1,2), and (3,3) pixels of the low-resolution temperature map data may not be judged as a boundary pixel because they include few pixels corresponding to the boundary line.

In addition, the thermal image generating unit 1360 can detect a pixel having an attribute inside or outside the object, which is not an attribute near the boundary line in the temperature map data. For example, it is possible to determine a pixel having a pixel address and / or a pixel address existing at one side of a boundary of a pixel address and / or a boundary of a low-resolution temperature map data that does not correspond to the low- And a pixel having a pixel address existing on the other side opposite to the one side can be determined to detect the low resolution second pixel. (4,1), (3,2), (4,2), (2,3), (1,4), and The pixel can be said to be a low resolution first pixel and the (1,1) and (4,4) pixels can be said to be a low resolution second pixel.

In addition, the thermal image generating unit 1360 can determine the pixel of the low-resolution temperature map data having the attribute in the vicinity of the boundary line but not the boundary pixel as a pixel having an attribute inside or outside the object. For example, the pixel may be determined as the first pixel and / or the second pixel according to the user and / or thermal image generating unit 1360. Also, for example, if it is determined that the first pixel is classified into a low resolution second pixel when it is close to the first resolution pixel and the first pixel is classified into a low resolution first pixel when the resolution is close to the second pixel, (1,2) pixels can be classified as a low-resolution second pixel because the pixel corresponding to the boundary line in the pixel having the (1,2) pixel address is close to the first pixel, ) Pixel is close to the second pixel, it can be the first low-resolution pixel.

3.3.2.2 Detecting pixel attributes in high resolution temperature map data

FIG. 19 is a diagram showing pixels and an object in high-resolution temperature map data corresponding to a boundary line according to an embodiment of the present invention. FIG.

Hereinafter, a method of detecting pixels in the high-resolution temperature map data of the thermal image generating unit will be described with reference to FIG.

The thermal image generating unit 1360 can detect a group of pixels having a property in the vicinity of the boundary line among the pixel groups in the high-resolution temperature map data. For example, A boundary pixel group, and a second boundary pixel group existing on the other side opposite to the one side. Also, for example, if one side is referred to as the object inward direction, the boundary pixel groups (2-1, 2-1), (2-1, 2-2) of the high-resolution temperature map data existing within the (2,2) (2-2, 2-2), (2-2, 2-2), (2-2, 2-2), and (2-1, 2-1), (2-1, 2-2), and (2-2, 2-1) can be determined as the second boundary pixel group.

In addition, the thermal image generating unit 1360 can detect a pixel group having an attribute inside or outside the object in the high-resolution temperature map data. For example, a group of pixels not corresponding to the high-resolution boundary pixel group may be determined to detect the first high-resolution pixel group, and a second high-resolution pixel group existing on the opposite side of the one side may be detected. Also, for example, the thermal image generating unit 1360 generates (3-1, 2-1), (3-1), , 2-2), (3-2, 2-1), and (3-2, 2-2) are all located at one side of the boundary line.

The process of detecting the attribute of the pixel in the temperature map data may include a process other than the listed process.

3.3.3 Detailed description for setting the temperature value of the pixels in the temperature map data

Hereinafter, in the &quot; Table of Contents 3.3.1 &quot;, a thermal image generation unit 1360 generates a thermal image by using a pixel and / or a pixel group definition and address, for example, I want to.

3.3.3.1 Setting the temperature value near the boundary line in the high-resolution temperature map data

20 is a diagram illustrating a method of setting a temperature value of a boundary pixel group of high-resolution temperature map data according to an embodiment of the present invention.

20, the thermal image generating unit 1360 generates high resolution first boundary pixel groups (3-3,1-2), (3-1,1-3), (3-2,1-3), and (3-3,1-3) is determined based on the temperature value of the (3, 2) pixel among the at least one low-resolution first pixel adjacent to the (3,1) pixel which is the corresponding low- Method can be set.

(3-3,1-2), (3-1,1-3), (3-2,1-3), (3-3,1-3) pixel groups of high resolution and high resolution, May be set based on the temperature value of the (3-a, 2-b) pixel group which is at least one high-resolution first pixel group adjacent to the (3,1) pixel which is the corresponding low resolution boundary pixel.

Hereinafter, a method of setting the temperature value by the method of extrapolation through Fig. 19 will be described in detail by way of example.

21 is a diagram illustrating a method of setting a temperature value by extrapolating boundary pixel groups of high-resolution temperature map data according to an embodiment of the present invention.

Referring to FIG. 21, high resolution first boundary pixel groups (3-3,1-2), (3-1,1-3), (3-2,1-3), (3-3,1-3 ) Pixel group can be set as the temperature value 50 of the (3, 2) pixel which is the first resolution pixel of the first resolution, or (3-1, 1-3) (3-2,1-3) The temperature value of the (3-3,1-3) pixel group can be set to 48, and the temperature value of the (3-3,1-2) pixel group can be set to 46. [

In addition, the degree of the trend may be determined by the judgment of the input and / or thermal image generating unit 1360 of the user.

Meanwhile, the thermal image generation unit 1360 according to an exemplary embodiment of the present invention generates a thermal image based on the first pixel and the second pixel based on the temperature values of the pixel group and / or the second boundary pixel group of the high- Or by taking an average of the temperature values calculated by extrapolating two times based on the case where the number of second pixels is two or more, for example.

3.3.3.2 Set the target internal temperature value in the high-resolution temperature map data

22 is a diagram illustrating a method of setting a temperature value of a pixel group of high-resolution temperature map data according to an embodiment of the present invention.

Referring to FIG. 22, the thermal image generator 1360 generates high resolution first boundary pixel groups (3-3,1-2), (3-1,1-3), (3-2,1-3), 3-3, 1-3) The temperature value of the pixel group is extrapolated based on the temperature value of the (3, 2) pixel among the at least one low-resolution first pixel adjacent to the (3,1) (3-3,1-2), (3-1,1-3), (3-2,1-3), (3-3,1-3), and 1-3) pixels and high resolution first pixels (3-1, 2-1), (3-2, 2-1), (3-3, 2-1), ), (3-2,2-2), (3-3,2-2), (3-1,2-3), (3-2,2-3), (3-3,2-3 ) You can set the temperature value by interpolation between pixels.

Hereinafter, a method of setting the temperature value by the extrapolation method will be described in detail with reference to FIG.

23 is a diagram illustrating a method of setting a temperature value by interpolating boundary pixel groups of high-resolution temperature map data according to an embodiment of the present invention.

Referring to FIG. 23, the thermal image generator 1360 calculates the temperature of the (3-3,1-2) pixel group, which is the high resolution first boundary pixel group, by 46, (3-1,1-3) , 1-3) When the temperature value of the (3-3,1-3) pixel group is 48, the gradual trend is (3-1, 2-1) (3-2,2-1) (3-3,2 -1) pixel group can be set to 49.

The order of setting the temperature values of the high-resolution temperature map data of the thermal image generating unit 1360 according to the embodiment of the present invention is not limited to the order listed above. For example, extrapolation methods and interpolation methods may not be orderly. After the temperature value of the pixel group of the high-resolution temperature map data is set through the interpolation method, the temperature value of the boundary pixel group of the high-resolution temperature map data can be set by an extrapolation method.

The operation of each component of the listed column image generating unit 1360 according to an embodiment of the present invention can be operated regardless of the order described.

In addition, the operation of setting the temperature value of the pixels in the temperature map data of the listed column image generating unit 1360 according to the embodiment of the present invention is not limited to the above-described operations, The method of setting the temperature value of the temperature map data according to an embodiment of the present invention is not limited to the methods listed above but may be other methods.

<Temperature Map Data Generation Method of Temperature Map Data Generation System>

24 is a flowchart showing a method of generating temperature map data in an embodiment of the present invention.

Referring to FIG. 24, the temperature map data generation method (S2400) acquires low resolution temperature map data and high resolution map data (S2410), detects boundary lines of map data (S2420), generates high resolution temperature map data (S2430) Setting of the temperature value of the map data (S2440), and display of the temperature map data (S2450).

In an embodiment of the present invention, steps S2410 to S2450 may be performed simultaneously, but either step may be performed earlier in time. Steps S2410 to S2450 may all be performed, but steps S2410 to S2450 are not always performed at all, and only at least one of steps S2410 to S2450 may be performed.

Hereinafter, each step will be described in detail.

In step S2410 of obtaining the low-resolution temperature map data and the high-resolution map data, the infrared sensor module 1200 and the visible light sensor module receive a resolution setting command or a resolution setting command from the image processing apparatus, An image can be captured.

If it is difficult to detect the boundary of the map data because the pixel value of the acquired map data is lower than or equal to the predetermined threshold value in step S2420 of detecting the boundary of the map data, .

In the step of generating the high-resolution temperature map data (S2430), at least one of the generated high-resolution temperature map data may be stored in the storage unit 1350 of the image processing apparatus 1300.

In the step of setting the temperature value of the temperature map data (S2440), there are various methods of setting the temperature value of the temperature map data depending on the user's control. There may be a change in the method of setting the temperature value of the pixel in the temperature map data by improvement of the software, firmware, etc. of the image processing apparatus 1300. [

In the step S2470 of displaying the temperature map data, the color data H (t) corresponding to the temperature value of the pixel of the target data of the pixel of the map data and / or the temperature map data, received from the user so as to select the type of the color data to be displayed, ) · Saturation (S) · Brightness (V) model, Red (R) · Green (G) · Blue (B) Or the like so as to display at least one visible light image and / or a thermal image.

In the creating method and browsing method according to the present invention described above, the steps constituting each embodiment are not essential, and therefore, each embodiment can selectively include the above-described steps. Moreover, each step constituting each embodiment is not necessarily performed according to the order described, and the step described later may be performed before the step described earlier. It is also possible that each step is repeatedly performed during operation.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be apparent to those skilled in the art that changes or modifications may fall within the scope of the appended claims.

1000: thermal imaging system
1100: Visible light sensor module
1110:
1120: Image sensor
1130:
1140:
1150:
1200: Infrared sensor module
1210:
1220: Infrared sensor
1230:
1240:
1250:
1300: Image Processing Equipment
1300a, 1300b, 1300c: extrapolated pixel address
1310:
1320:
1330:
1340:
1350:
1360: Thermal image generating unit
1400a, 1400b, 1400c: interpolation pixel address
2100: Low resolution pixels
2110: Border pixel
2120: Low resolution first pixel
2130: low-resolution second pixel
2200: High-resolution pixel
2210: Border pixel group
2211: first boundary pixel group
2212: second boundary pixel group
2220: Pixel group
2221: first pixel group
2222: second pixel group

Claims (14)

Obtaining a high-resolution visible light image and a low-resolution thermal image corresponding to each other;
Detecting a boundary line of a specific object from the high-resolution visible light image, and converting the low-resolution thermal image into a high-resolution thermal image based on the high-resolution visible light image;
Setting a temperature value of a pixel of the high resolution thermal image based on the low resolution thermal image,
Wherein the setting step comprises:
Determining a boundary pixel corresponding to the boundary line among the pixels of the low resolution column image based on the detected boundary line and determining a boundary pixel group of the high resolution column image corresponding to the boundary pixel,
The temperature value of the pixels of the high resolution thermal image other than the boundary pixel group is determined based on the temperature value of the pixel of the low resolution thermal image corresponding to the remaining pixels,
Wherein a temperature value of an inner pixel of the specific object including a pixel corresponding to the boundary line of the boundary pixel group is a temperature value of a pixel located inside the specific object adjacent to the boundary pixel, Is determined as a basis,
The temperature value of the outer pixel of the specific object among the boundary pixel groups is determined based on the temperature value of the pixel located outside the specific object among the pixels of the low resolution thermal image adjacent to the boundary pixel
How to create a thermal image.
Acquiring a visible light image and a low resolution thermal image corresponding to the visible light image and having a lower resolution than the visible light image;
Detecting a boundary line in the visible light image; And
Generating a high-resolution thermal image having a higher resolution than the low-resolution thermal image, the low-resolution thermal image being composed of high-resolution pixels obtained by dividing low-resolution pixels of the low-resolution thermal image using the boundary detection result and the temperature data of the low-
Setting a temperature value of the high resolution pixel of the high resolution thermal image based on temperature data of the low resolution pixel of the low resolution thermal image,
Wherein the setting step includes setting a temperature value of the high resolution pixel belonging to the low resolution pixel corresponding to a pixel representing the boundary line in the visible light image to a temperature value of another low resolution pixel adjacent to the low resolution pixel to which the high resolution pixel belongs Determining
How to create a thermal image.

3. The method of claim 2,
In the setting step, the adjacent other low-resolution pixels are located on the same side as the high-resolution pixel with respect to the boundary line.
How to create a thermal image.
3. The method of claim 2,
In the setting step, the adjacent other low-resolution pixels are located in an area to which the high-resolution pixel defined by the boundary line belongs
How to create a thermal image.
3. The method of claim 2,
The boundary line is related to a specific object,
In the determining, the high resolution pixel and the adjacent low resolution pixel are located within the specific object
How to create a thermal image.
3. The method of claim 2,
The temperature value is calculated by extrapolating using the temperature data of the other low resolution pixels
How to create a thermal image.
3. The method of claim 2,
Determining the temperature value of the high resolution pixel belonging to the low resolution pixel which does not correspond to the pixel representing the boundary line in the visible light image using the temperature data of the low resolution pixel to which the high resolution pixel belongs More included
How to create a thermal image.
8. The method of claim 7,
The temperature value of the high resolution pixel belonging to the low resolution pixel not corresponding to the pixel representing the boundary line is calculated by interpolation using the temperature data of the low resolution pixel
How to create a thermal image.
3. The method of claim 2,
Determining at least one of color elements of a predetermined color space based on the temperature value and displaying the high resolution thermal image
How to create a thermal image.
10. The method of claim 9,
Wherein the color space includes at least one of RGB system, HSV system, CMYK system, CIE system, PMS system, Munsell color system, NCS system, and YIQ system
How to create a thermal image.
3. The method of claim 2,
And displaying the detected boundary on the high resolution thermal image
How to create a thermal image.
3. The method of claim 2,
Characterized in that the high resolution thermal image has the same resolution as the visible light image
How to create a thermal image.
3. The method of claim 2,
Displaying at least one of the visible light image, the low resolution thermal image, and the high resolution thermal image
How to create a thermal image.
A visible light sensor module for capturing a high-resolution visible light image including a specific object;
A thermal image camera for capturing a low-resolution thermal image including the specific object; And
Resolution thermal image; converting the low-resolution thermal image into a high-resolution thermal image based on the high-resolution visible light image; and setting a temperature value of a pixel of the high-resolution thermal image based on the low- The module comprising:
The module
Determining a boundary pixel corresponding to the boundary line among the pixels of the low resolution column image based on the detected boundary line, determining a boundary pixel group of the high resolution column image corresponding to the boundary pixel, Wherein the temperature value of the remaining pixels excluding the boundary pixel group is determined based on a temperature value of a pixel of the low resolution column image corresponding to the remaining pixels, The temperature value of the inner pixel of the object is determined on the basis of the temperature value of the pixel located inside the specific object adjacent to the boundary pixel among the pixels of the low resolution thermal image, Of the pixels of the low resolution thermal image, Thermal image forming device, characterized in that determining on the basis of the temperature value of the pixel located at the outer side of the particular object group.

Images