JP6059012B2 - Optical communication apparatus, optical communication method, and skin imaging system - Google Patents

Description

  The present invention relates to an optical communication device, an optical communication method, and a skin imaging system using a solid light emitting element and an imaging element.

2. Description of the Related Art Conventionally, a conversion lens that is attached to a mobile electronic device with a camera such as a mobile phone with a camera (including a smartphone) or a tablet electronic device with a camera is known (for example, see Patent Document 1).
By attaching the conversion lens to the lens (master lens) on the electronic device side, the focal length can be changed to the wide angle side or the telephoto side even if the camera lens of the portable electronic device is a single focus lens. In some cases, a macro imaging may be possible using a lens having a close-up function (macro imaging function) as a conversion lens.

  Skin cameras are also known for analyzing the skin condition by magnifying the skin of the face. For example, the camera functions as a skin camera on a mobile phone camera. There is also known a conversion lens that adds a function to perform this function. In this case, for example, the skin analysis result may be sent back by e-mail by sending it to a company that analyzes the skin image by e-mail or uploading it to a website server. Easy to do.

  When a skin image taken from a smartphone with a conversion lens as described above is sent to a predetermined server (for example, a server of a cosmetics-related website), a service is performed to send back the analysis result of the skin image. In some cases, for example, it is possible to provide a service free of charge or at a low price to a customer who purchased a conversion lens.

  In this case, for example, the conversion lens accompanied by the service as described above and the fake conversion lens of substantially the same level are sold at a low price, and the customer who purchased the fake conversion lens receives the service described above. Can be considered.

  In this case, there is a possibility that the sales of the conversion lens will decrease, or that the operating cost will increase due to the analysis of the skin image captured using the fake conversion lens. Therefore, an authentication method is required that can distinguish between a skin image transmitted by a purchaser of a regular conversion lens and a skin image transmitted by a purchaser of a fake conversion lens.

  In addition, for example, when a cosmetic company sells a conversion lens as a set with a specific cosmetic product for skin care in a state where the above-described service is established, in addition to the above-mentioned service, the cosmetic company sells a set. For example, you may want to obtain only the skin image of the conversion lens.

In this case as well, it is necessary to identify the skin image of the customer who purchased the conversion lens alone and the skin image of the customer who purchased the conversion lens as a set.
Distinguishing between skin images taken with a regular conversion lens and skin images taken with a non-regular conversion lens, conversion lenses associated with a specific company (operator) by the above-mentioned set sales, etc., single item sales, etc. As a method of distinguishing conversion lenses that are not associated with a specific company, it is conceivable to assign identification information such as a serial number or ID number to each conversion lens. In addition, when identifying a conversion lens by single item sale and set sale, it is necessary to set identification information so that it can be classified into two.

When such identification information is used, it is necessary to transmit the identification information together with the skin image to a server that analyzes the skin image.
In this case, a method may be considered in which a customer inputs a serial number described in a conversion lens or a manual attached to the conversion lens from a smartphone when transmitting a skin image.

JP 2012-8283 A

Conventionally, it has not been performed to attach identification information to a conversion lens and transmit the identification information together with photographed image data.
As mentioned above, when a customer (user) inputs identification information to a smartphone, the user is dissatisfied that it takes time to input, or wrong identification information is registered due to an input error, There is a possibility that the identification information cannot be registered, or that the identification information is analyzed due to the leakage of the identification information through a user or a manual in which the identification information is described.

Therefore, even if the user does not perform any particular operation, the identification information individually assigned to the conversion lens is automatically recognized on the smartphone side, and the identification information is automatically provided to the server of the website that provides the above-mentioned service. There is a need to send.
That is, when skin image data is transmitted from a smartphone, identification information is required to be automatically transmitted or read from the conversion lens side to the smartphone side. In this case, it is preferable that identification information given to the conversion lens is obtained on the smartphone side at as low a cost as possible.

  The present invention has been made in view of the above circumstances, and provides an optical communication device, an optical communication method, and a skin imaging system suitable for obtaining identification information of a conversion lens in a portable electronic device such as a smartphone. With the goal.

In order to solve the above problems, an optical communication apparatus of the present invention includes an image sensor that captures a moving image at a set frame rate, and an image analysis unit that analyzes image data of each frame of the moving image captured by the image sensor. A receiving device comprising:
A flashable solid state light emitting device, and lighting and extinguishing at intervals of a time equal to or longer than one line at the frame rate every predetermined time when one or several frames of the solid state light emitting device are imaged at the frame rate A transmission device having a light emission control means for turning on and off with a plurality of blink patterns consisting of:
The light emission control means of the transmitting device relates to the relationship between the blinking pattern storage means storing each blinking pattern and each value of the digital signal in association with each other, and the blinking pattern stored in the blinking pattern storage means and the value. And a transmission conversion means for sequentially converting each value of the digital signal into the blinking pattern of the solid state light emitting device,
And the light emission control means blinks the solid state light emitting element based on the blinking pattern converted by the transmission conversion means,
The image analysis means of the receiving device, as an analysis result of the image data, is a light / dark pattern consisting of high and low brightness appearing along the vertical direction of the imaged image data due to blinking of the blinking pattern of the solid state light emitting element. Seeking
The receiving device includes: a light / dark pattern storage means for storing each light / dark pattern and each value of the digital signal in association with each other based on a correlation between the light / dark pattern and the blinking pattern obtained by the image analyzing means; Receiving conversion means for converting the light / dark pattern sequentially analyzed by the image analysis means into each value of a digital signal based on the relation between the light / dark pattern stored in the pattern storage means and the value; To do.

  According to such a configuration, the transmission device converts each value of the digital signal into a corresponding blinking pattern. Thus, while the solid-state light emitting element emits light with each blinking pattern in the transmission apparatus, the imaging element of the reception apparatus captures a moving image in a state where the light emission of the solid-state light emitting element is imaged. In addition, if the light of a solid light emitting element is included in the image imaged at this time, it is not necessary to image a solid light emitting element directly.

  As described above, when the imaging element captures an image while the solid state light emitting element is blinking in each blinking pattern, the captured image data has a brightness level corresponding to the blinking of the solid state light emitting element along the vertical direction. A change in brightness is imaged. The light / dark pattern as the change in light / dark corresponds to the above-described blinking pattern. If each value of the digital signal is assigned to the blinking pattern, each value can be read from the light / dark pattern corresponding to each blinking pattern. Each value here is preferably, for example, a value of 2 bits or more, for example, more than 1 bit data of 0 and 1, for example, each blinking pattern is 3 bits (0 to 7) or 4 bits. It is good also as a thing corresponding to the value (0-15).

As a result, the solid-state light emitting element is arranged on the transmission side, and the imaging element capable of capturing a moving image is arranged on the reception side, thereby enabling optical communication. In this case, it is possible to send and receive a digital signal with a bit number of 2 bits or more in each frame, instead of sending and receiving 1 bit data for each frame, whether it is bright or dark for each frame of the moving image. And high-speed communication becomes possible.
Note that there are known imaging devices that read one frame at a time and one frame one line at a time when reading the accumulated charge from each pixel. It is necessary to use an image sensor that reads the charge of each pixel.
In the present invention, it is preferable to use, for example, a C-MOS sensor that reads line by line as the image sensor.
Note that the present invention can also be applied to a configuration in which charges are read by several lines such as two lines.

In the above configuration of the present invention, the light emission control means of the transmission device controls turning on and off of the solid state light emitting element asynchronously with respect to a synchronization signal of a video signal output from the imaging element,
Each blinking pattern stored in the blinking pattern storage means of the transmitter is defined by the number of repetitions of turning on and off included in the predetermined time,
Each light and dark pattern stored in the light and dark pattern storage means of the receiver is the number of bright and / or dark bands along the horizontal direction represented by the level of brightness appearing in the vertical direction of the imaged image data. It is preferable that it is prescribed | regulated by.

  According to such a configuration, digital signals can be transmitted and received without synchronizing with the synchronization signal of the image sensor, and it is not necessary to provide a configuration for transmitting a synchronization signal from the reception side to the transmission side. Can be reduced, and the structure of the optical communication device can be simplified. In this case, since the timing of the image sensor frame and each line of the frame cannot be matched with the timing of lighting and extinguishing of the solid state light emitting element, for example, the solid state light emitting element is lit depending on the imaging time of each frame based on the frame rate. And turn off.

  In this case, for example, with 2 frames as one set, for example, a blinking pattern in which lighting and turning off are repeated 3 times in the time of 1 frame, a blinking pattern that is repeated 5 times, a blinking pattern that is chestnut-cleaned 7 times, etc. The solid state light emitting device is controlled with such a blinking pattern.

  On the other hand, the moving image data has bright and dark stripes corresponding to repeated turning on and off of the blinking pattern of the solid-state light emitting element in the image data of at least every other frame of the moving image data. By counting the number of bright bands and dark bands of the stripes, the light / dark pattern of the image data corresponding to the blinking pattern of the solid state light emitting device can be read, thereby transmitting a digital signal from the transmission side to the reception side. It will be done.

In other words, it is possible to send and receive data asynchronously without causing the solid-state light emitting element to emit light in synchronization with the synchronizing signal of the image sensor, and a structure for transmitting and receiving a synchronizing signal from the receiving side to the transmitting side is required. In addition, optical communication using a digital camera can be realized at low cost.
This method basically changes the frequency of blinking (brightness and darkness) within a predetermined time. For example, when the number of repetitions of blinking within a predetermined time is small, the frequency is low, and when the number of repetitions is large. High frequency. In this case, when representing a digital signal, the simplest case is 0 for a low frequency and 1 for a high frequency. This is to convert the change in frequency into a digital signal, and by using frequency shift keying (FSK), the number of repetitions of blinking within a predetermined time is converted into a digital signal. To do. Therefore, according to the present invention, on the transmission side, the digital signal is converted into high and low frequencies of repetition of turning on and off the LED per unit time (predetermined time), and this is imaged by the image sensor on the reception side. In the image for each frame, the high and low repetition frequency of the light and dark appearing corresponding to the lighting and extinguishing of the LED is converted into a digital signal.

Moreover, in the above configuration of the present invention,
The receiving apparatus includes a synchronization output unit that outputs a synchronization signal in synchronization with a synchronization signal of a video signal output from the imaging device,
The transmission device includes synchronization input means for receiving a synchronization signal transmitted from the synchronization output means,
The light emission control unit of the transmission device turns on and off the solid state light emitting device in synchronization with each frame and each line of each frame imaged by the imaging device by a synchronization signal input to the synchronization input unit. Control
Each flashing pattern stored in the flashing pattern of the transmitter is defined by a lighting time width and a lighting time width ,
Each light and dark pattern stored in the light and dark pattern storage means of the receiving device has a width of a bright band and a width of a dark band along the horizontal direction represented by high and low brightness appearing in the vertical direction of the captured image data. It is preferable that it is prescribed | regulated by these.

  According to such a configuration, the transmitting device side can control lighting and extinguishing of the solid state light emitting elements corresponding to each frame of the imaging device and each line of each frame of the receiving device. In addition, the value can be expressed by the width of each band, for example, instead of the value by the number of bright and dark bands appearing in the captured image data. For example, in a pattern where light and dark are repeated by dividing a bright band into a narrow band and a wide band and dividing a dark band into a narrow band and a wide band, such as a barcode, a narrow band is 0 and a wide band is 1 It is possible to represent a 1-bit value for each band. That is, communication can be performed by the number of bits that are repeatedly turned on and off within the time of one frame, and information of a large number of bits can be transmitted and received in one frame. Thereby, the communication speed can be increased. Note that the classification of the width of the band is not limited to two types, such as a narrow width and a wide width, and the number of band widths may be more than two.

Further, in the above configuration of the present invention, the solid-state light emitting element of the transmission device is a three-color LED that can be turned on and off for each color,
The light emission control means of the transmission device transmits a digital signal by a blinking pattern consisting of independent lighting and extinction for each color of the three-color LED,
Each blinking pattern stored in the blinking pattern storage means of the transmitting device is defined by turning on and off each color of the three-color LED,
The receiver has a color image sensor,
The image analysis means of the receiving device obtains a light and dark pattern that appears in the vertical direction corresponding to each color of the three-color LED of the imaged image data,
Each light and dark pattern stored in the light and dark pattern storage means of the receiving device appears in the vertical direction of the imaged image data and is represented by the luminance for each color corresponding to the emission color of the three-color LED. It is preferable to be defined as a bright band and a dark band along the horizontal direction.

  According to such a configuration, the solid-state light emitting element is a three-color LED that can be turned on and off independently for each color, and the image pickup element corresponds to the color, so that basically three times the information is obtained. Can be sent and received. Note that information may be individually transmitted in a blinking pattern for each color, or a configuration in which data of 3 bits is transmitted by a combination of lighting and extinguishing of three colors may be employed.

Further, the optical communication method of the present invention includes an imaging device that captures a moving image at a set frame rate, and a receiving device that includes an image analysis unit that analyzes image data captured by the imaging device;
A flashable solid state light emitting device, and lighting and extinguishing at intervals of a time equal to or longer than one line at the frame rate every predetermined time when one or several frames of the solid state light emitting device are imaged at the frame rate An optical communication method performed in an optical communication device comprising a transmission device having a light emission control means for turning on and off with a plurality of blink patterns consisting of:
The light emission control means of the transmission device sequentially converts the digital signal into the blinking pattern based on the relationship between each blinking pattern and each value of the digital signal, and based on the blinking pattern, Control on and off,
The imaging device of the receiving device captures a moving image by the imaging device in a state where the solid state light emitting device is turned on or off in the blink pattern,
Analyzing the image data of the captured moving image to obtain a light and dark pattern consisting of high and low brightness appearing along the vertical direction of the image data,
The receiving device obtains the light / dark pattern sequentially obtained based on the correlation between the light / dark pattern obtained and the blinking pattern based on the correlation between the light / dark pattern and each value of the digital signal. It is characterized by converting to a value.

In the above configuration of the present invention, the light emission control means of the transmission device controls turning on and off of the solid state light emitting element asynchronously with respect to a synchronization signal of a video signal output from the imaging element,
The transmission device defines each blinking pattern by the number of repetitions of turning on and off included in the predetermined time,
The receiving apparatus preferably defines each light / dark pattern by the number of bright bands and / or dark bands along the horizontal direction represented by the level of brightness appearing in the vertical direction of the captured image data.

In the above configuration of the present invention, the receiving device outputs a synchronizing signal in synchronization with a synchronizing signal of a video signal output from the imaging device,
The transmitter receives a synchronization signal transmitted from the receiver;
The light emission control means of the transmission device controls lighting and extinguishing of the solid state light emitting element in synchronization with each frame imaged by the imaging element and each line of each frame by the received synchronization signal,
The transmission device defines each blinking pattern by a lighting time width and a lighting time width ,
The receiving apparatus preferably defines each light / dark pattern by a width of a bright band and a width of a dark band along a horizontal direction represented by a level of brightness appearing in a vertical direction of the captured image data.

Further, in the above configuration of the present invention, the solid-state light emitting element of the transmission device is a three-color LED that can be turned on and off for each color,
The light emission control means of the transmission device transmits a digital signal by a blinking pattern consisting of independent lighting and extinction for each color of the three-color LED,
The transmitter defines each blinking pattern by turning on and off each color of the three-color LED,
The receiver has a color image sensor,
The image analysis means of the receiving device obtains a light and dark pattern that appears in a vertical direction corresponding to each color of the three-color LED of the imaged image data,
The light receiving device appears in the vertical direction of the image data obtained by capturing each light and dark pattern, and is bright and dark along the horizontal direction represented by the luminance for each color corresponding to the emission color of the three-color LED. It is preferable to be defined by a band.

  In the optical communication methods such as these, the same operational effects as the corresponding optical communication apparatus described above can be obtained.

A skin imaging system according to the present invention includes the above-described optical communication device, includes a digital camera having the imaging element, controls the imaging element, and captures a moving image captured by the imaging element. Image analysis means for analyzing image data of each frame of data, and based on the correlation between the light / dark pattern and the blinking pattern obtained by the image analysis means, the light / dark pattern and each value of the digital signal are associated with each other. Based on the relationship between the stored light / dark pattern storage means and the light / dark pattern stored in the light / dark pattern storage means and the value, the light / dark pattern sequentially analyzed by the image analysis means is converted into each value of a digital signal. A portable electronic device comprising control means functioning as reception conversion means;
The LED module for photographing illumination as the solid state light emitting element, a conversion lens used for photographing a skin image, and a lens module comprising the light emission control means for controlling light emission of the LED for lighting,
The light emission control means stores the blinking pattern and each value of the digital signal in association with each other, and the flashing pattern storage means stores the blinking pattern and the value stored in the blinking pattern storage means. It functions as transmission conversion means for sequentially converting each value of the signal into the blinking pattern of the solid state light emitting element.
According to such a configuration, the same operational effects as those of the above-described optical communication device can be achieved.

A skin image system according to the present invention includes a portable electronic device including a camera, and a conversion lens that is detachably attached to a position of a master lens of the camera of the portable electronic device and for taking close-up of the skin by the camera. A skin photographing system comprising:
A cylindrical lens housing that supports the conversion lens;
An opening for facing the skin to be photographed, formed at the tip of the lens housing;
Provided at the peripheral edge of the opening, showing identification information that enables the conversion lens to be identified on the surface facing the camera, and comprising an image recognizable code,
The code is photographed when photographing the skin facing the opening with the camera, and the identification information can be read from the code photographed together with the skin.

  According to such a configuration, for example, a bar code or a two-dimensional bar code that can be recognized as a code including identification information for identifying a conversion lens is provided on the peripheral edge of the opening, and the code is simultaneously displayed together with the skin. By photographing, the skin and the code are photographed as one image, and the code for identifying the conversion lens can be read from the conversion lens to the portable electronic device side. Thereby, even if there is no data communication means between the conversion lens and the portable electronic device, the identification information is transmitted from the conversion lens side to the portable electronic device side. Further, each skin image always includes an image of a code that can identify the used conversion lens, so that the used conversion lens can be identified at any time.

  According to the present invention, the transmission device includes the solid-state light emitting element, and the reception device includes the digital camera, thereby enabling optical communication. As the digital camera on the reception side, a camera of a portable electronic device such as a smartphone is used. Communication can be performed.

It is a front view which shows the skin imaging system as an optical communication apparatus which concerns on 1st Embodiment of this invention. It is a front view which shows the lens module as a transmission apparatus of the said skin imaging system. It is sectional drawing which shows the said lens module. It is sectional drawing which shows the lens housing | casing part of the said lens module. It is sectional drawing which shows the lens housing | casing part of the said lens module. It is a top view which shows the LED board of the said lens module. It is a block diagram which shows the said skin imaging system. It is a figure for demonstrating the optical communication method in the said skin imaging system. It is a block diagram which shows the skin imaging system as an optical communication apparatus which concerns on 2nd Embodiment of this invention. It is a figure for demonstrating experiment of the analysis method of the imaged image in the optical communication apparatus concerning the Example of this invention. It is a figure for demonstrating experiment about the analysis method of the image imaged in the said optical communication apparatus. It is a figure for demonstrating the experiment of the analysis method of the imaged image in the said optical communication apparatus. It is sectional drawing which shows the said lens housing | casing part of 3rd Embodiment. It is a figure which shows the skin image and barcode image which were imaged with the skin image system of 3rd Embodiment.

The first embodiment of the present invention will be described below with reference to the drawings.
As shown in FIGS. 1 to 7, the skin imaging system of this embodiment includes a smartphone 1 (FIG. 1 and FIG. 1) as a portable electronic device including a digital camera (camera 2 shown in FIG. 2 including an imaging device). 7) and two lenses 11a and 11b (shown in FIGS. 3 to 5 and 7) that are attached to the camera 2 of the smartphone 1 to enlarge and close-up the skin 24 (shown in FIG. 7). And a lens module 10 provided with LEDs 12, 13 (illustrated in FIGS. 3 to 7) for photographing illumination.

1 and 2, the lens module 10 includes a front surface portion of a housing 10 a excluding a later-described lens housing 20 including a barrel 20 a (shown in FIGS. 3 to 5) that supports the conversion lens 11. Although not shown, the circuit board 16b on which the internal battery 17, the power switch 19, and the electronic circuit unit 16a are mounted can be seen.
Moreover, this smart phone 1 and the lens module 10 comprise the optical communication apparatus, the smart phone 1 is a receiver, and the lens module 10 is a transmitter.

  The lens module 10 includes a housing 10a. The housing 10a is formed in a flat box shape excluding the lens housing 20, and is opposite to the front surface where the display (not shown) of the smartphone 1 is provided. The conversion lens 11 is arranged so as to overlap the lens portion of the rear camera 2.

  The case 10a is fixed to the smartphone 1 by, for example, a clip type or a band type. In the clip type, for example, a clip member (not shown) that sandwiches the smartphone 1 between the lens module 10 and the lens module 10 is provided. That is, the smartphone 1 is sandwiched by elastic force between the lens module 10 and a clip member fixed to the lens module 10.

In this case, the lens module 10 can be moved vertically and horizontally within an allowable range with respect to the back surface of the smartphone 1, and can correspond to the arrangement of the cameras 2 of each model of the smartphone 1.
In the band type, for example, a band (not shown) such as a rubber band is attached to a lens module, such as a wristwatch band, and the smartphone 1 is inserted into the band, whereby the lens module 10 is played on the smartphone 1. Also in this case, the conversion lens 11 can be moved vertically and horizontally with respect to the back surface of the smartphone 1.

The lens module 10 includes the conversion lens 11 and the LEDs 12 and 13 described above, an LED drive circuit 14 (shown in FIG. 7) for driving the LED 12, and an LED drive circuit 15 (shown in FIG. 7) for driving the LED 13.
In addition, the lens module 10 includes a lens housing 20 in which the conversion lens 11 is housed as shown in FIGS. The lens housing 20 includes a barrel 20 a that supports the conversion lens 11 on the base end side (the side that is attached to the camera 2), and the front end surface is a contact portion 20 b that contacts the skin 24 when the skin 24 is photographed. The outside light is blocked while the contact portion 20 b is in contact with the skin 24. Further, a rectangular (substantially square) opening 20c is formed in the contact portion 20b, and a portion of the skin 24 facing the opening 20c is photographed.

  As shown in FIGS. 3 to 5, the barrel 20 a supports the conversion lens 11 including the lens 11 a and the lens 11 b, and the base end portion (end portion on the side attached to the camera 2) of the lens housing 20. It is fixed inside. Moreover, the below-mentioned 2nd polarizing plate 22 is provided in the base end part of the barrel 20a. Also, in the lens housing 20, an LED substrate 12a (shown in FIGS. 3 to 6) including LEDs 12 and 13 that irradiate photographing illumination light toward the opening 20c of the contact portion 20b is supported. ing.

  The LED 12.13 is provided on the LED substrate 12a. Further, as will be described later, there is one LED 13 for photographing the texture of the skin 24 and two LEDs 12 for photographing the stain on the skin 24. Further, the two LEDs 12 on the LED substrate 12a are covered with a first polarizing plate 21 described later. Further, the LED drive circuits 14 and 15 are also provided on the LED substrate 12a. The LED drive circuits 14 and 15 may be provided on the circuit board 16b.

  As shown in FIGS. 3 to 5, the LED substrate 12 a is arranged so that the LEDs 12 and 13 directly illuminate the skin 24 facing the opening 20 c of the lens housing 20, but is reflected by the half mirror. By doing so, the light from the LEDs 12 and 13 may be applied to the skin 24 facing the opening 20c.

  Further, the lens module 10 includes an electronic circuit unit 16a including a CPU 16 serving as a light emission control unit that controls lighting and extinguishing of the LEDs 12 and 13 via the LED driving circuits 14 and 15. Note that a one-chip microcomputer is mounted on the electronic circuit unit 16a, and the one-chip microcomputer is provided with a CPU 16 (shown in FIG. 7) and storage means such as a ROM and a RAM.

  In addition, the lens module 10 includes a power supply circuit 18 (shown in FIG. 7) including a battery 17 for supplying power to the LED drive circuits 14 and 15, the CPU 16, and the like on a circuit board 16 b fixed to the housing 10 a. A switch 19 is provided. In addition, although the disk-shaped button battery was illustrated as the battery 17, an AAA battery may be sufficient as it, for example.

  Further, the two LEDs 12 and 13 described above have slightly different emission colors, and the LEDs 12 are used for imaging spots on the skin 24, and the emission color is substantially white. On the other hand, the LED 13 is used for imaging the texture of the skin 24, and has a strong yellow color with respect to the emission color of the LED 12 and a color close to the skin color.

  Further, the first polarizing plate 21 is provided in the spot photographing LED 12, and the second polarizing plate 22 is provided between the conversion lens 11 and the camera 2. This is for photographing a stain slightly below the surface of the skin 24, and the polarizing plate 21, 22 causes the skin 24 to reflect the reflected light inside the skin 24 when photographing the skin 24 stain with the LED 12. The reflected light on the surface is greatly reduced to make it easier to photograph spots. In the first polarizing plate 21 and the second polarizing plate 22, the polarization directions are orthogonal. The light of the LED 12 passes through the first polarizing plate 21 and is polarized, and this polarized light is maintained even if it is reflected by the skin 24, and is difficult to pass through the second polarizing plate 22. On the other hand, the light reflected inside the skin 24 is not polarized and passes through the second polarizing plate 22. The polarizing plates 21 and 22 may not be provided.

  In such a lens module 10, the control of the CPU 16 of the electronic circuit unit 16 a causes the LED 13 to emit light (turn on) when photographing and keep the LED 12 off, and cause the LED 12 to emit light while photographing a spot. Leave off.

  As is well known, the smartphone 1 has a function as a mobile phone and can make a call using a wireless line. In addition, the smartphone 1 has a display (not shown), can be connected to the Internet via a wireless line, can send and receive e-mails via the Internet, and can browse a website by a browser as an application (app). It has become. In addition, for example, a file can be uploaded to a server such as a website, and the file can be downloaded from the server such as a website.

  The smartphone 1 includes a control device (control means: illustrated in FIG. 7) 3 having a CPU, a ROM, a RAM, and the like, and can execute an application as a program. The control device 3 is connected to a flash memory 4 as a storage device, and can store downloaded apps, music files, moving image files, still image files, and other applications (programs).

In addition, the control device 3 can control the camera 2 to take a picture (capture), and can store the captured image data in the flash memory 4. Image data can be analyzed and processed by an application for image processing and image analysis.
The camera 2 has an image sensor and can capture still images and moving images. For example, the camera 2 captures images at a set frame rate such as 30 fps.

  For example, in a complementary metal oxide semiconductor (CMOS) image sensor as an image sensor of the camera 2, sensors for each pixel are arranged in a matrix.

  In other words, the sensor of each pixel is arranged for each line arranged in the horizontal direction and in the vertical direction. When reading the charge accumulated in the sensor of each pixel corresponding to the amount of incident light, for example, the charge of each sensor on the line is read for each line from the upper line in the vertical direction to the lower line. It is done. The read charge is output as a video signal.

  In this case, the amount of charge read from the sensor of each pixel corresponding to the synchronization signal is sequentially output as a signal. This synchronization signal includes one image by a Vsync signal (vertical synchronization signal) output in synchronization with a Pixel Clock signal (individual synchronization signal) corresponding to the output of charge from each sensor along the line in order. The output of the minute charge is started.

  When this Vsync signal is output from the image sensor, the first Hsync signal (horizontal synchronization signal: horizontal direction synchronization signal) is output, and the first horizontal line in the vertical (vertical) direction (topmost) (first in the horizontal direction). A signal based on the charge indicating the amount of received light is sequentially output from one sensor in the upper row) along one direction (for example, from left to right) for each Pixel Clock signal corresponding to each sensor. .

Further, the signal moves to the sensor on the next horizontal line on the basis of the next Hsync signal, and a signal corresponding to one sensor is output again along one direction for each Pixel Clock signal from the left side. Is done. Thereafter, for each Hsync signal, the signal moves to a line along the lower horizontal direction, and a signal corresponding to the above-described charge is output.
Then, when the output of the signal corresponding to the last line is completed, the video signal for one image (one frame) is output, and again based on the Vsync signal and the Hsync signal. A signal corresponding to the first line of the frame is output. By moving the video signal to the next line for each Hsync signal and arranging the signals corresponding to each sensor so as to shift to the next frame for each Vsync signal, a moving image can be displayed.

  The above-described lens module 10 shown in FIG. 7 functions as a transmission device of an optical communication device, and the CPU 16 as the light emission control means causes the LED 13 that is a solid light emitting element to blink in a blinking pattern for communication. Yes. (This is used because there is no polarizing plate in the optical system.) In the following description, the CPU 16 controls the light emission of the LED 13 when performing optical communication.

  Moreover, the smart phone 1 shown in FIG. 7 is a receiving side of an optical communication apparatus, and the mounted camera 2 functions as a receiving apparatus.

  If the transmitter device knows the timing of the Vsync signal and Hsync signal of the sensor chip, that is, if it can synchronize with the synchronization signal, the LED starts blinking at the timing of the Vsync signal, and the time for one frame is reached. (For example, in the case of 15 fps, 1/15 second) After the elapse of time, blinking for one frame is finished. This blinking assumes that the extinguishing time = lighting time. Here, the lens module 10 on the transmission side is set to 15 fps (set to 1/15 seconds for one frame), and the moving image setting is set to 15 fps by the application program on the smartphone 1 on the reception side. It shall be. In this case, the sensor can obtain an image of one frame with a stripe pattern corresponding to the blinking of the LED.

For example, consider a case of 8M, horizontal 3200 pixels, vertical 2400 pixels, and frame rate 15 fps.
When the initial state is off and lighting and extinguishing are repeated 5 times each for 1/15 seconds, a 1-frame image has a bright linear stripe from the top, a dark linear stripe below it, and this combination is 5 A repeated image is obtained, and the width of each stripe is 240 pixels (2400/10).
Therefore, if ten types of images having different stripe widths are prepared, the receiving side can recognize numerical values of 0 to 10 corresponding to the stripes.

Here, in the above example, the frame rate is set to 15 FPS, which is the same as that of the TV receiver, but can be changed as appropriate.
Note that since the image cut-out process is actually performed, only the central portion of the sensor size is used for recognition. However, for the sake of simplicity, the following description will be made assuming that the entire sensor is used.

First, the lens module 10 as a transmission device will be described. The CPU 16 is a one-chip microcomputer as described above, and includes a ROM. This ROM stores a blinking pattern as blinking pattern storage means.
In the case of a predetermined time for two frames at the above-described frame rate, for example, 15 fps, a plurality of blinking patterns are stored that repeatedly turn on and off with approximately the same time width in 2/15 seconds.

  If the lighting and extinguishing of the LED 13 can be controlled with the blinking pattern of turning on and off in synchronization with the Hsync signal and the Vsync signal at the time of imaging by the camera 2 described above, for example, one kind of blinking from 10 kinds of blinking patterns for each frame. By performing blinking with a pattern, it is possible to transmit one kind of blinking pattern among ten kinds of blinking patterns for each frame.

  However, here, since there is no means for outputting the synchronization signal output from the camera 2 of the smartphone 1 as the receiving device to the lens module 10 as the transmitting device, the time width of the entire blinking pattern of the LED 13 is determined when the camera 2 captures the image. You can only match the frame rate.

  In this case, except for the case of accidental synchronization with the blinking pattern of the time width of two frames, in the imaging by the camera 2, the start of blinking of the LED 13 with one kind of blinking pattern is the frame (first frame). In the next frame (second frame), the blinking of the LED 13 with one type of blinking pattern is captured over the next frame (second frame), and in the middle of the next frame (third frame). The blinking of the LED 13 with one kind of blinking pattern ends.

  Therefore, the blinking of the LED 13 with the same blinking pattern is captured in the second frame. When the LED 13 blinks continuously for every two frames in such a blink pattern for two frames, the above third frame becomes the first frame of the next blink pattern and corresponds to the second frame. The frames to be performed are every other frame of the consecutive frames.

  In every other frame, which is the second frame, the blinking pattern does not change in the middle of the frame, so the number of bright bands (dark bands) corresponding to the blinking pattern can be read. In every other frame corresponding to the first frame and the third frame, the flashing pattern changes in the middle unless a plurality of the same flashing patterns are continuous.

Note that even when the timing of switching each blinking pattern of the LED 13 and the timing of switching each frame of the camera 2 are accidentally synchronized, the blinking pattern is switched every two frames, so the blinking pattern is set every other frame. It is possible to read.
In addition, since the LED 13 is turned on (turned off) in each line of each frame, since it is synchronized with the synchronization signal of the camera 2, particularly the Hsync signal, the lighting starts from the middle of the line corresponding to the lighting start and corresponds to the lighting end. Although the light is turned off in the middle of the line, if each lighting time and each light-off time in the blinking of the LED 13 are an integral multiple of the time of one line in the frame, a bright band on the line orthogonal to the line of the frame Alternatively, the width indicated by the number of lines in the dark band is the same even if the horizontal position of the line orthogonal to the line is different.

  It should be noted that the lighting time and extinguishing time of the flashing pattern need not be an integral multiple of one line as long as it is longer than one line, preferably two or more lines. When the LED 13 can be blinked in synchronization with the above signal, the lighting time and the light-off time are preferably an integral multiple of the time for one line.

  As the blinking pattern, for example, the blinking pattern in which the light is turned off 7 times in the predetermined time of 2 frames, the lighted time is 6 times, and the lighted time is turned off 11 times. Flashing pattern of 10 times, flashing time of 15 times, flashing pattern of lighting time of 14 times, lighting time of 19 times, lighting time of 18 times If it represents only by the blink pattern and the frequency | count of lighting time (lighting number) below, it shall have the blink pattern of 22, 26 times, 30 times, 34 times, 38 times, and 42 times.

  For each blinking pattern, for example, 0 for lighting count 6, 1 for lighting count 10, 2 for lighting count 14, 3 for lighting count 18, 4 for lighting count 22, 5 for lighting count 26, 30 lighting count. 6, 7 for the lighting count 34, a start bit for the lighting count 38, and a stop bit for the lighting count 42.

  A combination of these blinking patterns and numerical values is stored in the ROM of the CPU 16 as a data table. The numerical value corresponds to 4-bit information. The combination of two blinking patterns results in 8-bit data and the combination of four blinking patterns results in 16-bit data. Thereby, a digital signal can be transmitted.

  The above blinking pattern is an example, and other combinations of blinking patterns may be used. Here, in order to prevent an error, each flashing pattern is set to have a difference of 4 or more in each lighting number, but the difference may be reduced or increased.

  Further, an ID code (serial number) is stored in the ROM of the CPU 16, and this ID code is read out and converted into a blinking pattern based on the data table described above. For example, the ID code is stored as an 8-bit ASCII code, and each ASCII code is converted into two blinking patterns representing, for example, 4-bit data here by the CPU 16 as the transmission conversion means. Become. Based on the converted blink pattern sequence, the CPU 16 controls lighting and extinguishing of the LED 13 to emit light along the blink pattern sequence.

Note that the ID code may be converted into a blinking pattern first and stored in the ROM in a state of being converted into the above-described blinking pattern.
When the power switch 19 is turned on, the CPU 16 starts its operation, and starts to control the turning on and off of the LED in the blinking pattern corresponding to the above-described ID code. Since the reception side may not be recognized correctly, it is preferable that the ID transmission by the lighting control is set to be performed a plurality of times.

  When the power switch 19 of the lens module 10 is turned on, the CPU 16 controls the light emission of the LED 13 based on the blinking pattern converted from the above ID code based on the program. At this time, the ID code is converted into a plurality of blinking pattern sequences whose order is determined.

The CPU 16 controls lighting and extinguishing of the LED 13 by repeating lighting and extinguishing indicated by each blinking pattern based on the blinking pattern column.
The camera 2 of the smartphone 1 captures the blinking pattern consisting of turning on and off of the LED 13.

Next, the smartphone 1 as a receiving device will be described.
As shown in FIG. 7, the smartphone 1 includes the control device 3 that controls the camera 2 and the camera 2 and that can analyze an image captured by the image sensor of the camera 2.
In the imaging device, based on the above-described synchronization signal, imaging of pixels for one frame is started for each Vsync signal, and imaging of pixels for one line is performed for each Hsync signal. In the CMOS sensor, the charge accumulated corresponding to the amount of light incident on the sensor of each pixel is read for each line. Here, for example, when the amount of light (brightness and darkness) input to the image sensor changes significantly when an image for one frame is captured, the charges accumulated in the sensors of each line differ. As described above, for each Hsync signal, for example, when the charge of the sensor is read line by line from the upper line to the lower line in order, if the bright state and the dark state are repeated, the bright image signal An output line and a line from which a dark video signal is output are generated. As a result, when the image data for one frame taken is viewed, it is possible to capture a striped pattern image in which a bright band portion along the horizontal direction and a dark band portion are repeated along the vertical direction. It becomes possible.

  Therefore, for example, when photographing the skin or a predetermined screen while the LED 13 is blinking in the blinking pattern, a light and dark pattern appears along the vertical direction of the image corresponding to the blinking pattern. . That is, an image in which bright bands and dark bands along the horizontal direction are alternately arranged is captured.

  Here, since the LED 13 is not synchronized with the synchronization signal of the image sensor, it cannot emit light in a blinking pattern every frame, and blinks in the same blinking pattern for a predetermined time of two frames. Therefore, as described later, moving image data having a complete light and dark pattern corresponding to a predetermined blinking pattern is obtained every other frame as will be described later.

The control device 3 can analyze and extract this image data to obtain a light / dark pattern in which high-luminance portions and low-luminance portions for one frame are alternately arranged. From this light / dark pattern, the number of repetitions of light / dark (the number of bright bands and dark bands) is obtained.
The number of repetitions of the light / dark pattern corresponds one-to-one with the blinking pattern of the LED 13 when imaged.

  The flash memory 4 as the light / dark pattern storage means of the smartphone 1 stores a light / dark pattern data table corresponding to the above-described blinking pattern data table as data for optical communication image programming of the skin imaging application. .

  In this light / dark pattern data table, since the flashing pattern is for two frames, the light / dark pattern is read from an image of one frame. The number of repetitions of light (high luminance) and dark (low luminance) is halved.

  Therefore, in the data table stored in the flash memory 4, for example, the number of bright (high luminance) bands is set to 0 for each light / dark pattern as the number of light / dark repetitions (for example, the number of bright bands). 1 in 5 bright bands, 2 in 7 bright bands, 3 in 9 bright bands, 4 in 11 bright bands, 5 in 13 bright bands, 6 in 15 bright bands, bright bands 7 is associated with the start bit, the bright bit number 19 is associated with the start bit, and the bright band number 21 is associated with the stop bit.

  As a result, 4 bits are assigned to each captured image data and can be converted into a digital signal, and the control device 3 as the image analysis unit and the reception conversion unit reads out the digital signal from the image data. This digital signal is an ID code, and the ID code of the lens module 10 is read out.

  The smartphone 1 can transmit an ID code together with a photographed skin image to a server that analyzes the skin image via the Internet. On the server side, the ID code identifies whether the lens module 10 is a genuine product, whether the lens module 10 is for a campaign (collaboration) with a specific cosmetic company (manufacturer or sales company), or a normal one. It becomes possible.

An optical communication method in the skin imaging system as such an optical communication device will be described with reference to FIG.
When the skin imaging application is activated on the smartphone 1, imaging of a moving image including the image 34 for each frame is started.
At this time, it is necessary to turn on the power switch 19 of the lens module 10.

  Therefore, for example, when the skin image capturing application is activated on the smartphone 1 side, a message stating “Please turn on the power switch 19 of the lens module 10” from the smartphone 1 before the start of moving image capturing. It will be done by display.

  In this case, it is preferable that the lens housing 20 is not affected by external light. For example, the state in which the tip of the lens housing 20 is pressed against the skin 24, for example, a dedicated screen, or the state in which a dedicated lid is placed on the tip. It is preferable that Therefore, it is preferable that the smartphone outputs, for example, a message for instructing the tip of the lens housing 20 to come into contact with the skin imaging portion after the message for turning on the power switch 19 described above.

  The camera 2 starts imaging after the power-on instruction of the lens module 10 described above, and in this state, the LED 13 of the lens module 10 is based on the blinking pattern converted as described above every two frames. It will turn off and on repeatedly.

  When the power switch 19 of the lens module 10 is turned on in this way, the skin 24 is imaged by the imaging element of the camera 2 under the situation where the LED 13 blinks according to the blinking pattern.

  Here, as described above, when the LED 13 blinks in a blinking pattern for two frames in a state asynchronous to the synchronization signal of the camera 2, the light / dark pattern that is a blinking pattern captured by the camera 2 is In the middle of the frame taken by the camera 2, the previous flashing pattern ends and the next flashing pattern is started. Therefore, in this frame, two blinking patterns are mixed.

  In the next frame, an image corresponding to one blinking pattern is taken. Furthermore, in the next frame pattern, one blinking pattern ends halfway, and the next blinking pattern is started, so that two blinking patterns are mixed. Since such a situation is repeated until imaging with all the blinking patterns is completed, frames having a light and dark pattern corresponding to one blinking pattern are generated every other frame.

  The control device 3 of the smartphone 1 as a receiving device first recognizes the blinking pattern associated with the start bit. Therefore, for example, the image data of every other frame from the frame in which the start bit is recognized may be analyzed, or the frames corresponding to one blinking pattern may be analyzed by sequentially analyzing all the frames. It may be found sequentially.

  After the controller 3 recognizes the image 31 having the repeated number of bright and dark as the start bit, the image 34 of every other frame has a light and dark pattern converted into an electronic signal (data). A plurality of images 32a, images 32b, images 32c,... Are included. Further, in the lens module 10 as a transmission device, the blinking pattern serving as the above-described start bit is transmitted at the start of data transmission by blinking of the LED 13, and the stop bit is transmitted at the end of the data transmission.

  Therefore, in the control device 3, when the image 34 for each frame of the moving image is analyzed and the image 33 having the light / dark pattern as a stop bit is recognized, the shooting of the moving image by the camera 2 is ended.

Next, as a skin imaging system, for example, with the LED 13 turned on, the camera 2 of the smartphone 1 captures a still image 35 for analyzing skin texture.
Next, the CPU 16 of the lens module 10 turns on the LED 12 to capture a still image 36 for analyzing skin spots. Then, the captured still image data is uploaded to a company server that analyzes the skin.

  In such a skin imaging system as an optical communication device, data can be transmitted from the lens module 10 as a transmission device to the imaging element of the camera 2 which is a digital camera. Accordingly, wireless data communication with a portable electronic device having a camera 2 such as a smartphone 1 with a device including a solid light emitting element such as an LED and a microcomputer that controls turning on and off of the solid light emitting element. Is possible.

  Therefore, the ID code of the conversion lens 11 (lens module 10) as described above can be transmitted to the smartphone 1 without a general communication device.

  In addition, since it is possible to receive data by capturing moving images with a built-in camera such as a smartphone 1 or a tablet, the camera captures the location where the influence of external light is low and where the transmission device is installed. Then, it is possible to construct a system in which data is input.

Next, a second embodiment of the present invention will be described.
As shown in FIG. 9, in the skin imaging system of the second embodiment, the smartphone 5 is provided with a flash 5 for the camera 2 made of LEDs. The flash 5 is provided in a general smartphone 1 and is turned on during normal shooting with the camera 2, but the lighting timing can be controlled by the control device 3. Yes.

  Further, the lens module 10 is provided with a PD (photodiode) 5 facing the LED which is the flash 5 of the smartphone 1, detects the light emission of the flash 5 as a synchronous output means, and outputs a detection signal to the CPU 16. It is like that.

  The flash 5 serves as a synchronization output means to the control device 3 so that a vertical synchronization signal as a synchronization signal of a video signal output from the camera 2 and a synchronization signal indicating the output timing of the horizontal synchronization signal are output by light. Be controlled. That is, when the camera 2 starts shooting a moving image, light is emitted in accordance with the timing of the output synchronization signal. This light emission is the timing of turning on and off the LED 13 controlled by the CPU 16 as a light emission control means in the lens module 10.

  Note that the difference between the vertical synchronization signal and the horizontal synchronization signal is, for example, that the light emission of a pulse is such that a pattern with a short light emission time and a long pattern, a pattern with a low light emission luminance, a high pattern, and a flash 5 are pulsed. Distinguish by the difference of patterns.

  In the CPU 16, for example, when the skin imaging application is activated and the flash 5 is turned on by the skin imaging application, this is detected by the PD 25, and a detection signal is input to the CPU. When the detected light emission of the flash 5 is a vertical synchronizing signal, lighting and extinguishing in a blinking pattern for one frame is started and switching between lighting and extinguishing is performed at the timing of the horizontal synchronizing signal.

  In this case, for example, when lighting and extinguishing for 100 lines are alternately performed, the lighting of the LED 13 is started by the input of light emission of the flash 5 as a synchronizing signal corresponding to the horizontal synchronizing signal. Then, the LED 13 is turned off when a synchronization signal corresponding to the 100th horizontal synchronization signal is input. Further, it can be repeatedly turned on when the 100th horizontal synchronization signal is input. In addition, when a vertical synchronization signal is input, the LED 13 is controlled to be turned on and off in the next flashing pattern in the stored flashing pattern column.

This makes it possible to transmit / receive data of one blinking pattern for each frame, not every other frame.
In addition, since it is possible to control with high accuracy, it is possible to accurately recognize the width of the bright band and the dark band in the light and dark pattern. 1 bit data is assigned to each band by assigning 0 and assigning 1 to wide and bright bands that are at least twice as wide as the narrow band and alternating light and dark. Can be assigned.

In this case, 0 is assigned to lighting and extinguishing with a narrow time width on the blinking pattern side, and 1 is assigned to lighting and extinguishing with a time width that is twice or more of the time width when narrow.
Also in this case, the time width of each turn-on / off is defined by the number of lines in one frame by being synchronized with the vertical synchronizing signal and the horizontal synchronizing signal as described above.

Here, when there is a delay in the synchronization signal, that is, when the synchronization signal is received and the LED 12 is turned on or off, the synchronization signal is delayed with respect to the imaging timing of one line. May be advanced by a delay time. For example, even if the camera 2 starts shooting a moving image on the smartphone 1 side and a synchronization signal is output, and the timing of the synchronization signal is obtained, the flash 5 as a synchronization signal may be emitted earlier than the actual synchronization signal. Good.
According to such an optical communication device, communication accuracy can be improved and communication speed can be increased.

In optical communication using light emission of the LED 13 and imaging by the imaging device of the camera 2, an experiment for reading digital data from the captured image was performed.
In this experiment, a still image, not a moving image, was shot in a state where the LED 13 was repeatedly turned on and off in the same time width, that is, in a state where the LED 13 was blinking with a predetermined blinking pattern and no external light was entered. An experiment was conducted as to whether or not a light / dark pattern corresponding to a blinking pattern could be read from the captured image data.

  In FIG. 10, reference numeral 41 indicates a captured image, reference numeral 42 indicates a graph of luminance data along a line in the Y direction as the vertical direction of the image 41, and reference numeral 43 indicates a graph. 42 shows a graph of luminance data obtained by shading correction of the luminance data shown in FIG.

The image 41 is an image of one frame taken by the smartphone camera, and 5 to 6 stripe patterns can be confirmed. However, the image 41 is clearly shown by the high brightness of the central portion and the integration action of the C-MOS sensor described later. It is not a striped pattern.
The graph 42 shows the luminance change of the line portion along the Y direction substantially in the center of the X direction of the image 41, the vertical axis shows the position along the Y direction on the image 41, and the horizontal axis shows the luminance. Is shown.

Here, due to the integral action of the C-MOS sensor, the contrast between the light when the LED 13 is turned on and the dark when the LED is turned off as shown in the image 41 and the graph 42 is low.
In addition, since the peripheral image around the central portion of the image data is not used in capturing the skin image, the peripheral portion of the image 41 is darkened by the LED 13, as shown in the graph 42. The luminance at the center of the image 41 is high, and the luminance at the top and bottom is low.

  Such luminance deviation can be corrected by shading correction that corrects the luminance distribution of the entire image to be uniform. The graph 43 is obtained by shading correction of the luminance shown by the graph 42, and the difference (contrast) between the luminance of the bright portion and the luminance of the dark portion becomes clear, and the boundary between the high luminance portion and the low luminance portion is clear. Thus, the number of repetitions of light and dark can be easily recognized.

  Next, an experiment was performed in which the correction was made not by shading correction but by the difference from the background. In FIG. 11, an image 44 is an LED modulation image captured by blinking the LED 13, and an image 45 is a background image captured while the LED 13 is turned off, for example. In this example, a still image is captured instead of a still image, and the frame rate is set to 15 fps. Here, an image is shown in a state where the field of view of the camera 2 is limited to a square by the lens housing 20 described above. The entire images 44 and 45 correspond to the imaging range of the image sensor.

The central square bright portion in this range is the skin portion photographed in a state where the field of view is restricted by the lens housing 20. That is, an opening 20C is provided on the front end side of the lens housing 20, and the skin facing the opening 20c is photographed.
Here, in this example, the entire imaging range of the image sensor has 3096 lines (pixels) in the vertical direction, but the part of the skin that is actually photographed is 1600 lines (pixels) in the vertical direction. Note that the number of pixels in the horizontal direction of the skin portion image is also set to 1600 pixels.

  Taking the difference between the LED modulation image 44 captured with the LED 13 blinking and a background image 45 captured with the LED turned off in a predetermined blinking pattern (the number of repetitions of predetermined turning on and off) 46. The luminance along the line in the Y direction of the image 46 is a graph 46a. In the graph 46a, the upper polygonal line (wave) indicates the change in luminance in the vertical direction of the image 46 with the difference, and the lower side indicates the wave of the fundamental frequency among the waves of the change in luminance. . On the horizontal axis that is the time axis in the graph 46a, the unit is the time for one line of the image sensor, that is, the time that is the interval between the horizontal synchronization signals, and the time is indicated by the number of lines.

In this example, when the on / off blinking pattern of the LED 13 is set to a wavelength of a set time duration of turning on and off in one second, the frequency is set to 590 Hz. That is, it is a state in which lighting and extinguishing are repeated 590 times per second.
On the other hand, since the line of one image captured by the image sensor is 3096 and the frame rate of the camera 2 is 15 fps, the number of lines from which charges are read per second is 3096 × 15.

  Here, when the LED modulation frequency is 590 Hz and the number of lines from which charges are read per second is 3096 × 15, the wavelength that is the sum of one high luminance time width and one low luminance time width is , Approximately 79 lines.

  A graph 46b shows a result of performing a fast Fourier transform (FFT) as a frequency analysis on the wave as the Y direction luminance signal change shown in the graph 46a. In this graph 46b, the vertical axis represents luminance, and the horizontal axis represents frequency (the reciprocal of the wavelength indicated by the number of lines). In the graph 46b, the upper side is the result of frequency analysis performed on the luminance wave read (corrected) from the actual image 46, and the lower side is the result of frequency analysis performed on the above-described fundamental wave.

  In the graph 46b, the reciprocal of the frequency of the wave consisting of the change in luminance read from the image 46 is the wavelength, but the frequency (frequency of the fundamental wave) obtained by frequency analysis is 0.0127 (about 1/79). It was. The reciprocal of this frequency indicates the wavelength and is indicated by the number of lines, which is 79. This corresponds to the wavelength that is the sum of the time width of the wave lighting and the time width of the light extinction formed by repeating the lighting and extinguishing of the LED 13 obtained by calculation as described above.

  Therefore, the wavelength (frequency) of the wave consisting of repetition of turning on and off in one frame of the LED 12 can be read from the image data picked up by the image pickup device of the camera 2.

  Therefore, data can be transmitted and received by a change in the number of repetitions of turning on and off the LED 13 within a predetermined time, that is, a change in the frequency of the wave consisting of the turning on and off of the LED 13. In other words, data communication is possible by frequency shift keying (FSK).

  Next, FIG. 12 shows experimental results when the LED 13 is actually turned on and off at different frequencies.

The experimental method is the same as in the case of FIG. 11, and the frame rate and the vertical resolution of the image sensor (the image captured by the image sensor) are also the same as the values shown in FIG. 11, except that the frequency of the LED modulation is different. It has become.
Here, the case where the wavelength of the repeated wave of turning on and off of the LED 13 is 150 lines is considered, and the case where the same wavelength is 30 lines. The upper image 47 is captured when the number of lines is 150, and the lower image 48 is captured when the number of lines is 30. The result of correcting the luminance change along the Y-direction line of the image 47 by the difference from the above-described background image is shown in a graph 49, and the luminance change along the Y-direction line of the image 48 is shown as the above-described background image. The graph 50 shows the result of correction based on the difference between.

A result of frequency analysis of the wave shown in the graph 49 by FFT is shown in a graph 51, and a result of frequency analysis of the wave shown in the graph 50 by FFT is shown in a fluff 52.
As shown in the graph 49 in the case where the wavelength is 150 and the graph 50 in the case where the wavelength is 30, the luminance change after the correction can sufficiently confirm the high luminance portion and the low luminance portion, and The graph 49 and the graph 50 can sufficiently confirm the difference in wavelength, that is, the difference in frequency.

  Further, as a result of frequency analysis, the analyzed frequency when the wavelength of the wave composed of repetition of turning on and off of the LED 13 is 150 is the reciprocal of the wavelength. It is shown that the frequency can be read from the image 47 photographed in FIG. In addition, in the graph 51, in addition to the obtained fundamental wave frequency, relatively large harmonics that are considered to be caused by analysis by FFT are recognized.

  In addition, the analyzed frequency in the case of 30 wavelengths is the reciprocal of the wavelength of repeated lighting and extinguishing of the LED 13, and from the image 48 obtained by imaging the repeated wave of lighting and extinguishing of the LED 13 with the image sensor. The frequency is shown to be readable. In the graph 52, the FFT analysis was performed by setting the FFT so as not to overlap, including frequencies and harmonics other than the fundamental wave.

  Thereby, it is possible to read the wave frequency (the number of repetitions per unit time) due to repetition of lighting and extinguishing of the LED 13 from an image captured under the illumination of the LED 13, and by frequency shift modulation between the LED and the image sensor. It was shown that data communication by light is possible.

Next, a third embodiment of the present invention will be described.
In the third embodiment, identification information is provided by an optically readable code (code readable by image recognition) such as a so-called barcode within the photographing range of the camera 2 around the opening 20c of the lens housing 20. It is a thing. Here, this code is referred to as a bar code, but the bar code is not limited to a general bar code, and may be various bar codes (including a two-dimensional bar code) or optically read. If it is a possible code, it may not be a so-called bar code.

  In this embodiment, a barcode portion 20d is provided around the opening 20c of the contact portion 20b of the lens housing 20, and the back surface of the barcode portion 20d (the surface facing the conversion lens 11 and the camera 2). A bar code 20e is provided. The barcode 20e may be affixed as a sticker or printed on the barcode portion 20d, for example. Further, the barcode portion 20 d may be formed integrally with the lens housing 20 or may be retrofitted to the lens housing 20.

  As shown in FIG. 14, the barcode 20e as the identification information is, for example, around the skin image 63 photographed in a rectangular shape (square shape) corresponding to the opening 20c in the image 61 photographed skin. The barcode image 20f (image obtained by photographing the barcode 20e) is displayed in a rectangular frame shape. That is, the barcode image 20f is present around the skin image 63 of the image 61 obtained by photographing the skin. One unit of barcode is displayed on one side of the rectangle, the same one is provided on four sides, and the reading side recognizes the clearest one.

  Thereby, the identification information can be read from the barcode image 20f of the image 61 obtained by photographing the skin. The identification information may be read by the smartphone 1 or may be read by the server side to which the skin image 63 is sent.

In FIG. 14, the circular portion indicates the shooting range 62 of the conversion lens 11, but the portion that protrudes from the image 61 is a portion that is not imaged.
Further, the position of the barcode 20e is preferably as close as possible to the skin exposed from the opening 20c in the optical axis direction of the conversion lens 11 and the direction orthogonal to the optical axis.

  According to such a configuration, in the photographed image 61, the barcode image 20f is included around the skin image 63, and the photographed image 61 includes the identification information by the barcode 20e and the skin image 63. Thus, it is possible to specify the conversion lens 11 (lens module 10) used for photographing the skin image 63.

Further, the barcode 20e is made of, for example, black and white, and the barcode may be used for color correction such as white balance.
Further, in the third embodiment, the barcode 20e is disposed on the portion of the lens housing 20 that is imaged with the skin, but a cap is detachably provided on the distal end side (contact portion 20b) of the lens housing 20. Therefore, optically readable codes such as various barcodes may be provided on the back surface of the cap (the surface facing the conversion lens 11 when worn).

  At this time, the barcode may be affixed to the back surface of the cap as a seal, or may be printed on the back surface of the cap. Moreover, since a barcode can be provided in the rectangular part corresponding to the opening part 20c, various two-dimensional barcodes can be used suitably, for example.

  In the above-described embodiment, information is transmitted using a blinking pattern using the LED 13 for photographing illumination. However, an LED for transmitting information using a blinking pattern may be provided separately from the illumination LED 13.

  In the above-described embodiment, in each type of flashing pattern, the time width of the high-luminance portion is substantially the same, the time width of the low-luminance portion is the same, and the time width of the high-luminance portion is the same. And the time width of the low-luminance portion are the same, but they may be different.

  For example, the time width of a portion with high luminance is set to a plurality of types such as a reference width, a double width of the reference width, and a triple width of the reference width. A plurality of types such as a double width and a triple width of the reference width may be used. In this case, for example, 1-bit data can be allocated to each bar as in the case of a bar code including a narrow white and black bar and a white and black bar that is twice or more wide. Further, by increasing the types of widths, for example, data such as 2 bits, 3 bits, and 4 bits can be allocated to one bar (band).

1 Smartphone (portable electronic device, receiver)
2 Camera (image sensor)
3 Control device (control means, image analysis means, reception conversion means)
4 Flash memory (light / dark pattern storage means)
5 Flash (synchronous output means)
10 Lens module (transmitter)
11 Conversion lens 12 LED (solid-state light emitting element, LED for photographing illumination)
13 LED (Solid-state light emitting device, LED for photographing illumination)
16 CPU (light emission control means, flashing pattern storage means, transmission conversion means)
25 PD (synchronous input means)

Claims (9)

An image sensor that captures a moving image at a set frame rate, and a receiving device that includes image analysis means that analyzes image data of each frame of the moving image captured by the image sensor;
A flashable solid state light emitting device, and lighting and extinguishing at intervals of a time equal to or longer than one line at the frame rate every predetermined time when one or several frames of the solid state light emitting device are imaged at the frame rate A transmission device having a light emission control means for turning on and off with a plurality of blink patterns consisting of:
The light emission control means of the transmitting device relates to the relationship between the blinking pattern storage means storing each blinking pattern and each value of the digital signal in association with each other, and the blinking pattern stored in the blinking pattern storage means and the value. And a transmission conversion means for sequentially converting each value of the digital signal into the blinking pattern of the solid state light emitting device,
And the light emission control means blinks the solid state light emitting element based on the blinking pattern converted by the transmission conversion means,
The image analysis means of the receiving device, as an analysis result of the image data, is a light / dark pattern consisting of high and low brightness appearing along the vertical direction of the imaged image data due to blinking of the blinking pattern of the solid state light emitting element. Seeking
The receiving device includes: a light / dark pattern storage means for storing each light / dark pattern and each value of the digital signal in association with each other based on a correlation between the light / dark pattern and the blinking pattern obtained by the image analyzing means; Receiving conversion means for converting the light / dark pattern sequentially analyzed by the image analysis means into each value of a digital signal based on the relation between the light / dark pattern stored in the pattern storage means and the value; Optical communication device. The light emission control means of the transmitter controls the turning on and off of the solid state light emitting element asynchronously with the synchronizing signal of the video signal output by the imaging element,
Each blinking pattern stored in the blinking pattern storage means of the transmitter is defined by the number of repetitions of turning on and off included in the predetermined time,
Each light and dark pattern stored in the light and dark pattern storage means of the receiver is the number of bright and / or dark bands along the horizontal direction represented by the level of brightness appearing in the vertical direction of the imaged image data. The optical communication device according to claim 1, wherein the optical communication device is defined by: The receiving apparatus includes a synchronization output unit that outputs a synchronization signal in synchronization with a synchronization signal of a video signal output from the imaging device,
The transmission device includes synchronization input means for receiving a synchronization signal transmitted from the synchronization output means,
The light emission control unit of the transmission device turns on and off the solid state light emitting device in synchronization with each frame and each line of each frame imaged by the imaging device by a synchronization signal input to the synchronization input unit. Control
Each blinking pattern stored in the blinking pattern storage means of the transmitting device is defined by a lighting time width and a lighting time width ,
Each light and dark pattern stored in the light and dark pattern storage means of the receiving device has a width of a bright band and a width of a dark band along the horizontal direction represented by high and low brightness appearing in the vertical direction of the captured image data. The optical communication apparatus according to claim 1, wherein the optical communication apparatus is defined by: The solid-state light emitting element of the transmitter is a three-color LED that can be turned on and off for each color,
The light emission control means of the transmission device transmits a digital signal by a blinking pattern consisting of independent lighting and extinction for each color of the three-color LED,
Each blinking pattern stored in the blinking pattern storage means of the transmitting device is defined by turning on and off each color of the three-color LED,
The receiver has a color image sensor,
The image analysis means of the receiving device obtains a light and dark pattern that appears in the vertical direction corresponding to each color of the three-color LED of the imaged image data,
Each light and dark pattern stored in the light and dark pattern storage means of the receiving device appears in the vertical direction of the imaged image data and is represented by the luminance for each color corresponding to the emission color of the three-color LED. The optical communication apparatus according to any one of claims 1 to 3, wherein the optical communication apparatus is defined by a bright band and a dark band along a horizontal direction. A receiving device having an image sensor that captures a moving image at a set frame rate, and an image analysis unit that analyzes image data captured by the image sensor;
A flashable solid state light emitting device, and lighting and extinguishing at intervals of a time equal to or longer than one line at the frame rate every predetermined time when one or several frames of the solid state light emitting device are imaged at the frame rate An optical communication method performed in an optical communication device comprising a transmission device having a light emission control means for turning on and off with a plurality of blink patterns consisting of:
The light emission control means of the transmission device sequentially converts the digital signal into the blinking pattern based on the relationship between each blinking pattern and each value of the digital signal, and based on the blinking pattern, Control on and off,
The imaging device of the receiving device captures a moving image by the imaging device in a state where the solid state light emitting device is turned on or off in the blink pattern,
Analyzing the image data of the captured moving image to obtain a light and dark pattern consisting of high and low brightness appearing along the vertical direction of the image data,
The receiving device obtains the light / dark pattern sequentially obtained based on the correlation between the light / dark pattern obtained and the blinking pattern based on the correlation between the light / dark pattern and each value of the digital signal. An optical communication method characterized by converting into a value. The light emission control means of the transmitter controls the turning on and off of the solid state light emitting element asynchronously with the synchronizing signal of the video signal output by the imaging element,
The transmission device defines each blinking pattern by the number of repetitions of turning on and off included in the predetermined time,
The receiving apparatus defines each light / dark pattern by the number of bright bands and / or dark bands along the horizontal direction represented by the level of luminance appearing in the vertical direction of the captured image data. The optical communication method according to claim 5. The receiving device outputs a synchronizing signal in synchronization with a synchronizing signal of a video signal output from the imaging device,
The transmitter receives a synchronization signal transmitted from the receiver;
The light emission control means of the transmission device controls lighting and extinguishing of the solid state light emitting element in synchronization with each frame imaged by the imaging element and each line of each frame by the received synchronization signal,
The transmission device defines each blinking pattern by a lighting time width and a lighting time width ,
The receiving device defines each light / dark pattern by a width of a bright band and a width of a dark band along a horizontal direction represented by a level of brightness appearing in a vertical direction of the captured image data. The optical communication method according to claim 5. The solid-state light emitting element of the transmitter is a three-color LED that can be turned on and off for each color,
The light emission control means of the transmission device transmits a digital signal by a blinking pattern consisting of independent lighting and extinction for each color of the three-color LED,
The transmitter defines each blinking pattern by turning on and off each color of the three-color LED,
The receiver has a color image sensor,
The image analysis means of the receiving device obtains a light and dark pattern that appears in a vertical direction corresponding to each color of the three-color LED of the imaged image data,
The light receiving device appears in the vertical direction of the image data obtained by capturing each light and dark pattern, and is bright and dark along the horizontal direction represented by the luminance for each color corresponding to the emission color of the three-color LED. The optical communication method according to claim 5, wherein the optical communication method is defined by a band. A skin imaging system comprising the optical communication device according to any one of claims 1 to 4,
An image analysis unit that includes the digital camera having the image sensor and controls the image sensor and analyzes image data of each frame of the moving image data captured by the image sensor, and is obtained by the image analysis unit Based on the correlation between the light / dark pattern and the blinking pattern, the light / dark pattern storage means for storing each light / dark pattern and each value of the digital signal in association with each other; the light / dark pattern stored in the light / dark pattern storage means; A portable electronic device comprising control means functioning as reception conversion means for converting the light and dark pattern sequentially analyzed by the image analysis means into each value of the digital signal based on the relationship with the value;
The LED module for photographing illumination as the solid state light emitting element, a conversion lens used for photographing a skin image, and a lens module comprising the light emission control means for controlling light emission of the LED for lighting,
The light emission control means stores the blinking pattern and each value of the digital signal in association with each other, and the flashing pattern storage means stores the blinking pattern and the value stored in the blinking pattern storage means. A skin imaging system, which functions as transmission conversion means for sequentially converting each value of a signal into the blinking pattern of the solid state light emitting device.

Images