CN115802236B

CN115802236B - Method for shortening delay of earphone with auxiliary hearing

Info

Publication number: CN115802236B
Application number: CN202310006175.2A
Authority: CN
Inventors: 李波
Original assignee: Chongqing Ambi Technology Co ltd; Chengdu Anbi Technology Co ltd
Current assignee: Chongqing Ambi Technology Co ltd; Chengdu Anbi Technology Co ltd
Priority date: 2023-01-04
Filing date: 2023-01-04
Publication date: 2023-04-14
Anticipated expiration: 2043-01-04
Also published as: CN115802236A

Abstract

The invention discloses a method for shortening the delay of an earphone with auxiliary hearing, which relates to the technical field of Bluetooth earphones, and comprises an analog microphone, an analog amplifier, an analog-to-digital converter, a down-sampling converter, a first asynchronous sampling converter, a DMA1, a memory and an MCU/DSP which are sequentially connected, as well as a playing device, an amplitude power amplifier, a digital-to-analog converter, an up-sampling converter, a second asynchronous sampling converter, a DMA2 and the MCU/DSP which are sequentially connected, wherein the DMA1 is started, data is moved to a designated initial address of the memory from the FIFO of the first asynchronous sampling converter, and the DMA2 is started again after the MCU/DSP obtains, processes and moves the data to a designated destination address of the memory, and the data is moved to the FIFO of the second asynchronous sampling converter from the designated destination address. The invention shortens the delay of the traditional software for realizing the auxiliary listening method, has simple realization method and does not increase any hardware cost.

Description

Method for shortening delay of earphone with auxiliary hearing

Technical Field

The invention relates to the technical field of Bluetooth earphones, in particular to a method for shortening delay of an earphone with auxiliary hearing.

Background

The larger and more earphones with auxiliary hearing are on the market at present, the implementation methods are also very many, but generally speaking, the method can be divided into hardware implementation and software implementation. However, regardless of the method, there is a very important index to determine the performance of an auxiliary hearing earphone, and this index is the delay. The delay refers to the time required by the whole process that a sound receiving device (generally referred to as a microphone) with an auxiliary hearing earphone receives a sound signal and then the sound signal passes through a signal path, and the received signal is played through a sound generating device (generally referred to as a loudspeaker) of the earphone after signal processing. It is generally required that this delay be less than 10ms, the shorter the delay the better. As shown in fig. 1, the signal processing flow of the auxiliary listening device is radio reception device- > input signal path- > signal processing- > output signal path- > sound generation device, the upper module of the hardware-implemented auxiliary listening device is implemented by hardware, software does not participate in any process, the software-implemented auxiliary listening device is generally different from the hardware-implemented auxiliary listening device in the input signal path and the output signal path, the signal processing part is implemented by software, and the software runs on the MCU or the DSP. The auxiliary listening realized by hardware has the advantages of very low delay, and the defects of not enough flexibility and incapability of changing the algorithm in the signal processing module at the later stage. The auxiliary listening realized by software has the advantages of flexible algorithm, capability of changing the signal processing algorithm at any time according to the requirement in the later period so as to meet the requirements of different users, and the disadvantage that the delay is generally higher than that of hardware realization. As shown in fig. 2, in the method for implementing auxiliary listening by software in the prior art, an analog microphone converts a sound signal into an electrical signal, and for a sound receiving device of an auxiliary listening earphone, the collected sound is amplified by an analog amplifier, converted into an analog-to-digital signal, and reduced by an integral multiple of a sampling rate and converted into a sampling frequency to be output, and the sampling frequency is consistent with that of a microprocessor unit/digital signal processor MCU/DSP, and a signal output by a first asynchronous sampling converter is DMA-transmitted to a designated memory location by a direct memory access controller, so that the MCU/DSP can conveniently acquire a digital signal to be processed at a designated memory address. Similarly, the DMA can also move the digital signal processed by the MCU/DSP to the input of the second asynchronous sampling converter, and convert the electrical signal into a sound signal after performing sampling frequency conversion of the second asynchronous sampling converter and integer-multiple lifting of the sampling rate, digital-to-analog conversion, amplitude and power amplification by the up-sampling converter, and play the sound signal by the sound device of the auxiliary listening headphone.

The sound collection process is as shown in fig. 3, the MCU/DSP sets a source address for the DMA of the collection channel, the source address is the output FIFO address of the first asynchronous sampling converter, and also sets two destination addresses, the two addresses are the start address of the Ping buffer1 and the start address of the Ping buffer1, the size of the Ping buffer1 is the same as that of the Ping buffer1, the two buffers are located in the memory, and also set the size of the data size to be moved at one time, which is called a frame data fragment size1, ping buffer1, and Ping buffer 1. When the DMA of the acquisition channel is started, the DMA firstly moves the FIFO data of the first asynchronous sampling converter output FIFO queue to Ping buffer1, when the data of fragment size1 is moved, the DMA generates an interrupt to the MCU/DSP and informs the moved Memory address signal, and the MCU/DSP receives the interrupt and then removes the corresponding Memory address to fetch the data for processing. Meanwhile, the DMA starts to move the output FIFO data of the first asynchronous sampling converter to the Pang buffer1, and similarly, after the data of one fragment size1 is moved, the DMA can generate an interrupt to the MCU/DSP and tell the moved memory address signal. Then moving the data to Ping buffer1, and then Pang buffer1, thus repeatedly circulating for 8230, 8230

The process of playing the sound signal is shown in fig. 4, the MCU/DSP sets two source addresses to the DMA of the playing path of the playback device, such as the speaker, where the two source addresses are the start address of the Ping buffer2 and the start address of the Ping buffer2, and similarly, the size of the Ping buffer2 is the same as that of the Ping buffer2, and the two buffers are also located in the memory, and also set the size of the data size to be moved once, which is called fragment size2, fragment size2 is the same as that of the Ping buffer and the Ping buffer 2. The size of Fragment size2 may or may not be the same as the size of Fragment size 1. There is usually no particular reason to set the two sizes to be the same. A destination address is also set, which is the input FIFO address of the second asynchronous sample converter. When the DMA is started, the DMA firstly moves the Ping buffer2 data to the input FIFO of the second asynchronous sampling converter, when the fragment size2 data is moved, the DMA generates an interrupt to the MCU/DSP to inform that the Ping buffer2 data is moved completely, and starts to move the Pang buffer2 data, the MCU/DSP receives the interrupt and then takes out the collected audio data with the fragment size2 size from the memory to send to the Ping buffer2, and similarly, when the fragment is moved completely, the DMA generates an interrupt to the MCU/DSP to inform that the Pang buffer2 data is moved completely, and then moves the Ping buffer2 data. After receiving the interrupt, the MCU/DSP removes the MIC data with the size of fragment size2 from the memory and sends the MIC data to the Pang buffer2, thus repeatedly circulating the processes of 82308230, 8230

The steps of implementing auxiliary listening by software are as follows:

1. firstly, configuring parameters of each hardware module;

2. simultaneously starting a first direct memory access controller DMA1 and a second direct memory access controller DMA2;

3. DMA1 and DMA2 move one fragment size (fragment size = fragment size1= fragment size2, and normally one fragment size is called one frame data) and generate an interrupt

4. The MCU/DSP processes the respective transactions in the interrupt function processing of DMA1, DMA 2. The DMA1 interrupt processing function processes the received signal, and the DMA2 interrupt processing is mainly used for sending the processed data to Ping buffer2 or Pang buffer2;

5. and repeating the steps 3 and 4.

Delay analysis of the prior art:

similarly, after the hardware is fixed, all hardware processing modules, here, the analog microphone, the analog amplifier, the analog-to-digital converter, the down-sampling converter, the first asynchronous sampling converter, the second asynchronous sampling converter, the up-sampling converter, the digital-to-analog converter, the amplitude power amplifier, and the speaker generate fixed delay, generally in the order of microseconds (us), which is about tens of microseconds to one or two hundred microseconds, and the delay is mainly generated between data movement of the DMA. As shown in fig. 5 below, it can be seen that the collected first frame data is played only after the third frame, and the first two frames played by the speaker are invalid null data, which is usually 0, so:

total latency = hardware latency (fixed) +2 × fragment size;

the prior art methods of reducing the delay are generally to minimize the size of the fragment size. For example, if the sampling rate is 16K and the fragment size is 240 samples, the delay for that fragment size is 15ms.

Disclosure of Invention

The invention aims to provide a method for shortening the delay of an earphone with auxiliary hearing, which further shortens the delay time of realizing auxiliary hearing by software in the prior art.

The invention solves the problems through the following technical scheme:

a method for shortening the delay of an earphone with auxiliary hearing comprises an analog microphone, an analog amplifier, an analog-to-digital converter, a down-sampling converter, a first asynchronous sampling converter, a first direct memory access controller DMA1, a memory and a microprocessor unit/digital signal processor MCU/DSP which are sequentially in communication connection, and also comprises a playing device, an amplitude power amplifier, a digital-to-analog converter, an up-sampling converter, a second asynchronous sampling converter, a second direct memory access controller DMA2, the memory and the microprocessor unit/digital signal processor MCU/DSP which are sequentially in communication connection, wherein the audio processing method comprises the following steps:

firstly, a first direct memory access controller DMA1 is started, the DMA1 transfers data from an output first-in first-out queue FIFO of a first asynchronous sampling converter to a designated initial address of a memory, when a microprocessor unit/digital signal processor MCU/DSP acquires and processes a frame of data from the designated initial address of the memory and transfers the data to a designated destination address of the memory, a second direct memory access controller DMA2 is started, and the DMA2 transfers data from the designated destination address of the memory to an input first-in first-out queue of a second asynchronous sampling converter.

Because the first direct memory access controller DMA1 is started firstly, when one DMA1 is interrupted, the MCU/DSP acquires first frame data, and then the MCU/DSP performs signal processing for delta; after the signal processing is finished, copying the processed frame data to Ping buffer2, then starting a second direct memory access controller DMA2, and starting the DMA2 to start to carry the Ping buffer2 data; when the second DMA1 is interrupted, the MCU/DSP acquires the second frame of data, and after processing, sends the second frame of data to Ping buffer2 (since DMA2 is started one frame + Δ later than DMA1, so the analog microphone is still carrying Ping buffer2 data, DMA2 is also carrying Ping buffer2 data. The total delay of the method = hardware delay (fixed) + fragment size + Δ, where the fragment size carries the time of one frame of data, the time of Δ depends on the algorithm complexity, MCU/DSP operational performance and operational dominant frequency.

The audio processing method specifically comprises the following steps:

when an audio signal acquired by an analog microphone is amplified by an analog amplifier, converted by an analog-to-digital converter, converted by a down-sampling converter at integral multiple of sampling rate and converted into a signal with a set sampling frequency by a first asynchronous sampling converter, starting a first direct memory access controller DMA1, alternately transferring data from an output first-in first-out queue FIFO of the first asynchronous sampling converter to a buffer area Ping buffer1 and a buffer area Pang buffer1 in a memory after the DMA1 is started, and generating an interrupt signal to a microprocessor unit/digital signal processor MCU/DSP after the DMA1 is transferred each time; after receiving the first interrupt signal, the MCU/DSP reads data from the buffer Ping buffer1, processes the data and then moves the data to a buffer Ping buffer2 in the memory, and after receiving the interrupt signal again, the MCU/DSP reads data from the buffer Pang buffer1, processes the data and then moves the data to the buffer Pang buffer2 in the memory, and the data is moved alternately in this way; after the MCU/DSP finishes moving the first frame data, starting a second direct memory access controller DMA2, and alternately moving the data of the buffer area Ping buffer2 and the buffer area Pang buffer2 to an input first-in first-out queue FIFO of a second asynchronous sampling converter by the DMA2; and the data is subjected to inverse conversion of the sampling rate of the second asynchronous sampling converter, inverse conversion of integral multiple of the sampling rate of the up-sampling converter, conversion of the digital-to-analog converter and amplification of the power and amplitude amplifier, and then is played by the playing device.

Compared with the prior art, the invention has the following advantages and beneficial effects:

the invention further shortens the time delay of the conventional software for realizing the auxiliary listening method, and has simple realization method without increasing any hardware cost.

Drawings

FIG. 1 is a flow chart of signal processing of an auxiliary listening device in the prior art;

FIG. 2 is a schematic diagram of hardware components in the prior art;

FIG. 3 is a schematic diagram of a process for acquiring and processing a sound signal according to the prior art;

FIG. 4 is a schematic diagram illustrating a process of playing a sound signal in the prior art;

FIG. 5 is a diagram illustrating a software-based implementation of auxiliary listening in the prior art;

fig. 6 is a schematic diagram of the auxiliary listening implemented by software according to the present invention.

Detailed Description

The present invention will be described in further detail with reference to examples, but the embodiments of the present invention are not limited thereto.

Example 1:

the invention provides a method for shortening the delay of an earphone with auxiliary hearing, which comprises an analog microphone, an analog amplifier, an analog-to-digital converter, a down-sampling converter, a first asynchronous sampling converter, a first direct memory access controller DMA1, a memory, a microprocessor unit/a digital signal processor MCU/DSP which are sequentially in communication connection, and further comprises a playing device, an amplitude power amplifier, a digital-to-analog converter, an up-sampling converter, a second asynchronous sampling converter, a second direct memory access controller DMA2, the memory and the microprocessor unit/the digital signal processor MCU/DSP which are sequentially in communication connection, wherein an audio signal acquired by the analog microphone is amplified by the analog amplifier, converted by the analog-to-digital converter and converted by integral multiple of sampling rate of the down-sampling converter (for example, a 1000k sampling signal is converted into a 10k sampling rate), and then is converted into a signal with a set sampling frequency by the first asynchronous sampling converter (for example, the sampling frequency of an input signal is arbitrarily converted into a sampling frequency which is required to be output, the sampling frequency which can be in a decimal relation), the first direct memory access controller DMA1 is started, and the first direct memory access controller DMA1 is moved to a buffer unit of the MCU/DSP to finish the data processing of a buffer unit after the output of the FIFO/buffer unit; then DMA1 transfers data from the output FIFO of the first asynchronous sampling converter to a buffer area Pang buffer1 in the memory, and generates an interrupt signal to the MCU/DSP after the transfer is finished; the data are alternately transferred from the output FIFO of the first asynchronous sampling converter to the buffer area Ping buffer1 and the buffer area Pang buffer1 in the circulating way;

after receiving the first interrupt signal, the MCU/DSP reads the first frame data from the buffer Ping buffer1, processes the first frame data and moves the first frame data to a buffer Ping buffer2 in the memory; after receiving the interrupt signal again, the MCU/DSP reads data from the buffer Pang buffer1, processes the data and moves the data to a buffer Pang buffer2 in the memory, and the process is circulated, and the data are alternately moved from the buffer Ping buffer1 and the buffer Pang buffer1 to the buffer Ping buffer2 and the buffer Pang buffer2;

when the MCU/DSP reads the first frame data from the buffer Ping buffer1 and moves the first frame data to the buffer Ping buffer2 after processing, the second direct memory access controller DMA2 is started at the moment, and the DMA2 starts to alternately move the data of the buffer Ping buffer2 and the buffer Pang buffer2 to the input FIFO of the second asynchronous sampling converter;

as shown in fig. 6, the specific process is as follows:

1. firstly, configuring parameters of each hardware module;

2. starting DMA1;

3. after one DMA1 is interrupted, the MCU/DSP receives first frame data, and then the MCU/DSP performs signal processing for delta;

4. after the signal processing is finished, copying the processed frame data to Ping buffer2, and then starting DMA2;

5. DMA2 starts to carry the data of Ping buffer2;

6. when the second DMA1 is interrupted, the MCU/DSP processes signals, and sends the second frame data to the Pang buffer2 after the second frame data is processed (because the starting time of the DMA1 is one frame + delta later than that of the DMA2, the DMA2 still carries the Ping buffer2 data when the analog microphone sends the data to the Pang buffer 2);

7. when the third DMA1 is interrupted, the MCU/DSP processes signals, and sends the third frame of data to the Ping buffer2 after the processing is finished (because the starting time of the DMA2 is one frame + delta later than that of the DMA1, the DMA2 still carries the data of the Ping buffer2 when the analog microphone sends the data to the Ping buffer 2);

8. and 6,7, repeatedly circulating.

And the data is subjected to inverse conversion of the sampling rate of the second asynchronous sampling converter, inverse conversion of integral multiple of the sampling rate of the up-sampling converter, conversion of the digital-to-analog converter and amplification of the power and amplitude amplifier, and then is played by the playing device.

This patented method total delay = hardware delay (fixed) + fragment size + Δ, where: the time of delta depends on the algorithm complexity, MCU/DSP operation performance and operation dominant frequency. Usually Δ must be less than the fragment size time (if Δ > fragment size, it is usually considered that CPU/DSP performance is not sufficient or that the main frequency is not high enough, and that the entire secondary listening system design is problematic to complete the signal processing at one fragment size). Therefore, the invention can further shorten the delay on the method for realizing the auxiliary listening by the traditional software, and compared with the prior art, the invention shortens the time ST: ST = fragment size- Δ.

Although the invention has been described herein with reference to the illustrated embodiments thereof, which are intended to be the only preferred embodiments of the invention, it is not intended that the invention be limited thereto, since many other modifications and embodiments will be apparent to those skilled in the art and will be within the spirit and scope of the principles of this disclosure.

Claims

1. A method for shortening the delay of an earphone with auxiliary hearing comprises an analog microphone, an analog amplifier, an analog-to-digital converter, a down-sampling converter, a first asynchronous sampling converter, a first direct memory access controller DMA1, a memory and a microprocessor unit/digital signal processor MCU/DSP which are sequentially in communication connection, and also comprises a playing device, an amplitude power amplifier, a digital-to-analog converter, an up-sampling converter, a second asynchronous sampling converter, a second direct memory access controller DMA2, the memory and the microprocessor unit/digital signal processor MCU/DSP which are sequentially in communication connection, and is characterized in that the audio processing method comprises the following steps:

firstly, starting a first direct memory access controller DMA1, wherein the DMA1 transfers data from an output first-in first-out queue FIFO of a first asynchronous sampling converter to a designated initial address of a memory, and after a microprocessor unit/digital signal processor MCU/DSP acquires and processes a frame of data from the designated initial address of the memory and transfers the frame of data to a designated destination address of the memory, then starting a second direct memory access controller DMA2, and the DMA2 transfers data from the designated destination address of the memory to an input first-in first-out queue FIFO of a second asynchronous sampling converter.

2. The method of claim 1, wherein the audio processing method specifically comprises:

when an audio signal acquired by an analog microphone is amplified by an analog amplifier, converted by an analog-to-digital converter, converted by a down-sampling converter at integral multiple of sampling rate and converted into a signal with a set sampling frequency by a first asynchronous sampling converter, starting a first direct memory access controller DMA1, alternately transferring data from an output first-in first-out queue FIFO of the first asynchronous sampling converter to a buffer area Ping buffer1 and a buffer area Pang buffer1 in a memory after the DMA1 is started, and generating an interrupt signal to a microprocessor unit/digital signal processor MCU/DSP after the DMA1 is transferred each time; after receiving the first interrupt signal, the MCU/DSP reads data from the buffer Ping buffer1, processes the data and then moves the data to a buffer Ping buffer2 in the memory, and after receiving the interrupt signal again, the MCU/DSP reads data from the buffer Pang buffer1, processes the data and then moves the data to the buffer Pang buffer2 in the memory, and the data is moved alternately in this way; after the MCU/DSP finishes moving the first frame data, starting a second direct memory access controller DMA2, and starting the DMA2 to alternately move the data of the buffer area Ping buffer2 and the buffer area Pang buffer2 to an input first-in first-out queue FIFO of a second asynchronous sampling converter; the data is played by the playing device after being subjected to sampling rate inverse conversion by a second asynchronous sampling converter, sampling rate integral multiple inverse conversion by an upsampling converter, conversion by a digital-to-analog converter and amplification by a power and amplitude amplifier PA.