Implementing Equalizer into nrf5340 Audio DK application

[flamefire]

Hello, i am looking to implement an equalizer into the Audio application for the nrf5340 Audio DK. I want to implement this, specifically on the gateway side, and alter the sound before sending it to the headset.

My question is where in the code would be an appropriate place to alter the sound, between recieving it from USB and sending it to Network core or DAC?

My current idea would be to implement another "step" in the audio_datapath function -> audio_datapath_stream_out. However the description says the following 

/**

 * @brief Input an audio data frame which is processed and outputted over I2S

 *

 * @note A frame of raw encoded audio data is inputted, and this data then is decoded

 *       and processed before being outputted over I2S. The audio is synchronized

 *       using sdu_ref_us

 *

 * @param buf Pointer to audio data frame

 * @param size Size of audio data frame in bytes

 * @param sdu_ref_us ISO timestamp reference from BLE controller

 * @param bad_frame Indicating if the audio frame is bad or not

 * @param recv_frame_ts_us Timestamp of when audio frame was received

 */

void audio_datapath_stream_out(const uint8_t *buf, size_t size, uint32_t sdu_ref_us, bool bad_frame,

                   uint32_t recv_frame_ts_us);

It says outputtet over I2C, and i would also want it to be outputtet over BLE.

Thanks

[Amanda Hsieh]

Hi, 

Are you going to implement the equalizer directly in the application core firmware?

The team would suggest to implement this new "step" in the le_audio_rx_data_handler() callback function in streamctrl.c. It is possible to access BLE send functions from streamctrl.c, in addition to the audio_datapath_stream_out(). You of course have to remember to account for the time delay this new equalizing introduces. Maybe the presentation delay has to be increased.

Regards,
Amanda H.

[flamefire]

Thanks for the reply.

There is something i dont quite understand. sumup: I want to equalize the audio data on the gateway between receiving it from I2S or USB and sending it to headset with BLE. The headset it just the standard configuration and recieves equalized audio.

Im wondering where the data is coming from in le_audio_rx_data_handler(), i see that it is connected to the callback function le_audio_enable(le_audio_receive_cb recv_cb) in the file (in my case) le_audio_bis_gateway.c, which starts the broadcast gateway, but where does le_audio_enable get the data from?

I have setup my input data to be from USB, and i presume thats where le_audio_enable gets the data from. Or if i change it to I2S then it would be from that input. Is this wrong?

Presumably its the right data from USB or I2S, the pointer to received data in le_audio_rx_data_handler() seems like its encoded, since its of type const uint8_t *const data so we should decode it? 

[Amanda Hsieh]

Hi, 

Sorry for misunderstanding the question (since the audio_datapath_stream_out() function was mentioned).

The proper position to manipulate the audio PCM data is before the sw_codec_encode():
https://github.com/nrfconnect/sdk-nrf/blob/8d552d2f049c096a6c7188299d39c5b4afe1b64e/applications/nrf5340_audio/src/audio/audio_system.c#L146

-Amanda H.

[flamefire]

Thanks for reply! That was also where we thought would be the best place. We have a problem with unpacking the  char pcm_raw_data, as we dont know the structure of the data, could you provide more information or point me in a direction? And how can we split it into left and right channel?

[Amanda Hsieh]

Hi, 

The data is split into the left and right channels in pscm_two_channel_split which is in sw_codec_encode().

So if you want to equalize the left and right separately I would do it here: https://github.com/nrfconnect/sdk-nrf/blob/main/applications/nrf5340_audio/src/audio/sw_codec_select.c#L64. The char pcm_data_mono is just pure PCM data.

pcm_raw_data is PCM data as well, but it's both left and right and the bit depth is 16 bits, so every two bytes it will alternate between the two channels.

-Amanda H.