Best place for audio processing adjustments.

[Jorritv3]

Hello all,

I am working on a project with 2 nrf5340 Audio DK's, and I have been working for a while now on getting myself familiar with the audio application, but I have a few questions. First of all I assumed it would be wise to develop on top of the provided audio application since writing the entire pairing and data transmission yourself would be near impossible, is this a correct assumption?

Secondly I have the walkie talkie demo running, and am now trying to encode and decode the data that I am sending over the line, for now I just want a simple xor operation to encrypt the data and decrypt it. Right now I am doing it before the codec encode in the audio_system.c and decrypting it after the decode, but I just get a horrible noise instead of the orginal audio. I tried a few other spots like inside the callback, but I saw another thread saying this was ill advised.

So my question is what would be the correct places to place the encrypt and decrypt of the data? 

[Einar Thorsrud]

Hi,

The codec is a lossy codec, so you cannod encrypt (with simple xoring or any other method) before coding. Any such operations will have to happen after coding, and then the reverse operation must be performed before decoding. (If you were doing simple one way audio processing where the output signal is also an audio signal, and there is no reverse operation in the other end, that could have been doen before encoding).

[Jorritv3]

That makes a lot of sense, thank you.

So I implemented it the way you are saying, and it seems to work. However if I only implement the encoding side without the decoding to check if the audio gets scrambled it doesn't output anything anymore. This feels like a checksum of some sort is being implemented somewhere.

Do you have an idea of why this doesn't audio anymore? I would like to check wether the audio gets encrypted correctly, so it would be nice if it still reaches the speaker of my headset.

Thank you.

P.S.

I placed a very simple

const uint16_t xor_key = 0xAA;

for (size_t i = 0; i < audio_frame->len; i++) {
	audio_frame->data[i] ^= xor_key;
}

before the sw_codec_decode in audio.datapath.c and the same after the sw_codec_encode in audio_system.c.

[Jorritv3]

Hi,

So I pass valid data to LC3DecodeSessionData(), not LC3DecodeOutput(), and it returns 0, so it should be valid. This is modified data in the sense that I applied an XOR operation to the bytes after encoding. And indeed they are not empty, but still the output frames are zero.

This problem only happens when I only encrypt the data (so after encoding). What I would expect is to hear noise (meaning in this case encrypted audio), but when I also apply the decryption it sounds normal again. I want to hear the noise to prove the audio is correctly being processed, but instead the audio gets set to 0 at the LC3DecodeSessionData().

[Einar Thorsrud]

Can you Is this data now encrypted before or after encoding? If you encrypt data before encoding I am not sure how the result will be, as the encoder is lossy and expects an audio signal. The encrypted signal is effectively white noise and impossible to compress in any measningfull way. It could perhaps also be that as there is no data erroc concelment will result in zero output (no noise)? If so, te return value will be 0 (LC3_RESULT_NO_ERROR) according to the API documentation for LC3DecodeSessionData.

[Jorritv3]

No I am encrypting after encoding, and decrypting before decoding.

[Einar Thorsrud]

I see. But then there should not be any difference with encryption or not, as the data into the decoder will be indentical to what it would be if ti was not encrypted. Is there a difference or did I misunderstand something? If this fails with encryption but not without it, you need to look into your encryption/decryption, as ther eseems to be an issue there. Can you verify that the decrypted data is identical to the data before encryption?

[Jorritv3]

I think you are misunderstanding, sorry for my unclear explanation.

I am doing an encryption after encoding, this would mean the encoded data that gets send to the other device should be encrypted. At the other device I do the decryption before the decoding, so the original sound should come out. 

When I do both encryption and decryption, I Do hear the original audio.
But when I Only do the encryption (So no decryption at the other side), I hear nothing.

I expect to hear noise since my data received at the other device it garbage. 

My question is why all PCM data gets set to 0 when decoding.

So at the decoding side encrypted data comes in, but that device shouldn't know it is encrypted, so how why does it produce no sound at all, instead of garbled noise.

[Einar Thorsrud]

Aha, then I se. I do not think that is the case. The codec will see that the input data is gabage and use error concealment. And as there is no data of value anywhere, the best concealment is simply no signal/silence.

[Jorritv3]

Hi,

Thank you for your reply, sorry for my late response.
But how does the codec know it is garbage?

[Einar Thorsrud]

I have not looked into the details, but there are several ways it could detect it. As it is encrypted and would look like just random noise, I expect it detects that the frame is invalid and sets  BEC_detect flag (see page 70 in LC3 spec v1.0.1). Also, I recommend you have a look at Appendix B Packet Loss Concealment.