nrf5340 stereo output using external codec on SDK 2.6.1

[EchoCharlieWhiskey]

I am trying to make my gateway send stereo data to the headset where it will be played on an external codec. I want this to be a single stereo stream.

I had this working in SDK 2.5.1 but can't seem to get it working with SDK 2.6.1.

The gateway is an unmodified nrf5340 audio DK using the standard board files.

The headset is also an nrf5430 audio DK with an external codec wired up. It uses a Zephyr board file to keep the codec interface line high (codec LED is orange). The headset code does not use the cirrus_cs47l63 driver (#def'd out with CONFIG_HW_CODEC_CIRRUS_LOGIC), my external codec is being configured externally.

I've added STEREO_CIS_HEADSET to my KConfig and set it in prj.conf along with:

CONFIG_STEREO_CIS_HEADSET=y

CONFIG_STREAM_BIDIRECTIONAL=y

CONFIG_TRANSPORT_CIS=y

CONFIG_AUDIO_SOURCE_I2S=y

In unicast_server\Kconfig.defaults, I modified LC3_DEC_CHAN_MAX:

config LC3_DEC_CHAN_MAX

    default 2 if STEREO_CIS_HEADSET

    default 1

In audio_system.c, audio_headset_configure, I added

#endif /* (CONFIG_STREAM_BIDIRECTIONAL) */

	if (IS_ENABLED(CONFIG_STEREO_CIS_HEADSET)) {
		sw_codec_cfg.decoder.num_ch = 2;
		sw_codec_cfg.decoder.channel_mode = SW_CODEC_STEREO;
	} else {
		sw_codec_cfg.decoder.num_ch = 1;
		sw_codec_cfg.decoder.channel_mode = SW_CODEC_MONO;
	}

In bt_le_audio_tx.c, bt_le_audio_tx_send, I modified

	if (streams_to_tx > SRC_STREAM_COUNT) {
		return -ENOMEM;
	}
	if (IS_ENABLED(CONFIG_STEREO_CIS_HEADSET)) {
		data_size_pr_stream = enc_audio.size;
	} else {
		if ((enc_audio.num_ch == 1) || (enc_audio.num_ch == streams_to_tx)) {
			data_size_pr_stream = enc_audio.size / enc_audio.num_ch;
		} else {
			LOG_ERR("Num encoded channels must be 1 or equal to num streams");
			return -EINVAL;
		}
	}

which led to runtime errors in the gateway about "The encoded data size does not match the SDU size"
where data_size_pr_stream was 120, LE_AUDIO_SDU_SIZE_OCTETS(bitrate) was 60, and bitrate was 48000.

I commended out the error, further down in the same function:

//        if (data_size_pr_stream != LE_AUDIO_SDU_SIZE_OCTETS(bitrate)) {
//            LOG_ERR("The encoded data size does not match the SDU size");
//            return -EINVAL;
//        }

In unicast_client.c, at the top, I added

static struct bt_audio_codec_qos lc3_preset_sink_stereo_qos = BT_AUDIO_CODEC_LC3_QOS_10_UNFRAMED(
	LE_AUDIO_SDU_SIZE_OCTETS(CONFIG_BT_AUDIO_BITRATE_UNICAST_SINK) * 2,
	CONFIG_BT_AUDIO_RETRANSMITS, CONFIG_BT_AUDIO_MAX_TRANSPORT_LATENCY_MS,
	CONFIG_BT_AUDIO_PRESENTATION_DELAY_US);

and then later in the same file, in unicast_client_enable I added the pointer to lc3_preset_sink_stereo_qos:

		if ((i % 2) == 0) {
			if (IS_ENABLED(CONFIG_STEREO_CIS_HEADSET)) {
				group_stream_params[i].qos = &lc3_preset_sink_stereo_qos;
			} else {
				group_stream_params[i].qos = &lc3_preset_sink.qos;
			}
			group_stream_params[i].stream = &headsets[headset_iterator].sink_stream;

In unicast_server.c, near the top, I added the two channels support:

static struct bt_audio_codec_cap lc3_codec_sink = BT_AUDIO_CODEC_CAP_LC3(
	BT_AUDIO_CODEC_CAPABILIY_FREQ,
	(BT_AUDIO_CODEC_LC3_DURATION_10 | BT_AUDIO_CODEC_LC3_DURATION_PREFER_10),
#if CONFIG_STEREO_CIS_HEADSET
	BT_AUDIO_CODEC_LC3_CHAN_COUNT_SUPPORT(2),
#else
	BT_AUDIO_CODEC_LC3_CHAN_COUNT_SUPPORT(1), 
#endif /* CONFIG_STEREO_CIS_HEADSET */
	LE_AUDIO_SDU_SIZE_OCTETS(CONFIG_LC3_BITRATE_MIN),
	LE_AUDIO_SDU_SIZE_OCTETS(CONFIG_LC3_BITRATE_MAX), 1u, AVAILABLE_SINK_CONTEXT);

I made no changes to the child image. 

Using a logic analyzer on my headset-to-codec I2S connection I can see that only one channel of data is being sent from the gateway tones mode. (Once tones works, I will hook the gateway up to USB and get stereo audio from my computer.)

As I said, I had this working in 2.5.1 based on reading past forum posts but can't get it in 2.6.1. Advice would be greatly appreciated.

Thank you very much,
ECW

[dejans]

Hi,

Are all mentioned configuration and code changes specified for NCS v2.6.1?

Best regards,
Dejan

[EchoCharlieWhiskey]

Yes.

[dejans]

Hi,

Could you specify all changes which you made when you used NCS v2.5.1 and when everything was working as expected?

When providing code changes you mentioned unicast_server and unicast_client. Did you modify these samples on the headset and gateway side?

Best regards,
Dejan

[EchoCharlieWhiskey]

My code aside, how should a CIS stereo stream from the gateway to the headset (with an external codec) work for SDK 2.6.1? Note that I also want a mono stream from the headset to the gateway and I have only one headset.

(And yes, my modifications above were for the unicast_server and unicast_client example code in 2.6.1, recompiling both gateway and headset.)

Also, from the forums here, having stereo in unicast mode is a common application. Does a white paper exist? 

[dejans]

Hi,

I had a discussion with our developers. Below are some relevant points.

EchoCharlieWhiskey said:

how should a CIS stereo stream from the gateway to the headset (with an external codec) work for SDK 2.6.1?

At the moment our application supports only mono bidirectional. However, you should be able to do either stereo in one BIS to the headset or 2 mono BIS to the headset, but this would require some code changes.

EchoCharlieWhiskey said:

having stereo in unicast mode is a common application. Does a white paper exist?

There is currently no such white paper. We are aiming at implementing full support for this.

The error that you see is what is expected. You could try to print sizes and compare values. You would need to split the data yourself and decode it. Please note that the tone module is mono by design. Even if this approach works, you would still see mono data on I2S. 
The suggestion is to first start with mono stream and duplicate the data after it has been received. Audio part of it can then be modified so that one gets 2 mono outputs, Then, changes can happen on the BLE and sending side.

Best regards,
Dejan

[EchoCharlieWhiskey]

In NRF5340: CIS Groups, and Streams: Stereo headset + stereo gateway + NUS?, you (dejans ) suggested I move to the new SDK though I described what I wanted in terms of strereo. Additionally, you said

"An audio stream is unidirectional stream which means there can be up to 2 audio streams in a single CIS. Each of the 2 audio streams could go different way as CIS can be bidirectional. It is possible to have multiple audio channels in a single audio stream. In other words, it is possible to have stereo in a single audio stream."

So I'm having some trouble understanding your current answer which seems to contradict that. I do not want to have two mono streams that need to be sync'd on my single device, I need it to be one stereo stream. How do I do that?

Thank you,

ECW

[dejans]

Hi,

I have asked internally for some clarifications. I will get back to you during next week.

Best regards,
Dejan

[dejans]

Hi,

I am sorry for a delayed reply. I have not yet got internal response. I will reach out to developers on Monday. I will get back to you hopefully during next week.

Thank you for your patience.

Best regards,
Dejan

[dejans]

Hi,

Please find below some suggestions from our developers.
It is possible to send both left and right data in a single ISO stream by stacking both left and right data in the same SDU. After encoding on the sender side, the data would be combined. Data would later be split on the receiving side. The downside of this is that it is rather uncommon and packets over the air might get too large potentially causing retransmissions due to CRC errors.
If the packets in the same CIG/BIG are sent with the same SDU reference, the receiving side would also perceive it like that. If you have one peripheral receiving 2 CISes, you should receive the same timestamp as long as they are in the same CIG. If the streams are sent with the same timestamp, then they should be in sync. You can send Left and Right in separate CISes to the same peripheral. On the receiving side, you can check if timestamps are the same and if they are that means that the streams are in sync (as they should be). If the timestamps happen to be different then this would need to be handled by the application. 

Best regards,
Dejan