Passing data from PDM callback to USB audio device

RE: Passing data from PDM callback to USB audio device

luke.zheng — Mon, 27 May 2024 02:11:48 GMT

Hi yoyou，

Have you solved the question?

I've the same problem in sdk_17

RE: Passing data from PDM callback to USB audio device

yoyou — Wed, 03 May 2023 01:22:35 GMT

i've the same porbem in nRF5_SDK_17,

I2S mic 16bit 48Khz stereo transferre to USB AUIDO.

some time app_usbd_audio_class_tx_start() return code 17.

RE: Passing data from PDM callback to USB audio device

Jeffrey Haynes — Tue, 09 Mar 2021 10:45:25 GMT

Did you ever figure this out? Looks like maybe the buffer size is meant to be 16 bytes but I'm getting time compression and whole bunch of noise.

RE: Passing data from PDM callback to USB audio device

NewtoM — Tue, 28 Apr 2020 22:43:27 GMT

Is there any news on this topic? The solution would interest me much!

RE: Passing data from PDM callback to USB audio device

Or — Mon, 02 Dec 2019 09:00:51 GMT

I send exactly what I get from the PDM callback when I have a buffer of 64 bytes:

16 samples X 2 channels (stereo) X 16 bit sampling (2 bytes) = 64 bytes

The audio is 16KHz which means the transfer rate is 64Kb/second

RE: Passing data from PDM callback to USB audio device

abhipray — Mon, 02 Dec 2019 05:02:07 GMT

How many samples are you sending via app_usbd_audio_class_tx_start()?

It should be 16 samples every 1ms SOF event for 16KHz audio.

RE: Passing data from PDM callback to USB audio device

DrYang — Mon, 18 Nov 2019 08:37:39 GMT

I have same trouble....!!!

RE: Passing data from PDM callback to USB audio device

Or — Mon, 18 Nov 2019 08:36:00 GMT

But the sample rate of the source is 16KHz ?!

If I set the sample rate to 48KHz wouldnt the host expect 48KHz data and distort the audio ?

RE: Passing data from PDM callback to USB audio device

DrYang — Mon, 18 Nov 2019 08:28:14 GMT

The reason is your sample rate, this problem will be fixed if you set the sample rate to 48KHz, and also repeat if you set the sample rate to other value!

RE: Passing data from PDM callback to USB audio device

Kenneth — Thu, 17 Oct 2019 16:41:19 GMT

I see your point, unfortunately I don't see this being prioritized at the moment.

Kenneth

RE: Passing data from PDM callback to USB audio device

Or — Tue, 15 Oct 2019 07:50:37 GMT

I've changed both NRFX_USBD_CONFIG_IRQ_PRIORITY and USBD_CONFIG_IRQ_PRIORITY to 0 (highest, they were set to 6) to no avail.

Is there a chance one of the SDK developers can take a look at my code ? I'm pretty sure I follow exactly the code example and I still cant make it work (PDM audio through USB).

There is no code example on the internet that does just that and actually works. Making this one work will be a breakthrough that will be useful for other developers who are interested in using nrf52840 for USB audio.

RE: Passing data from PDM callback to USB audio device

Kenneth — Mon, 14 Oct 2019 15:07:24 GMT

Maybe you are not handling the USB callbacks fast enough due to other application interrupts processing, for instance that the app_usbd_core_event_handler() give a response to late. Maybe try to change NRFX_USBD_CONFIG_IRQ_PRIORITY.

RE: Passing data from PDM callback to USB audio device

Or — Mon, 14 Oct 2019 09:11:48 GMT

What am I doing wrong in the firmware code (which I attached) that causes this behavior ? is this a bug in the SDK ?

IS this related to the distorted audio I get ?

RE: Passing data from PDM callback to USB audio device

Kenneth — Mon, 14 Oct 2019 08:15:27 GMT

Adding the USB sniffer file, I do believe you should be able to open it by downloading Ellisys Visual USB. Seems as the application does not reply to the first 'get device descriptor' from the USB host, instead the intended data is sent on the next 'get configuration descriptor'.

Sniffer trace:

devzone.nordicsemi.com/.../Case_5F00_236141.zip

Seems as the IN transaction from the nRF52840 which should be a response to 'get device descriptor', is instead replied on the next 'get configuration descriptors' from the USB host.

For comparison see the USB audio reference:

devzone.nordicsemi.com/.../Case_5F00_236141_5F00_reference_5F00_usb_5F00_audio_5F00_sdk15.2.zip

RE: Passing data from PDM callback to USB audio device

Or — Thu, 26 Sep 2019 09:07:17 GMT

I'm attaching here all information that might be relevant

Linux dmesg:

[590038.946652] usb 1-3: new full-speed USB device number 6 using xhci_hcd
[590039.097423] usb 1-3: New USB device found, idVendor=1915, idProduct=521a
[590039.097430] usb 1-3: New USB device strings: Mfr=1, Product=1, SerialNumber=1
[590039.097434] usb 1-3: Product: 000000000000
[590039.097437] usb 1-3: Manufacturer: 000000000000
[590039.097439] usb 1-3: SerialNumber: 000000000000
[590109.320154] retire_capture_urb: 431 callbacks suppressed
[590153.033160] retire_capture_urb: 708 callbacks suppressed

Query USB device under Linux (lsusb -d 1915:521a -v):

Bus 001 Device 006: ID 1915:521a Nordic Semiconductor ASA 
Couldn't open device, some information will be missing
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x1915 Nordic Semiconductor ASA
  idProduct          0x521a 
  bcdDevice            1.00
  iManufacturer           1 
  iProduct                1 
  iSerial                 1 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength          113
    bNumInterfaces          2
    bConfigurationValue     1
    iConfiguration          4 
    bmAttributes         0xc0
      Self Powered
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      1 Control Device
      bInterfaceProtocol      0 
      iInterface              0 
      AudioControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      1 (HEADER)
        bcdADC               1.00
        wTotalLength           43
        bInCollection           1
        baInterfaceNr( 0)       1
      AudioControl Interface Descriptor:
        bLength                12
        bDescriptorType        36
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bTerminalID             1
        wTerminalType      0x0201 Microphone
        bAssocTerminal          0
        bNrChannels             2
        wChannelConfig     0x0003
          Left Front (L)
          Right Front (R)
        iChannelNames           0 
        iTerminal               0 
      AudioControl Interface Descriptor:
        bLength                13
        bDescriptorType        36
        bDescriptorSubtype      6 (FEATURE_UNIT)
        bUnitID                 2
        bSourceID               1
        bControlSize            2
        bmaControls( 0)      0x01
        bmaControls( 0)      0x00
          Mute Control
        bmaControls( 1)      0x01
        bmaControls( 1)      0x00
          Mute Control
        bmaControls( 2)      0x01
        bmaControls( 2)      0x00
          Mute Control
        iFeature                0 
      AudioControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bTerminalID             3
        wTerminalType      0x0101 USB Streaming
        bAssocTerminal          0
        bSourceID               2
        iTerminal               0 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol      0 
      iInterface              0 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       1
      bNumEndpoints           1
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol      0 
      iInterface              0 
      AudioStreaming Interface Descriptor:
        bLength                 7
        bDescriptorType        36
        bDescriptorSubtype      1 (AS_GENERAL)
        bTerminalLink           1
        bDelay                  0 frames
        wFormatTag              1 PCM
      AudioStreaming Interface Descriptor:
        bLength                11
        bDescriptorType        36
        bDescriptorSubtype      2 (FORMAT_TYPE)
        bFormatType             1 (FORMAT_TYPE_I)
        bNrChannels             2
        bSubframeSize           2
        bBitResolution         16
        bSamFreqType            1 Discrete
        tSamFreq[ 0]        16000
        AudioControl Endpoint Descriptor:
          bLength                 7
          bDescriptorType        37
          bDescriptorSubtype      1 (EP_GENERAL)
          bmAttributes         0x00
          bLockDelayUnits         0 Undefined
          wLockDelay              0 Undefined
      Endpoint Descriptor:
        bLength                 9
        bDescriptorType         5
        bEndpointAddress     0x88  EP 8 IN
        bmAttributes            1
          Transfer Type            Isochronous
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval               1
        bRefresh                0
        bSynchAddress           0

Full source code (main.c 414 lines based on usb_audio example):

#include <stdint.h>
#include <stdbool.h>
#include <stddef.h>

#include "nrf.h"
#include "app_util.h"
#include "nrf_drv_usbd.h"
#include "nrf_drv_clock.h"
#include "nrf_gpio.h"
#include "nrf_delay.h"
#include "nrf_drv_power.h"
#include "nrf_drv_pdm.h"

#include "app_usbd.h"
#include "app_usbd_core.h"
#include "app_usbd_string_desc.h"
#include "app_usbd_audio.h"
#include "app_error.h"
#include "boards.h"

#include "nrf_drv_clock.h"
#include "nrf_fstorage_nvmc.h"

#include "nrf_log.h"
#include "nrf_log_ctrl.h"
#include "nrf_log_default_backends.h"

// buffers for the PDM source (base NRFX_PDM_MAX_BUFFER_SIZE)
#define NRFX_PDM_BUFFER_SIZE (32)
static int8_t pdm_buffer_i = 0;
static int16_t pdm_buffer[2][NRFX_PDM_BUFFER_SIZE];
static volatile uint8_t usb_tx_idle = 1, audio_buffer_ready = 0;

#define LED_USB_RESUME (BSP_BOARD_LED_0)
#define LED_USB_START  (BSP_BOARD_LED_1)
#define LED_AUDIO_RX   (BSP_BOARD_LED_2)
#define LED_AUDIO_TX   (BSP_BOARD_LED_3)

/**
 * @brief Enable power USB detection
 *
 * Configure if example supports USB port connection
 */
#ifndef USBD_POWER_DETECTION
#define USBD_POWER_DETECTION true
#endif

/**
 * @brief Audio class user event handler
 */
static void mic_audio_user_ev_handler(app_usbd_class_inst_t const * p_inst,
                                      app_usbd_audio_user_event_t   event);

/* Channels and feature controls configuration */

/**
 * @brief   Input terminal channel configuration
 */
#define MIC_TERMINAL_CH_CONFIG()                                                                       \
        (APP_USBD_AUDIO_IN_TERM_CH_CONFIG_LEFT_FRONT | APP_USBD_AUDIO_IN_TERM_CH_CONFIG_RIGHT_FRONT)

/**
 * @brief   Feature controls
 *
 *      general
 *      channel 0
 *      channel 1
 */
#define MIC_FEATURE_CONTROLS()                                                                     \
        APP_USBD_U16_TO_RAW_DSC(APP_USBD_AUDIO_FEATURE_UNIT_CONTROL_MUTE),                         \
        APP_USBD_U16_TO_RAW_DSC(APP_USBD_AUDIO_FEATURE_UNIT_CONTROL_MUTE),                         \
        APP_USBD_U16_TO_RAW_DSC(APP_USBD_AUDIO_FEATURE_UNIT_CONTROL_MUTE)


/* Microphone descriptors */

/**
 * @brief   Audio class specific format descriptor
 */
APP_USBD_AUDIO_FORMAT_DESCRIPTOR(m_mic_form_desc, 
                                 APP_USBD_AUDIO_AS_FORMAT_I_DSC(    /* Format type 1 descriptor */
                                    2,                              /* Number of channels */
                                    2,                              /* Subframe size */
                                    16,                             /* Bit resolution */
                                    1,                              /* Frequency type */
                                    APP_USBD_U24_TO_RAW_DSC(16000)) /* Frequency */
                                );

/**
 * @brief   Audio class input terminal descriptor
 */
APP_USBD_AUDIO_INPUT_DESCRIPTOR(m_mic_inp_desc, 
                                APP_USBD_AUDIO_INPUT_TERMINAL_DSC(
                                    1,                                     /* Terminal ID */
                                    APP_USBD_AUDIO_TERMINAL_IN_MICROPHONE, /* Terminal type */
                                    2,                                     /* Number of channels */
                                    MIC_TERMINAL_CH_CONFIG())              /* Channels config */
                                );

/**
 * @brief   Audio class output terminal descriptor
 */
APP_USBD_AUDIO_OUTPUT_DESCRIPTOR(m_mic_out_desc, 
                                 APP_USBD_AUDIO_OUTPUT_TERMINAL_DSC(
                                    3,                                     /* Terminal ID */
                                    APP_USBD_AUDIO_TERMINAL_USB_STREAMING, /* Terminal type */
                                    2)                                     /* Source ID */
                                );

/**
 * @brief   Audio class feature unit descriptor
 */
APP_USBD_AUDIO_FEATURE_DESCRIPTOR(m_mic_fea_desc, 
                                  APP_USBD_AUDIO_FEATURE_UNIT_DSC(
                                    2,                      /* Unit ID */
                                    1,                      /* Source ID */
                                    MIC_FEATURE_CONTROLS()) /* List of controls */
                                 );

/* Interfaces lists */

/**
 * @brief Interfaces list passed to @ref APP_USBD_AUDIO_GLOBAL_DEF
 */
#define MIC_INTERFACES_CONFIG() APP_USBD_AUDIO_CONFIG_IN(0, 1)

/*lint -save -e26 -e64 -e123 -e505 -e651*/

/**
 * @brief Microphone Audio class instance
 */
APP_USBD_AUDIO_GLOBAL_DEF(m_app_audio_microphone,
                          MIC_INTERFACES_CONFIG(),
                          mic_audio_user_ev_handler,
                          &m_mic_form_desc,
                          &m_mic_inp_desc,
                          &m_mic_out_desc,
                          &m_mic_fea_desc,
                          0,
                          APP_USBD_AUDIO_AS_IFACE_FORMAT_PCM,
                          64,
                          APP_USBD_AUDIO_SUBCLASS_AUDIOSTREAMING
);

/**
 * @brief Actual microphone mute state
 */
static uint8_t  m_mute_mic;

/**
 * @brief Actual microphone sampling frequency
 */
static uint32_t m_freq_mic;

/**
 * @brief Audio class specific request handle (microphone)
 */
static void mic_audio_user_class_req(app_usbd_class_inst_t const * p_inst)
{
    app_usbd_audio_t const * p_audio = app_usbd_audio_class_get(p_inst);
    app_usbd_audio_req_t * p_req = app_usbd_audio_class_request_get(p_audio);

    UNUSED_VARIABLE(m_mute_mic);
    UNUSED_VARIABLE(m_freq_mic);

    switch (p_req->req_target)
    {
        case APP_USBD_AUDIO_CLASS_REQ_IN:
        	NRF_LOG_INFO("APP_USBD_AUDIO_CLASS_REQ_IN");

            if (p_req->req_type == APP_USBD_AUDIO_REQ_SET_CUR)
            {
                //Only mute control is defined
                p_req->payload[0] = m_mute_mic;
            }

            break;
        case APP_USBD_AUDIO_CLASS_REQ_OUT:
        	NRF_LOG_INFO("APP_USBD_AUDIO_CLASS_REQ_OUT");

            if (p_req->req_type == APP_USBD_AUDIO_REQ_SET_CUR)
            {
                //Only mute control is defined
                m_mute_mic = p_req->payload[0];
                NRF_LOG_INFO("m_mute_mic = %d", m_mute_mic);
            }

            break;
        case APP_USBD_AUDIO_EP_REQ_IN:
        	NRF_LOG_INFO("APP_USBD_AUDIO_EP_REQ_IN");

            break;
        case APP_USBD_AUDIO_EP_REQ_OUT:
        	NRF_LOG_INFO("APP_USBD_AUDIO_EP_REQ_OUT");

            if (p_req->req_type == APP_USBD_AUDIO_REQ_SET_CUR)
            {
                //Only set frequency is supported
            	NRF_LOG_INFO("APP_USBD_AUDIO_EP_REQ_OUT -> APP_USBD_AUDIO_REQ_SET_CUR");
                m_freq_mic = uint24_decode(p_req->payload);
            }

            break;
        default:
        	NRF_LOG_INFO("mic_audio_user_class_req(default)");
            break;
    }
}

/**
 * @brief User event handler @ref app_usbd_audio_user_ev_handler_t (microphone)
 */
static void mic_audio_user_ev_handler(app_usbd_class_inst_t const * p_inst,
                                      app_usbd_audio_user_event_t   event)
{
    app_usbd_audio_t const * p_audio = app_usbd_audio_class_get(p_inst);
    UNUSED_VARIABLE(p_audio);

    switch (event)
    {
        case APP_USBD_AUDIO_USER_EVT_CLASS_REQ:
            mic_audio_user_class_req(p_inst);
            break;
        case APP_USBD_AUDIO_USER_EVT_TX_DONE:
        {
        	usb_tx_idle = 1;
            bsp_board_led_invert(LED_AUDIO_TX);
            //NRF_LOG_INFO("mic frame txed");
            break;
        }
        default:
        	NRF_LOG_INFO("mic_audio_user_ev_handler(default)");
            break;
    }
}

/**
 * @brief USBD library specific event handler.
 *
 * @param event     USBD library event.
 */
static void usbd_user_ev_handler(app_usbd_event_type_t event)
{
    switch (event)
    {
        case APP_USBD_EVT_DRV_SOF:
            break;

        case APP_USBD_EVT_DRV_SETUP:
        	NRF_LOG_INFO("USB setup");

        	break;
        case APP_USBD_EVT_DRV_SUSPEND:
        	NRF_LOG_INFO("USB suspend");

            bsp_board_leds_off();
            break;
        case APP_USBD_EVT_DRV_RESUME:
        	NRF_LOG_INFO("USB resume");

            bsp_board_led_on(LED_USB_RESUME);
            break;
        case APP_USBD_EVT_STARTED:
        	NRF_LOG_INFO("USB started");

            bsp_board_led_on(LED_USB_START);

            // start PDM
            APP_ERROR_CHECK(nrfx_pdm_start());

            break;
        case APP_USBD_EVT_STOPPED:
        	NRF_LOG_INFO("USB stopped");

        	// start PDM
        	APP_ERROR_CHECK(nrfx_pdm_stop());

            app_usbd_disable();
            bsp_board_leds_off();
            break;
        case APP_USBD_EVT_POWER_DETECTED:
            NRF_LOG_INFO("USB power detected");

            if (!nrf_drv_usbd_is_enabled())
            {
                app_usbd_enable();
            }
            break;
        case APP_USBD_EVT_POWER_REMOVED:
            NRF_LOG_INFO("USB power removed");
            app_usbd_stop();
            break;
        case APP_USBD_EVT_POWER_READY:
            NRF_LOG_INFO("USB ready");
            app_usbd_start();
            break;
        default:
            break;
    }
}

static void mic_sof_ev_handler(uint16_t framecnt)
{
	// make sure we're in proper state (configured)
	if (APP_USBD_STATE_Configured != app_usbd_core_state_get()) return;

	// are we in the middle of a transfer ?
	if (!usb_tx_idle) return;

	// do we have a buffer ready for transfer ?
	if (audio_buffer_ready == 0) return;

	// transfer the off-line buffer to the USB device now
	usb_tx_idle = 0;
	ret_code_t ret = app_usbd_audio_class_tx_start(&m_app_audio_microphone.base,
			pdm_buffer[pdm_buffer_i == 0 ? 1 : 0], NRFX_PDM_BUFFER_SIZE*2);

	// haven't started transferring -> wont reach TX done
	if (ret != NRF_SUCCESS) {
		usb_tx_idle = 1;
		NRF_LOG_ERROR("mic_sof_ev_handler() -> app_usbd_audio_class_tx_start() = %d", ret);
	}

	// this buffer is being transferred
	audio_buffer_ready = 0;
}

static void nrfx_pdm_event_handler(nrfx_pdm_evt_t const *const p_evt)
{
	// we have an error ?
	if (p_evt->error != NRFX_PDM_NO_ERROR) {
		NRF_LOG_INFO("PDM error event: %d", p_evt->error);
		return;
	}

	// expected to set a new buffer ?
	if (p_evt->buffer_requested) {
		// update buffer index
		pdm_buffer_i = pdm_buffer_i == 0 ? 1 : 0;

		// set new buffer
		nrfx_pdm_buffer_set(pdm_buffer[pdm_buffer_i], NRFX_PDM_BUFFER_SIZE);
	}

	// got a buffer with data ?
	if (p_evt->buffer_released != NULL) {
		// we have buffer ready
		audio_buffer_ready = 1;
	}
}

int main(void)
{
    ret_code_t ret;
    static const app_usbd_config_t usbd_config = {
        .ev_state_proc = usbd_user_ev_handler,
        .enable_sof = true
    };

    ret = NRF_LOG_INIT(NULL);
    APP_ERROR_CHECK(ret);
    NRF_LOG_DEFAULT_BACKENDS_INIT();

    ret = nrf_drv_clock_init();
    APP_ERROR_CHECK(ret);

    nrfx_pdm_config_t config = NRFX_PDM_DEFAULT_CONFIG(ARDUINO_SCL_PIN, ARDUINO_SDA_PIN);
    ret = nrfx_pdm_init(&config, nrfx_pdm_event_handler);
    APP_ERROR_CHECK(ret);

    NRF_LOG_INFO("USBD audio example started.");

    // Initialize LEDs and buttons
    bsp_board_init(BSP_INIT_LEDS | BSP_INIT_BUTTONS);

    ret = app_usbd_init(&usbd_config);
    APP_ERROR_CHECK(ret);

    app_usbd_class_inst_t const * class_inst_mic =
        app_usbd_audio_class_inst_get(&m_app_audio_microphone);

    ret = app_usbd_audio_sof_interrupt_register(class_inst_mic, mic_sof_ev_handler);
    APP_ERROR_CHECK(ret);

    ret = app_usbd_class_append(class_inst_mic);
    APP_ERROR_CHECK(ret);

    if (USBD_POWER_DETECTION)
    {
        ret = app_usbd_power_events_enable();
        APP_ERROR_CHECK(ret);
    }
    else
    {
        NRF_LOG_INFO("No USB power detection enabled\r\nStarting USB now");

        app_usbd_enable();
        app_usbd_start();
    }

    NRF_LOG_INFO("Entering main loop");

    while (true)
    {
        while (app_usbd_event_queue_process())
        {
            /* Nothing to do */
        }

        UNUSED_RETURN_VALUE(NRF_LOG_PROCESS());
        /* Sleep CPU only if there was no interrupt since last loop processing */
        __WFE();
    }
}

/** @} */

hex file (for nRF52840-DK):
devzone.nordicsemi.com/.../5241.nrf52840_5F00_xxaa.hex

If you need anything else just say

RE: Passing data from PDM callback to USB audio device

Kenneth — Wed, 25 Sep 2019 19:41:26 GMT

Unfortunately I am not able to take time to sit down and debug your project at the moment, but if you can send me the precompiled hex file I can take a quick look at the USB sniffer trace.

RE: Passing data from PDM callback to USB audio device

Or — Tue, 24 Sep 2019 10:24:13 GMT

You dont need to strip the PDM parts, it will perform just the same without microphones connected and will fill the buffers with trash.

The USB part will still behave the same

RE: Passing data from PDM callback to USB audio device

Kenneth — Mon, 23 Sep 2019 16:58:21 GMT

Is there no way you can strip away the PDM for now so I may run this on a devkit without PDM?

Kenneth

RE: Passing data from PDM callback to USB audio device

Or — Mon, 23 Sep 2019 13:36:25 GMT

I have embedded the whole main.c source file above 6 days ago.

You can comment out the fstorage parts since you wont need them and find 2 PDM mics for input source.
This was based on usb_audio code so same sdk_config (but you need to add PDM support).

Thanks for helping, something doesnt make sense in the USB functionality

RE: Passing data from PDM callback to USB audio device

Kenneth — Mon, 23 Sep 2019 13:28:36 GMT

If you can send me a hex file I may program to an nRF52840 DK here, then I can take an USB sniffer trace to check if I can see issues.

Kenneth

RE: Passing data from PDM callback to USB audio device

Or — Sun, 22 Sep 2019 11:00:10 GMT

After debugging the SDK and checking why I get NRF_ERROR_INVALID_STATE when I call app_usbd_audio_class_tx_start() I found out that it is returned from inside the following if statement:

if (!nrf_drv_usbd_ep_enable_check(ep))
{
    return NRF_ERROR_INVALID_STATE;
}

inside the file "nRF5_SDK_15.3.0/components/libraries/usbd/app_usbd.c" inside function app_usbd_ep_transfer() (line 1820).

What can disable the endpoint ? why does it happen in the middle of using it ?

RE: Passing data from PDM callback to USB audio device

Or — Tue, 17 Sep 2019 20:06:16 GMT

Notice where I placed app_usbd_audio_class_tx_start() ..

It is inside the usbd_user_ev_handler() in a switch case of APP_USBD_EVT_DRV_SOF.
Wouldnt that event be called only if a device is at least in activated state ?

Another thing - first thing I check is that the device is configured (if (APP_USBD_STATE_Configured != app_usbd_core_state_get()) break;).
Doesnt a device reach configured state after it is activated ?

I attach here the entire main.c for you to take a look at (maybe you will notice something I have missed):

#include <stdint.h>
#include <stdbool.h>
#include <stddef.h>

#include "nrf.h"
#include "app_util.h"
#include "nrf_drv_usbd.h"
#include "nrf_drv_clock.h"
#include "nrf_gpio.h"
#include "nrf_delay.h"
#include "nrf_drv_power.h"
#include "nrf_drv_pdm.h"

#include "app_usbd.h"
#include "app_usbd_core.h"
#include "app_usbd_string_desc.h"
#include "app_usbd_audio.h"
#include "app_error.h"
#include "boards.h"

#include "nrf_drv_clock.h"
#include "nrf_fstorage_nvmc.h"

#include "nrf_log.h"
#include "nrf_log_ctrl.h"
#include "nrf_log_default_backends.h"

/**@file
 * @defgroup usbd_aduio_example main.c
 * @{
 * @ingroup usbd_audio_example
 * @brief USBD Audio class example
 *
 */

/* FSTORAGE
 */
void fstorage_callback(nrf_fstorage_evt_t * p_evt);

#define FSTORAGE_BASE_ADDR 0x60000
#define FSTORAGE_END_ADDR 0x80000

static volatile uint32_t flash_addr = FSTORAGE_BASE_ADDR;
static volatile uint8_t fstorage_write_done = 1;

NRF_FSTORAGE_DEF(nrf_fstorage_t fstorage_instance) =
{
    .evt_handler    = fstorage_callback,
    .start_addr     = FSTORAGE_BASE_ADDR,
    .end_addr       = FSTORAGE_END_ADDR,
};

// buffers for the PDM source
#define NRFX_PDM_BUFFER_SIZE 500
static volatile int8_t pdm_buffer_i = 0;
static int16_t pdm_buffer[2][NRFX_PDM_BUFFER_SIZE];
static volatile int8_t usb_tx_done = 1, audio_buffer_txed = 1;

#define LED_USB_RESUME (BSP_BOARD_LED_0)
#define LED_USB_START  (BSP_BOARD_LED_1)
#define LED_AUDIO_RX   (BSP_BOARD_LED_2)
#define LED_AUDIO_TX   (BSP_BOARD_LED_3)

/**
 * @brief USB audio samples size
 */
#define BUFFER_SIZE  (48)

/**
 * @brief Enable power USB detection
 *
 * Configure if example supports USB port connection
 */
#ifndef USBD_POWER_DETECTION
#define USBD_POWER_DETECTION true
#endif

/**
 * @brief Audio class user event handler
 */
static void mic_audio_user_ev_handler(app_usbd_class_inst_t const * p_inst,
                                      app_usbd_audio_user_event_t   event);

/* Channels and feature controls configuration */

/**
 * @brief   Input terminal channel configuration
 */
#define MIC_TERMINAL_CH_CONFIG()                                                                       \
        (APP_USBD_AUDIO_IN_TERM_CH_CONFIG_LEFT_FRONT | APP_USBD_AUDIO_IN_TERM_CH_CONFIG_RIGHT_FRONT)

/**
 * @brief   Feature controls
 *
 *      general
 *      channel 0
 *      channel 1
 */
#define MIC_FEATURE_CONTROLS()                                                                     \
        APP_USBD_U16_TO_RAW_DSC(APP_USBD_AUDIO_FEATURE_UNIT_CONTROL_MUTE),                         \
        APP_USBD_U16_TO_RAW_DSC(APP_USBD_AUDIO_FEATURE_UNIT_CONTROL_MUTE),                         \
        APP_USBD_U16_TO_RAW_DSC(APP_USBD_AUDIO_FEATURE_UNIT_CONTROL_MUTE)


/* Microphone descriptors */

/**
 * @brief   Audio class specific format descriptor
 */
APP_USBD_AUDIO_FORMAT_DESCRIPTOR(m_mic_form_desc, 
                                 APP_USBD_AUDIO_AS_FORMAT_I_DSC(    /* Format type 1 descriptor */
                                    2,                              /* Number of channels */
                                    2,                              /* Subframe size */
                                    16,                             /* Bit resolution */
                                    1,                              /* Frequency type */
                                    APP_USBD_U24_TO_RAW_DSC(16000)) /* Frequency */
                                );

/**
 * @brief   Audio class input terminal descriptor
 */
APP_USBD_AUDIO_INPUT_DESCRIPTOR(m_mic_inp_desc, 
                                APP_USBD_AUDIO_INPUT_TERMINAL_DSC(
                                    1,                                     /* Terminal ID */
                                    APP_USBD_AUDIO_TERMINAL_IN_MICROPHONE, /* Terminal type */
                                    2,                                     /* Number of channels */
                                    MIC_TERMINAL_CH_CONFIG())              /* Channels config */
                                );

/**
 * @brief   Audio class output terminal descriptor
 */
APP_USBD_AUDIO_OUTPUT_DESCRIPTOR(m_mic_out_desc, 
                                 APP_USBD_AUDIO_OUTPUT_TERMINAL_DSC(
                                    3,                                     /* Terminal ID */
                                    APP_USBD_AUDIO_TERMINAL_USB_STREAMING, /* Terminal type */
                                    2)                                     /* Source ID */
                                );

/**
 * @brief   Audio class feature unit descriptor
 */
APP_USBD_AUDIO_FEATURE_DESCRIPTOR(m_mic_fea_desc, 
                                  APP_USBD_AUDIO_FEATURE_UNIT_DSC(
                                    2,                      /* Unit ID */
                                    1,                      /* Source ID */
                                    MIC_FEATURE_CONTROLS()) /* List of controls */
                                 );

/* Interfaces lists */

/**
 * @brief Interfaces list passed to @ref APP_USBD_AUDIO_GLOBAL_DEF
 */
#define MIC_INTERFACES_CONFIG() APP_USBD_AUDIO_CONFIG_IN(2, 3)

/*lint -save -e26 -e64 -e123 -e505 -e651*/

/**
 * @brief Microphone Audio class instance
 */
APP_USBD_AUDIO_GLOBAL_DEF(m_app_audio_microphone,
                          MIC_INTERFACES_CONFIG(),
                          mic_audio_user_ev_handler,
                          &m_mic_form_desc,
                          &m_mic_inp_desc,
                          &m_mic_out_desc,
                          &m_mic_fea_desc,
                          0,
                          APP_USBD_AUDIO_AS_IFACE_FORMAT_PCM,
                          192,
                          APP_USBD_AUDIO_SUBCLASS_AUDIOSTREAMING
);


/*lint -restore*/

/**
 * @brief Internal audio temporary buffer
 */
//static int16_t  m_temp_buffer[2 * BUFFER_SIZE];



/**
 * @brief The size of last received block from the microphone
 */
//static size_t m_temp_buffer_size;

/**
 * @brief Actual microphone mute state
 */
static uint8_t  m_mute_mic;

/**
 * @brief Actual microphone sampling frequency
 */
static uint32_t m_freq_mic;

/**
 * @brief Audio class specific request handle (microphone)
 */
static void mic_audio_user_class_req(app_usbd_class_inst_t const * p_inst)
{
    app_usbd_audio_t const * p_audio = app_usbd_audio_class_get(p_inst);
    app_usbd_audio_req_t * p_req = app_usbd_audio_class_request_get(p_audio);

    UNUSED_VARIABLE(m_mute_mic);
    UNUSED_VARIABLE(m_freq_mic);

    switch (p_req->req_target)
    {
        case APP_USBD_AUDIO_CLASS_REQ_IN:

            if (p_req->req_type == APP_USBD_AUDIO_REQ_SET_CUR)
            {
                //Only mute control is defined
                p_req->payload[0] = m_mute_mic;
            }

            break;
        case APP_USBD_AUDIO_CLASS_REQ_OUT:

            if (p_req->req_type == APP_USBD_AUDIO_REQ_SET_CUR)
            {
                //Only mute control is defined
                m_mute_mic = p_req->payload[0];
            }

            break;
        case APP_USBD_AUDIO_EP_REQ_IN:
            break;
        case APP_USBD_AUDIO_EP_REQ_OUT:

            if (p_req->req_type == APP_USBD_AUDIO_REQ_SET_CUR)
            {
                //Only set frequency is supported
                m_freq_mic = uint24_decode(p_req->payload);
            }

            break;
        default:
            break;
    }
}

/**
 * @brief User event handler @ref app_usbd_audio_user_ev_handler_t (microphone)
 */
static void mic_audio_user_ev_handler(app_usbd_class_inst_t const * p_inst,
                                      app_usbd_audio_user_event_t   event)
{
    app_usbd_audio_t const * p_audio = app_usbd_audio_class_get(p_inst);
    UNUSED_VARIABLE(p_audio);

    switch (event)
    {
        case APP_USBD_AUDIO_USER_EVT_CLASS_REQ:
            mic_audio_user_class_req(p_inst);
            break;
        case APP_USBD_AUDIO_USER_EVT_TX_DONE:
        {
        	usb_tx_done = 1;
            bsp_board_led_invert(LED_AUDIO_TX);
            break;
        }
        default:
            break;
    }
}

/**
 * @brief USBD library specific event handler.
 *
 * @param event     USBD library event.
 */
static void usbd_user_ev_handler(app_usbd_event_type_t event)
{
    switch (event)
    {
        case APP_USBD_EVT_DRV_SOF:
            // make sure we're in proper state (configured)
            if (APP_USBD_STATE_Configured != app_usbd_core_state_get()) break;

            // are we in the middle of a transfer ?
            if (!usb_tx_done) break;

            // do we have a buffer ready for transfer ?
            if (audio_buffer_txed) break;

        	// transfer the off-line buffer to the USB device now
            usb_tx_done = 0;
            ret_code_t ret = app_usbd_audio_class_tx_start(&m_app_audio_microphone.base,
        			pdm_buffer[pdm_buffer_i == 0 ? 1 : 0], NRFX_PDM_BUFFER_SIZE);

            // haven't started transferring -> wont reach TX done
            if (ret != NRF_SUCCESS) {
            	usb_tx_done = 1;
            	NRF_LOG_INFO("APP_USBD_EVT_DRV_SOF -> app_usbd_audio_class_tx_start = %d", ret);
            }

            // this buffer is being transferred
            else audio_buffer_txed = 1;

            break;
        case APP_USBD_EVT_DRV_SUSPEND:
            bsp_board_leds_off();
            break;
        case APP_USBD_EVT_DRV_RESUME:
            bsp_board_led_on(LED_USB_RESUME);
            break;
        case APP_USBD_EVT_STARTED:
            bsp_board_led_on(LED_USB_START);

            // start PDM
            APP_ERROR_CHECK(nrfx_pdm_start());

            break;
        case APP_USBD_EVT_STOPPED:
        	// start PDM
        	APP_ERROR_CHECK(nrfx_pdm_stop());

            app_usbd_disable();
            bsp_board_leds_off();
            break;
        case APP_USBD_EVT_POWER_DETECTED:
            NRF_LOG_INFO("USB power detected");

            if (!nrf_drv_usbd_is_enabled())
            {
                app_usbd_enable();
            }
            break;
        case APP_USBD_EVT_POWER_REMOVED:
            NRF_LOG_INFO("USB power removed");
            app_usbd_stop();
            break;
        case APP_USBD_EVT_POWER_READY:
            NRF_LOG_INFO("USB ready");
            app_usbd_start();
            break;
        default:
            break;
    }
}

static void nrfx_pdm_event_handler(nrfx_pdm_evt_t const *const p_evt) {
	/*NRF_LOG_INFO("nrfx_pdm_event_handler(buffer_released:%d buffer_requested:%d error:%d)",
			p_evt->buffer_released == NULL ? 0 : 1,
			p_evt->buffer_requested ? 1 : 0,
			p_evt->error);*/

	// we have an error ?
	if (p_evt->error != NRFX_PDM_NO_ERROR) {
		NRF_LOG_INFO("PDM error event: %d", p_evt->error);
		return;
	}

	// got a buffer with data ?
	if (p_evt->buffer_released != NULL) {
		// we have buffer ready
		audio_buffer_txed = 0;

		// do we have space to write ?
		if (flash_addr + NRFX_PDM_BUFFER_SIZE*2 < FSTORAGE_END_ADDR) {
			if (fstorage_write_done /*nrf_fstorage_is_busy(&fstorage_instance)*/) {
				// write to flash
				fstorage_write_done = 0;
				ret_code_t rc = nrf_fstorage_write(
					&fstorage_instance,     /* The instance to use. */
					flash_addr,             /* The address in flash where to store the data. */
					p_evt->buffer_released, /* A pointer to the data. */
					NRFX_PDM_BUFFER_SIZE*2, /* Lenght of the data, in bytes. */
					NULL                    /* Optional parameter, backend-dependent. */
				);

				if (rc != NRF_SUCCESS)
					NRF_LOG_ERROR("nrf_fstorage_write() = %d", rc);
			} else NRF_LOG_ERROR("fstorage busy -> dumping buffer");
		}
	}

	// expected to set a new buffer ?
	if (p_evt->buffer_requested) {
		NRF_LOG_INFO("PDM buffer requested");

		// update buffer index
		pdm_buffer_i = pdm_buffer_i == 0 ? 1 : 0;

		// set new buffer
		nrfx_pdm_buffer_set(pdm_buffer[pdm_buffer_i], NRFX_PDM_BUFFER_SIZE);
	}
}

void fstorage_callback(nrf_fstorage_evt_t * p_evt) {
	if (p_evt->result != NRF_SUCCESS) {
		NRF_LOG_INFO("--> Event received: ERROR while executing an fstorage operation.");
		return;
	}

	switch (p_evt->id) {
	case NRF_FSTORAGE_EVT_WRITE_RESULT:
		NRF_LOG_INFO("--> Event received: wrote %d bytes at address 0x%x.", p_evt->len, p_evt->addr);
		flash_addr += NRFX_PDM_BUFFER_SIZE*2;
		fstorage_write_done = 1;
		break;

	default:
		break;
	}
}

int main(void)
{
    ret_code_t ret;
    static const app_usbd_config_t usbd_config = {
        .ev_state_proc = usbd_user_ev_handler,
        .enable_sof = true
    };

    ret = NRF_LOG_INIT(NULL);
    APP_ERROR_CHECK(ret);
    NRF_LOG_DEFAULT_BACKENDS_INIT();

    ret = nrf_drv_clock_init();
    APP_ERROR_CHECK(ret);

    ret = nrf_fstorage_init(
		&fstorage_instance, /* You fstorage instance, previously defined. */
		&nrf_fstorage_nvmc, /* Name of the backend. */
		NULL                /* Optional parameter, backend-dependant. */
	);
    APP_ERROR_CHECK(ret);

    nrfx_pdm_config_t config = NRFX_PDM_DEFAULT_CONFIG(ARDUINO_SCL_PIN, ARDUINO_SDA_PIN);
    ret = nrfx_pdm_init(&config, nrfx_pdm_event_handler);
    APP_ERROR_CHECK(ret);

    NRF_LOG_INFO("USBD audio example started.");

    // Initialize LEDs and buttons
    bsp_board_init(BSP_INIT_LEDS | BSP_INIT_BUTTONS);

    ret = app_usbd_init(&usbd_config);
    APP_ERROR_CHECK(ret);

    app_usbd_class_inst_t const * class_inst_mic =
        app_usbd_audio_class_inst_get(&m_app_audio_microphone);
    ret = app_usbd_class_append(class_inst_mic);
    APP_ERROR_CHECK(ret);

    if (USBD_POWER_DETECTION)
    {
        ret = app_usbd_power_events_enable();
        APP_ERROR_CHECK(ret);
    }
    else
    {
        NRF_LOG_INFO("No USB power detection enabled\r\nStarting USB now");

        app_usbd_enable();
        app_usbd_start();
    }

    NRF_LOG_INFO("Entering main loop");

    while (true)
    {
        while (app_usbd_event_queue_process())
        {
            /* Nothing to do */
        }

        UNUSED_RETURN_VALUE(NRF_LOG_PROCESS());
        /* Sleep CPU only if there was no interrupt since last loop processing */
        __WFE();
    }
}

/** @} */

RE: Passing data from PDM callback to USB audio device

Kenneth — Tue, 17 Sep 2019 18:58:51 GMT

I assume that app_usbd_audio_class_tx_start() return NRF_ERROR_INVALID_STATE because the USB driver is not in active state (SUSTATE_ACTIVE). I would have started looking at why the driver haven't transitioned to active state (compared to default example for instance \nRF5_SDK_15.3.0_59ac345\examples\peripheral\usbd_audio).