Edge Impulse Wrapper - without additional buffer

Hi,

we are currently trying to implement an edge impulse model (keyword spotting, 600 ms window @ 16kHz) on a nRF52832.

The problem is that the SoC's RAM is already very full and barely fits our audio slab buffer, the library itself and the stuff that's already on it.

It seems like the wrapper requires an additional internal buffer to hold the samples. This unfortunately doesn't fit and we would like to point directly to the audio buffer that already contains our raw samples. 

I would be grateful if you could give us a hint on how to achieve that.

Thank you very much.

Edit 1: What I probably have to do is to drastically reduce the "audio slab" size and use the wrapper's buffer to accumulate the data using the "ei_wrapper_add_data()" function. 
What creates a problem then, however, is that the wrapper uses a float array. I have space for my 600 ms window filled with int16 samples but not for a float32 array of the same length :/

Edit 2: It seems like the ei_wrapper_add_data() function only accepts data inputs of the window length or larger. If I am not mistaken, that means that I need space for at least 2x the window length (which I don't have). For me only the following scenarios would work:

1. Small memory slab for the input audio that gradually fills the wrapper's buffer. When full -> run classifier on buffer data
2. Using only the audio data slab of total size = window size. Wait until slab is full -> run classifier on slab data

I don't need any continuous streaming functionality right now. Therefore "Filling buffer -> running inference -> filling buffer -> ...." would work for me.

  • Hi,

    The nRF52832 is as you say rather limited w.r.t flash and RAM so it is, as you say, required with some action to optimize either the load or the general footprint. The suggestions you make seems to be sound, albeit being brought down yet again by the available flash/RAM on the device.

    I'm curious is the SoC choice something that comes from the scope of your project, i.e that you want to create this edge impulse application on a RAM limited device, or due to the availability of no other boards?

    Kind regards,
    Andreas

  • Hi,
    we have a custom development board available right now that has not been designed with AI applications in mind. Therefore the SoC is under-dimensioned for many tasks that include DSP+NN processing. However, since we need to work with this particular board right now, it would be great if we could still get the edge impulse running on it somehow. 

    Right now I am trying to change the wrapper to 

    1. work with int16_t instead of floats
    2. Constructing the necessary input signal to the run_classifier function manually and calling it directly. 

    But I don't know how promising this approach is.

    Best regards, Nordix

  • I see, that makes sense.

    I will check in with my colleagues and see they have any other thoughts regarding suggestions you could try investigate, but my thoughts are that your angle of attack seems sane. As long as the changes you make, such as modifying ei_wrapper_add_data to work with int16 instead of float, does not propagate into the rest of the api and/or library in any unexpected ways. I can't guarantee that it will work, but I can't see why it should not as of now.

    I'll let you know about a status later this week w.r.t input from the mentioned discussion.

    Kind regards,
    Andreas

  • It's also worth mentioning that it may be worth investigating any suggestions/comments on the Zephyr discord forums as well as Edge Impulse forums as well while waiting for some input from us.

    Kind regards,
    Andreas

  • Thank you. I opened a topic in the Edge Impulse forum too:
    https://forum.edgeimpulse.com/t/c-library-input-datatype-int16-float/11792/3
    Looking forward to hearing back from you/your team!

Related