.svg)
The Thingy:53 is our newest prototyping platform based on the nRF5340 SoC and it comes preprogrammed with firmware developed by Edge Impulse. This firmware supports all Edge Impulse features, including ingestion, remote management, and inferencing.
With the launch of the Thingy:53 there is also the nRF Edge Impulse app for Android and iOS which allows you to upload raw sensor data via your mobile device to the cloud-based Edge Impulse Studio and deploy fully trained ML models to the Nordic Thingy:53 over Bluetooth® Low Energy (LE).
Edge Impulse offers a good starting guide on how to use the nRF Edge Impulse app.
Machine Learning
Machine learning (ML) is a branch of artificial intelligence (AI) that enables machines and systems to learn and improve from patterns and structures in the data on its own , i.e. the ML algorithms learn autonomously without human intervention or assistance and adjust actions accordingly.
ML has shown useful in many fields such as monitoring industrial machines and alerting the machine operator when an anomaly is detected. ML can also be used for predictive purposes such as predicting faulty behavior before it occurs with the goal of avoiding the occurrence in the first place. The list of applications ML can be useful for is long and includes energy optimization, medicine, finance, and many more.
Edge Impulse offers embedded machine learning designed for embedded devices with the use of TinyML. This allows performing machine learning on embedded IoT devices, analyzing sensor data at extremely low power, typically in the mW range, and therefore enabling a variety of always-on use-cases targeting battery-operated devices.
Links: Edge Impulse: What is embedded machine learning anyway?
Testing the preprogrammed application on Thingy:53
When opening the nRF Edge Impulse app for the first time you are presented with an Edge Impulse login. A free Edge Impulse account is required to connect the app with Edge Impulse Studio. This account can be created from the nRF Edge Impulse app or at www.edgeimpulse.com.
From here you can either choose an existing project or create a new one. The Edge Impulse team have collected several open projects on their web page which you can clone to your dashboard for testing.
Once connected, click on Data Acquisition and Record New Data. This will allow you to record the data that you want to use depending on the sensor.
During the sensor sampling, or data collecting stage, I would recommend looking at the tutorials and guides that the Edge Impulse team offers. They have an excellent explanation on how to generate your first model or impulse.
Links:
Edge Impulse YouTube Tutorial playlist
Edge Impulse Getting started documentation
Edge Impulse Machine Learning projects
Edge Impulse in nRF Connect SDK
After following the getting started guide and tutorials offered by Edge Impulse you should know how to prepare and deploy the ML model, and this model can then be integrated into any application running the Edge Impulse Wrapper library. The library source files can be found under nrf\lib\edge_impulse in the SDK folder.
The complete machine learning model can be added to your application by configuring the build system to download the model directly (as a C++ library source), as described in Using Edge Impulse with nRF Connectk SDK. This will download the model to the build folder of your application, e.g. build\edge_impulse, and will include among other files, build\edge_impulse\src\edge_impulse_project\model-parameters\model_metadata.h which includes the labels and other important used in the model.
Samples:
- Edge Impulse: Data forwarder - demonstrates how the library works.
- Edge Impulse: Wrapper - demonstrates the Edge Impulse data forwarder protocol
Applications:
The nRF Machine Learning application offers full support for the Thingy:53. This is an out-of-the-box reference design including the Edge Impulse Wrapper library and the Edge Impulse Data Forwarder protocol that allows you to collect sensor data in the Edge Impulse Studio.
When building the application for the Thingy:53, the application uses default sensor data from the accelerometer ADXL362 which is used in the NCS hardware accelerometer machine learning model, a public Edge Impulse project.
The application can switch between data forwarding or running the machine learning model by long-pressing the user-programmable button (SW3). When the device is in data forwarding mode, the red LED will blink at a high rate.
In order to use the data forwarder protocol in the nRF Machine Learning application, you need to install the Edge Impulse CLI on your computer. This includes among others the Edge Impulse data forwarder which can relay data from any device over serial to Edge Impulse. The Thingy:53 uses Nordic UART Service (NUS) to forward data to a device running the Bluetooth: Central UART sample. This is explained more detailed in the Testing with Thingy devices in nRF Machine Learning chapter.
The nRF Machine Learning application uses the CAF: Sensor manager module from the Common Application Framework (CAF) to ensure a consistent event-based architecture in your application. CAF is a set of ready-for-use modules and events built on top of Application Event Manager.
Links:
nRF Connect SDK - Using Edge Impulse with the nRF Connect SDK
nRF Connect SDK - Edge Impulse wrapper library
Using the Edge Impulse wrapper
To test the Edge Impulse wrapper sample on the Thingy:53 you will need an external debug probe in order to use RTT for debug output.
The standard output from this sample uses data from a pre-trained machine learning model based on input data representing a sine wave. The sample does not use any sensors.
* Booting Zephyr OS build v2.6.99-ncs1-14-g15be5e615498 * Machine learning model sampling frequency: 52 Labels assigned by the model: - idle - sine - triangle Prediction started... Classification results ====================== Value: 1.00 Label: sine Value: 0.00 Label: triangle Value: 0.00 Label: idle Anomaly: -0.07
To test another machine learning model, and to verify the model, you can edit the CONFIG_EDGE_IMPULSE_URI in the prj.conf file of your nRF Connect SDK project, as described in Downloading model directly from Edge Impulse studio. Make sure to edit XYZ to the correct Project ID from Edge Impulse Studio.
For this example, I have cloned the Edge Impulse Tutorial: Responding to your voice to a private Edge Impulse account.
And if you are using a private Edge Impulse project, then make sure to add EI_API_KEY_HEADER CMake option in the correct format i.e.
-DEI_API_KEY_HEADER="x-api-key:[ei api key]
This is also described in step 2 of Downloading from a private project.
Depending on the size of the machine learning model, you might receive an assert during build. The assert refers to edge impulse library and states: BUILD_ASSERT(DATA_BUFFER_SIZE > INPUT_WINDOW_SIZE);
This indicates that you need to configure the buffer size of the Edge Impulse wrapper i.e. CONFIG_EI_WRAPPER_DATA_BUF_SIZE. The INPUT_WINDOW_SIZE is defined by
#define EI_CLASSIFIER_RAW_SAMPLE_COUNT 16000 #define EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME 1 #define EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE (EI_CLASSIFIER_RAW_SAMPLE_COUNT * EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME)
Based on this information you can then configure the buffer to e.g. 16384
CONFIG_EI_WRAPPER_DATA_BUF_SIZE=16384
00> *** Booting Zephyr OS build v3.0.99-ncs1 *** 00> Machine learning model sampling frequency: 16000 00> Labels assigned by the model: 00> - helloworld 00> - noise 00> - unknown 00> 00> Prediction started... 00> 00> Classification results 00> ====================== 00> Value: 0.99 Label: helloworld 00> Value: 0.01 Label: unknown 00> Value: 0.00 Label: noise
Technical information
The Thingy:53 integrates the nRF5340 System on Chip (SoC), supporting Bluetooth Low Energy, 802.15.4, a Near Field Communication (NFC) passive tag, and the nRF21540 RF Front-End Module (FEM) for extended range. In addition, it has the nPM1100 Power Management IC (PMIC) for power management and battery charging of the 1350mA Li-Po battery.
The NFC in Nordic Thingy:53 operates as a passive tag (i.e. it does not feature a reader function).
There are two 2.4GHz antennae, one connected to the nRF5340 through an RF switch for TX output power up to +3dBm, while the second connects to the nRF21540 RF FEM, with TX output power up to +20dBm.
Further recommended reading
If you want to get more familiar with the Thingy:53 and its components, have a look at our documentation
From our Infocenter,
From our Get Connected Blog,
- Cellular IoT: A more intelligent way to combat water pollution
- Powerful wearables herald new dawn for healthcare
- Tiny machine learning now more accessible for developers
From the nRF Connect SDK (NCS) have a look at the Thingy:53 Getting Started guide and the Developing with Thingy:53, where the latter includes a list of compatible applications and samples for Thingy:53