Trouble with nrf_802154_transmit_raw after SDK upgrade

Hi,

I just noticed that there's another issue with my project after upgrading from SDK 2.3.0 to 3.1.1. 

I am using the 802.15.4 radio for low level packet communications with nrf_802154_transmit_raw() to send packets. I overrode the weak definitions of nrf_802154_received_raw(), nrf_802154_transmitted_raw(), and nrf_802154_transmit_failed() to handle packet reception and TX completion. See attached radio_interface.c file.

After the upgrade I noticed that every time I transmit, I get a "Transmit failure" message in my log. So my call to nrf_802154_transmit_raw(txdata, NULL) is returning false every time. I confirmed that this was not happening before the SDK upgrade.

I am still getting calls to nrf_802154_transmitted_raw() for every attempted transmit, and I am generally not getting calls to nrf_802154_transmit_failed(). So it seems like maybe my transmits are working even though the transmit call returns a fail.

I just tried sending a reset command to one of my wireless nodes to see if transmit packets are actually working, and it does look like they are - though apparently something went off the rails when that device replied, as I got an "Illegal use of the EPSR" usage failure and my processor rebooted.

Note that this is on a Fanstel module with a FEM, so I'm using MPSL because I believe it's required for FEM control. I attached my prj.conf in case that's helpful.

I tried comparing my project to the only sample I can find that uses the raw interface, but I haven't found the problem yet: \ncs\v3.1.1\nrf\samples\peripheral\802154_phy_test\

Thanks,

Glen

Log snippet showing the usage fault:

U/A to transmit to 837632Transmit failure
Forward
Rx Data
Data from 1067402[01:19:54.213,500] <err> os: ***** USAGE FAULT *****
[01:19:54.219,085] <err> os: Illegal use of the EPSR
[01:19:54.224,945] <err> os: r0/a1: 0x00000000 r1/a2: 0x00000020 r2/a3: 0
x00000001
[01:19:54.233,612] <err> os: r3/a4: 0x00000000 r12/ip: 0x20003640 r14/lr: 0
x0001e455
[01:19:54.242,309] <err> os: xpsr: 0x00000000
[01:19:54.247,528] <err> os: Faulting instruction address (r15/pc): 0x0001e468
[01:19:54.255,432] <err> os: >>> ZEPHYR FATAL ERROR 35: Unknown error on CPU 0
[01:19:54.263,336] <err> os: Current thread: 0x20003870 (main)
[01:19:54.269,836] <err> os: Halting system
à*** Booting nRF Connect SDK v3.1.1-e2a97fe2578a ***
*** Using Zephyr OS v4.1.99-ff8f0c579eeb ***

4111.prj.conf

//============================================================================
//  Title   : radio_interface
//  Desc    : Nordic radio 802.15.4 interface functions
//  2022-8-5   Glen     Created
//============================================================================
// Documentation on the 802.15.4 driver here:
// https://developer.nordicsemi.com/nRF_Connect_SDK/doc/1.9.1/nrfxlib/nrf_802154/README.html
//
// Sample code used for low level radio functions, rx pool, and how to receive
// are in the SDK here: \nRF_Connect\v2.0.0\nrf\samples\peripheral\802154_phy_test\src
//
//== Includes ================================================================
#include "radio_interface.h"
#include <string.h>
#include <zephyr/kernel.h>
#include <zephyr/device.h>
#include <zephyr/devicetree.h>
#include <zephyr/sys/byteorder.h>
#include <zephyr/sys/printk.h>
#include <hal/nrf_radio.h>
#include <string.h>
#include <stdio.h>
#include "radio.h"
#include "iomapping.h"

//== Definitions and Types ===================================================
//#define NO_TRANSMIT_FCC_VERSION                  1
#define RF_FCS_SIZE                                (2u)
#define POWER_OUTPUT_LEVEL_INTO_AMPLIFIER          0      // dBm
#define POWER_INCREASE_AMPLIFIER                   22     // dBm

typedef uint16_t ptt_pkt_len_t;

struct rf_rx_pkt_s {
	uint8_t *rf_buf; /**< pointer to buffer inside radio driver with a received packet */
	uint8_t *data; /**< pointer to payload */
	ptt_pkt_len_t length; /**< size of payload of received packet */
	int8_t rssi; /**< RSSI */
	uint8_t lqi; /**< LQI */
};

//== Global Variables ========================================================
static volatile bool m_tx_complete = true;

//== Function prototypes =====================================================

//============================================================================

//----------------------------------------------------------------------------
// Name  : radio_interface_config
// Desc  : Meant to reset the radio but there isn't an obvious nrf function to
//         that and it might not even make sense for an on-processor radio
// Ins   : none
// Outs  : none
//----------------------------------------------------------------------------
void radio_interface_config(void)
{
   // Clear the TX in progress flag
   m_tx_complete = true;
}

//----------------------------------------------------------------------------
// Name  : radio_interface_receive
// Desc  : Start the radio to receive packets
// Ins   : none
// Outs  : none
//----------------------------------------------------------------------------
void radio_interface_receive(void)
{
  nrf_802154_receive();
}

//----------------------------------------------------------------------------
// Name  : radio_interface_init
// Desc  : Configure the radio
// Ins   : none
// Outs  : none
//----------------------------------------------------------------------------
int radio_interface_init(void)
{
  /* nrf radio driver initialization */
   nrf_802154_init();

   uint8_t panId[] = { 0x00, 0x01 };
   nrf_802154_pan_id_set(&panId[0]);
   nrf_802154_channel_set(25);
   nrf_802154_promiscuous_set(false);
   return 0;
}

//----------------------------------------------------------------------------
// Name  : radio_interface_channel_set
// Desc  : Send the channel to the radio driver
// Ins   : none
// Outs  : none
//----------------------------------------------------------------------------
void radio_interface_channel_set(uint8_t channel)
{
   nrf_802154_channel_set(channel);
}

//----------------------------------------------------------------------------
// Name  : nrf_802154_received_raw
// Desc  : Received a packet from the nrf radio. Save off needed data and call 
//         rf_process_rx_packets to handle it
// Ins   : none
// Outs  : none
//----------------------------------------------------------------------------
void nrf_802154_received_raw(uint8_t *data, int8_t power, uint8_t lqi)
{
	struct rf_rx_pkt_s rx_pkt;
	bool pkt_placed = false;

	if (data == NULL)
   {
      return;
   }
   
   rx_pkt.rf_buf = data;
   rx_pkt.data = &data[1];
   rx_pkt.length = data[0] - RF_FCS_SIZE;

   rx_pkt.lqi = lqi; // It looks like we don't need these ED conversions after all
   pkt_placed = true;

   // Convert the RSSI from dBm to a custom scale
   rx_pkt.rssi = radio_interface_signal_strength_lookup(power);

   radio_interrupt(true, false, rx_pkt.data, rx_pkt.length, rx_pkt.rssi, rx_pkt.lqi, 0);
   nrf_802154_buffer_free_raw(rx_pkt.rf_buf);
}

//----------------------------------------------------------------------------
// Name  : nrf_802154_transmitted_raw
// Desc  : Override the WEAK defined library "tx done" function so that I can
//         add a call to the radio_interrupt in radio.c for status tracking
//         If ACK was requested for the transmitted frame, this function is 
//         called after a proper ACK is received. If ACK was not requested, 
//         this function is called just after transmission has ended.
// Ins   : none
// Outs  : none
//----------------------------------------------------------------------------
void nrf_802154_transmitted_raw(uint8_t * p_frame, const nrf_802154_transmit_done_metadata_t * p_metadata)
{
    (void)p_frame;

    uint8_t * p_ack = p_metadata->data.transmitted.p_ack;

    if (p_ack != NULL)
    {
        nrf_802154_buffer_free_raw(p_ack);
    }

    radio_interrupt(false, true, NULL, 0, 0, 0, 0);   // Call the radio TX complete interrupt, no failure
    //printk("TXDONE m_tx_complete = true\r\n");
    m_tx_complete = true;
}

//----------------------------------------------------------------------------
// Name  : nrf_802154_transmit_failed
// Desc  : Notifies that a frame was not transmitted due to a busy channel.
// Ins   : p_frame      Pointer to a buffer that contains PHR and PSDU of the frame that was not transmitted
// Ins   : error        Reason of the failure.
// Ins   : p_metadata   Pointer to a metadata structure describing frame passed in @p p_frame.
// Outs  : none
//----------------------------------------------------------------------------
void nrf_802154_transmit_failed(uint8_t * p_frame, nrf_802154_tx_error_t error, const nrf_802154_transmit_done_metadata_t * p_metadata)
{
   radio_interrupt(false, true, NULL, 0, 0, 0, 1);   // Call the radio TX complete interrupt, with a failure
   m_tx_complete = true;
   //printk("TXFAIL m_tx_complete = true\r\n");
   printk("\r\nrf_802154_transmit_failed\r\n");
}

//----------------------------------------------------------------------------
// Name  : radio_interface_transmit_packet
// Desc  : Send a packet out
// Ins   : A pointer to the data to send
// Outs  : none
//----------------------------------------------------------------------------
void radio_interface_transmit_packet(uint8_t *txdata)
{
   bool txstatus; 

   // The library is aware of the gain added by the PA, so you have to set the final output
   // power level you want, not just the level you want to go into the amplifier like it used to be
   nrf_802154_tx_power_set(POWER_OUTPUT_LEVEL_INTO_AMPLIFIER + POWER_INCREASE_AMPLIFIER);

   //printk("PRE-TX Set m_tx_complete false, Was %d\r\n", m_tx_complete);
   m_tx_complete = false;  // Set a flag that will be cleared when the transmission is either complete (plus acknowledgment) or it has failed      

   txstatus = nrf_802154_transmit_raw(txdata, NULL);

   if (txstatus == false)  // Failure to transmit 
   {
      printk("\r\nTransmit failure\r\n");
   }
}


//----------------------------------------------------------------------------
// Name  : radio_interface_get_tx_busy
// Desc  : Returns 1 if the radio has not yet completed its transmit (including receiving the hardware ACK if requested)
// Ins   : none
// Outs  : 1 for busy, 0 for not busy
//----------------------------------------------------------------------------
uint8_t radio_interface_get_tx_busy(void)
{
   if (m_tx_complete == false)
   {
      return 1;
   }
   else
   {
      return 0;
   }
}

Parents
  • Hi Glen,

    I just noticed that there's another issue with my project after upgrading from SDK 2.3.0 to 3.1.1. 

    That is a bit of a leap. I would assume there to be many API changes between those versions. Though the only one I happen to find that might be relevant is the return type being changed as mentioned here, though that might not count for NCS 3.1.1 as well, but its worth checking.

    You could also check this failing address: [01:19:54.247,528] <err> os: Faulting instruction address (r15/pc): 0x0001e468

    Might also be an idea to see if you are getting the same issue with the old weak definitions.

    Regards,

    Elfving

  • It looks like that change happened only with the latest 3.2 version.

    I asked Google's AI about the issue and it said "Between these versions, Nordic moved more logic into the MPSL (Multi-Protocol Service Layer). In 2.3.0, the 802.15.4 driver was "lighter" and did more direct register access. In 3.1.1, every radio action must be negotiated with the Arbiter."

    Is this true, and if so how does this arbiter work? I've tried putting in yields and sleeps thinking that maybe I just needed to let some driver level stuff run, but no luck so far.

    Is there some way for me to verify that the driver is ready for a new transmit packet or something I need to do do service the radio to let it finish whatever it's doing?

  • I made some progress - I had my DCDC regulator turned off and it seems like that might be needed to power everything properly. I turned that on in the board files and added CONFIG_NRFX_POWER=y

    Now I'm not getting the false results from the TX call, so that's great, but I am still occasionally hitting

    nrf_802154_transmit_failed() for maybe 10-15% of my packets.

    nrf_802154_state_get() does not exist that I can find, and on that 802154 driver log level I have it included but it's giving me a wiggly line saying "CONFIG_IEEE802154_DRIVER_LOG_LEVEL_DBG was assigned the value y, but got the value n. Missing dependencies:
    <choice IEEE802154_DRIVER_LOG_LEVEL_CHOICE>"  How can I fix that and get the rest of the way to the bottom of what's going on.

    Also, I still have to have 
    CONFIG_NRF_802154_TEMPERATURE_UPDATE=n or I get a kernel panic at startup. I don't need the temperature, but that seems like a red flag that something else might be up or that I'm missing something I need to get the driver working properly. Any thoughts on that?


  • Does explicitly enabling CONFIG_IEEE802154_DRIVER_LOG_LEVEL_DBG and CONFIG_IEEE802154_DRIVER_LOG_LEVEL_CHOICE help?

    Glen M said:
    Also, I still have to have CONFIG_NRF_802154_TEMPERATURE_UPDATE=n or I get a kernel panic at startup. I don't need the temperature, but that seems like a red flag that something else might be up or that I'm missing something I need to get the driver working properly. Any thoughts on that?

    Oh. Yeah I agree that sounds odd. Could you get me the log of that?

    I think I'll have to run this by the relevant R&D team.

    Regards,

    Elfving

  • That log level choice thing isn't something I can define - it requires digging into the driver kconfig stuff to figure out which thing has to be present for that item to be selected.

  • Regarding the temperature update config:

    I set CONFIG_NRF_802154_TEMPERATURE_UPDATE=y and this is what I get with just that one change:
    *** Booting nRF Connect SDK v3.1.1-e2a97fe2578a ***
    *** Using Zephyr OS v4.1.99-ff8f0c579eeb ***
    [00:00:00.009,796] <err> os: r0/a1: 0x00000004 r1/a2: 0x00000014 r2/a3: 0x020228d5
    [00:00:00.018,493] <err> os: r3/a4: 0x00000004 r12/ip: 0x0000000f r14/lr: 0x0002a775
    [00:00:00.027,160] <err> os: xpsr: 0x21000000
    [00:00:00.032,409] <err> os: Faulting instruction address (r15/pc): 0x0002a1a6
    [00:00:00.040,344] <err> os: >>> ZEPHYR FATAL ERROR 4: Kernel panic on CPU 0
    [00:00:00.048,095] <err> os: Current thread: 0x20001820 (sysworkq)
    [00:00:00.054,962] <err> os: Halting system

    I used addr2line to find the line of code where the panic happened and it's just in nrf_802154_assert_handler()

    ...arm-zephyr-eabi-addr2line.exe -e ...\trunk\Fanstel\build\Fanstel\zephyr\zephyr.elf -a 0x0002a1a6
    0x0002a1a6
    C:/ncs/v3.1.1/zephyr/modules/hal_nordic/nrf_802154/nrf_802154_assert_handler.c:24

    I recompiled for debug and set a breakpoint in the assert handler. Here is what I found in the call stack window:

  • I decided to capture the call stack for my other errors in case it helps you help me - so I turned  the temperature config back off. I found that I am still occasionally getting a false response to 

    nrf_802154_transmit_raw in addition to hitting 
    nrf_802154_transmit_failed - I said previously that I wasn't getting that first one anymore but apparently that was incorrect.
    Screenshots for nrf_802154_transmit_raw  returning false:
     
    Screenshots for nrf_802154_transmit_failed:

Reply
  • I decided to capture the call stack for my other errors in case it helps you help me - so I turned  the temperature config back off. I found that I am still occasionally getting a false response to 

    nrf_802154_transmit_raw in addition to hitting 
    nrf_802154_transmit_failed - I said previously that I wasn't getting that first one anymore but apparently that was incorrect.
    Screenshots for nrf_802154_transmit_raw  returning false:
     
    Screenshots for nrf_802154_transmit_failed:

Children
  • Hi Elfving,

    I've done some rewriting to make my code look more like the 802.15.4 phy test example, with workqueues etc. It seems like i'm getting fewer errors, and with improved debug prints I know more about the kinds of errors I'm getting (mostly NRF_802154_TX_ERROR_NO_ACK and NRF_802154_TX_ERROR_ABORTED 0x06).

    I'm letting it run overnight to see how many errors I get and to collect a new debug log, but in the meantime I've attached the two relevant radio files I'm using to send and receive data. Can you please code-review them and just see if it looks like I'm doing everything properly with regards to the radio?

    Thanks very much,

    Glen

    //============================================================================
    //  Title   : radio_interface
    //  Desc    : Nordic radio 802.15.4 interface functions
    //  2022-8-5   Glen     Created
    //============================================================================
    // Documentation on the 802.15.4 driver here:
    // https://developer.nordicsemi.com/nRF_Connect_SDK/doc/1.9.1/nrfxlib/nrf_802154/README.html
    //
    // Sample code used for low level radio functions, rx pool, and how to receive
    // are in the SDK here: \nRF_Connect\v2.0.0\nrf\samples\peripheral\802154_phy_test\src
    //
    //== Includes ================================================================
    #include "radio_interface.h"
    #include <string.h>
    #include <zephyr/kernel.h>
    #include <zephyr/sys/atomic.h>
    #include <zephyr/device.h>
    #include <zephyr/devicetree.h>
    #include <zephyr/sys/byteorder.h>
    #include <zephyr/sys/printk.h>
    #include <hal/nrf_radio.h>
    #include <string.h>
    #include <stdio.h>
    #include "radio.h"
    #include "iomapping.h"
    
    //== Definitions and Types ===================================================
    //#define NO_TRANSMIT_FCC_VERSION                  1
    #define RF_FCS_SIZE                                (2u)
    #define POWER_OUTPUT_LEVEL_INTO_AMPLIFIER          0      // dBm
    #define POWER_INCREASE_AMPLIFIER                   22     // dBm
    
    typedef uint16_t ptt_pkt_len_t;
    
    struct rf_rx_pkt_s {
    	uint8_t *rf_buf; /**< pointer to buffer inside radio driver with a received packet */
    	uint8_t *data; /**< pointer to payload */
    	ptt_pkt_len_t length; /**< size of payload of received packet */
    	int8_t rssi; /**< RSSI */
    	uint8_t lqi; /**< LQI */
    };
    
    #define RF_WORKQ_STACK_SIZE 2048
    #define RF_WORKQ_PRIORITY   5
    
    K_THREAD_STACK_DEFINE(rf_wq_stack, RF_WORKQ_STACK_SIZE);
    
    /* RX queue item (stores the driver buffer pointer until we process+free it) */
    struct rx_item {
        sys_snode_t node;
        uint8_t *rf_buf;     /* driver-owned buffer */
        uint8_t *data;       /* &rf_buf[1] */
        uint16_t length;     /* rf_buf[0] - FCS */
        int8_t rssi;
        uint8_t lqi;
    };
    
    typedef enum {
        TX_FAIL_NONE = 0,
        TX_FAIL_START,
        TX_FAIL_ON_AIR,
    } tx_fail_reason_t;
    
    //== Global Variables ========================================================
    static volatile tx_fail_reason_t tx_fail_reason;
    
    static uint8_t failure_reason;
    static struct k_work_q rf_wq;
    static struct k_work_delayable tx_start_work;
    
    /* Serialize RF operations (similar to ptt_rf lock) */
    static atomic_t rf_tx_in_flight;
    
    /* Stage TX frame to guarantee lifetime until TX done */
    static uint8_t tx_frame_buf[RADIO_MAX_PAYLOAD_LEN + 1]; /* length byte + payload */
    
    static sys_slist_t rx_list;
    static struct k_work rx_work;
    
    /* TX completion work */
    static struct k_work tx_done_work;
    static struct k_work tx_fail_work;
    
    
    static volatile bool m_tx_complete = true;
    uint8_t m_rssi_lookup[54] = 
    {
       1, 2, 5, 9, 13, 18, 23, 27, 32, 37, 43, 48, 53, 58, 63, 68, 
       73, 78, 83, 89, 95, 100, 107, 111, 117, 121, 125, 129, 133, 
       138, 143, 148, 153, 159, 165, 170, 176, 183, 188, 193, 198, 
       203, 207, 212, 216, 221, 225, 228, 233, 239, 245, 250, 253, 254
    };
    
    //== Function prototypes =====================================================
    uint8_t radio_interface_signal_strength_lookup(int8_t rssi);
    static void tx_start_handler(struct k_work *work);
    static void rx_work_handler(struct k_work *work);
    static void tx_done_handler(struct k_work *work);
    static void tx_fail_handler(struct k_work *work);
    
    //============================================================================
    
    //----------------------------------------------------------------------------
    // Name  : radio_interface_config
    // Desc  : Meant to reset the radio but there isn't an obvious nrf function to
    //         that and it might not even make sense for an on-processor radio
    // Ins   : none
    // Outs  : none
    //----------------------------------------------------------------------------
    void radio_interface_config(void)
    {
       // Clear the TX in progress flag
       //printk("Config m_tx_complete = true\r\n");
       m_tx_complete = true;
       //nrf_802154_deinit();
       //radio_interface_init();
    }
    
    //----------------------------------------------------------------------------
    // Name  : radio_interface_receive
    // Desc  : Start the radio to receive packets
    // Ins   : none
    // Outs  : none
    //----------------------------------------------------------------------------
    void radio_interface_receive(void)
    {
      nrf_802154_receive();
    }
    
    //----------------------------------------------------------------------------
    // Name  : radio_interface_init
    // Desc  : Configure the radio
    // Ins   : none
    // Outs  : none
    //----------------------------------------------------------------------------
    int radio_interface_init(void)
    {
       atomic_clear(&rf_tx_in_flight);
       sys_slist_init(&rx_list);
    
       k_work_queue_start(&rf_wq, rf_wq_stack,
                         K_THREAD_STACK_SIZEOF(rf_wq_stack),
                         RF_WORKQ_PRIORITY, NULL);
    
       k_work_init(&rx_work, rx_work_handler);
       k_work_init(&tx_done_work, tx_done_handler);
       k_work_init(&tx_fail_work, tx_fail_handler);
       k_work_init(&tx_start_work, tx_start_handler);
    
      /* nrf radio driver initialization */
       nrf_802154_init();
    
       uint8_t panId[] = { 0x00, 0x01 };
       nrf_802154_pan_id_set(&panId[0]);
       nrf_802154_channel_set(25);
       nrf_802154_promiscuous_set(false);
       
       return 0;
    }
    
    //----------------------------------------------------------------------------
    // Name  : radio_interface_channel_set
    // Desc  : Send the channel to the radio driver
    // Ins   : none
    // Outs  : none
    //----------------------------------------------------------------------------
    void radio_interface_channel_set(uint8_t channel)
    {
       nrf_802154_channel_set(channel);
    }
    
    //----------------------------------------------------------------------------
    // Name  : nrf_802154_received_raw
    // Desc  : Received a packet from the nrf radio. Save off needed data and call 
    //         rf_process_rx_packets to handle it
    // Ins   : none
    // Outs  : none
    //----------------------------------------------------------------------------
    /* Simple fixed pool to avoid malloc in callback */
    #define RX_ITEM_POOL_N 16
    static struct rx_item rx_pool[RX_ITEM_POOL_N];
    
    static struct rx_item *rx_item_alloc(void)
    {
        for (int i = 0; i < RX_ITEM_POOL_N; i++) 
        {
            if (rx_pool[i].rf_buf == NULL) 
            {
                return &rx_pool[i];
            }
        }
        return NULL;
    }
    
    static void rx_item_free(struct rx_item *it)
    {
        it->rf_buf = NULL;
        it->data = NULL;
        it->length = 0;
        it->rssi = 0;
        it->lqi = 0;
    }
    
    void nrf_802154_received_raw(uint8_t *data, int8_t power, uint8_t lqi)
    {
        if (data == NULL) 
        {
            return;
        }
    
        struct rx_item *it = rx_item_alloc();
        if (!it) 
        {
            /* No space: drop (phy_test does this too) */
            printk("FAILURE - RX buffer overrun\r\n");
            nrf_802154_buffer_free_raw(data);
            return;
        }
    
        it->rf_buf = data;
        it->data = &data[1];
        it->length = data[0] - RF_FCS_SIZE;
    
        /* RSSI conversion */
        it->rssi = radio_interface_signal_strength_lookup(power);
        it->lqi = lqi;
    
        /* enqueue and schedule work */
        sys_slist_append(&rx_list, &it->node);
        k_work_submit_to_queue(&rf_wq, &rx_work);
    }
    
    static void rx_work_handler(struct k_work *work)
    {
        while (1) 
        {
            sys_snode_t *n = sys_slist_get(&rx_list);
            if (!n) {
                break;
            }
    
            struct rx_item *it = CONTAINER_OF(n, struct rx_item, node);
    
            radio_interrupt(true, false, it->data, it->length, it->rssi, it->lqi, 0);
    
            nrf_802154_buffer_free_raw(it->rf_buf);
            rx_item_free(it);
        }
    }
    
    //----------------------------------------------------------------------------
    // Name  : nrf_802154_transmitted_raw
    // Desc  : Override the WEAK defined library "tx done" function so that I can
    //         add a call to the radio_interrupt in radio.c for status tracking
    //         If ACK was requested for the transmitted frame, this function is 
    //         called after a proper ACK is received. If ACK was not requested, 
    //         this function is called just after transmission has ended.
    // Ins   : none
    // Outs  : none
    //----------------------------------------------------------------------------
    void nrf_802154_transmitted_raw(uint8_t *p_frame,
                                   const nrf_802154_transmit_done_metadata_t *p_metadata)
    {
        (void)p_frame;
    
        uint8_t *p_ack = p_metadata->data.transmitted.p_ack;
        if (p_ack != NULL) {
            /* You can either free here (as you do now), or defer like phy_test.
               Freeing here is usually OK, but deferring is more consistent. */
            nrf_802154_buffer_free_raw(p_ack);
        }
    
        k_work_submit_to_queue(&rf_wq, &tx_done_work);
    }
    
    //----------------------------------------------------------------------------
    // Name  : nrf_802154_transmit_failed
    // Desc  : Notifies that a frame was not transmitted due to a busy channel.
    // Ins   : p_frame      Pointer to a buffer that contains PHR and PSDU of the frame that was not transmitted
    // Ins   : error        Reason of the failure.
    // Ins   : p_metadata   Pointer to a metadata structure describing frame passed in @p p_frame.
    // Outs  : none
    //----------------------------------------------------------------------------
    void nrf_802154_transmit_failed(uint8_t *p_frame, nrf_802154_tx_error_t error,
                                    const nrf_802154_transmit_done_metadata_t *p_metadata)
    {
        (void)p_frame;
        (void)error;
        (void)p_metadata;
    
        printk("Failure: TX_ON_AIR_FAIL: error=%d\r\n", error);
        tx_fail_reason = TX_FAIL_ON_AIR;
        k_work_submit_to_queue(&rf_wq, &tx_fail_work);
    }
    
    //----------------------------------------------------------------------------
    // Name  : radio_interface_transmit_packet
    // Desc  : Send a packet out
    // Ins   : A pointer to the data to send
    // Outs  : none
    //----------------------------------------------------------------------------
    void radio_interface_transmit_packet(uint8_t *txdata)
    {
    #ifndef NO_TRANSMIT_FCC_VERSION
        if (txdata == NULL) {
            return;
        }
    
        /* Prevent overlapping TX attempts */
        if (atomic_cas(&rf_tx_in_flight, 0, 1) == false) {
            /* Already busy */
            printk("FAILURE - TX request while busy\r\n");
            return;
        }
    
        /* Set power */
        nrf_802154_tx_power_set(POWER_OUTPUT_LEVEL_INTO_AMPLIFIER + POWER_INCREASE_AMPLIFIER);
    
        /* Stage TX buffer */
        uint16_t total_len = txdata[0] + 1;  /* length byte + PSDU */
        if (total_len > sizeof(tx_frame_buf)) 
        {
            atomic_clear(&rf_tx_in_flight);
            printk("FAILURE - TX frame too large\r\n");
            return;
        }
    
        memcpy(tx_frame_buf, txdata, total_len);
    
        /* Start TX in rf workqueue context */
        m_tx_complete = false;
    
        //k_work_submit_to_queue(&rf_wq, &tx_start_work);
        k_work_schedule_for_queue(&rf_wq, &tx_start_work, K_MSEC(1));
    #endif
    }
    
    //----------------------------------------------------------------------------
    // Name  : radio_interface_get_tx_busy
    // Desc  : Returns 1 if the radio has not yet completed its transmit (including receiving the hardware ACK if requested)
    // Ins   : none
    // Outs  : 1 for busy, 0 for not busy
    //----------------------------------------------------------------------------
    uint8_t radio_interface_get_tx_busy(void)
    {
       if (m_tx_complete == false)
       {
          return 1;
       }
       else
       {
          return 0;
       }
    }
    
    //----------------------------------------------------------------------------
    // Name  : radio_interface_signal_strength_lookup
    // Desc  : Looks up the RSSI value needed by this app from the RSSI/dBm value
    // Ins   : traditional RSSI in dBm
    // Outs  : converted RSSI value per the lookup table in the MRF24J20 datasheet lookup table 3-8.
    //----------------------------------------------------------------------------
    uint8_t radio_interface_signal_strength_lookup(int8_t rssi)
    {
       // After lookup, RSSI will be a value from 255 (great) to 0 (bad)
       // An RSSI of -90dBm or below is 0
       // An RSSI of -35dBm or above is 255
       // In-between lookup table
       uint8_t signal_strength_lookup;
       if (rssi <= -90 )
       {
          signal_strength_lookup = 0;
       }
       else if (rssi >= -35 )
       {
          signal_strength_lookup = 255;
       }
       else
       {  // It's somewhere in the middle. Use a lookup table
          signal_strength_lookup = m_rssi_lookup[rssi + 89];
       }
       return signal_strength_lookup;
    }
    
    // This function is called when we start to send an ACK. Just using it temporarily for amplifier control debug
    //void nrf_802154_tx_ack_started(const uint8_t * p_data)
    //{
    //   iomapping_set_tp2(0);      
    //}
    
    static void tx_start_handler(struct k_work *work)
    {
        bool ok;
    
        /* NOTE: we are already logically "locked" here via rf_tx_in_flight */
        const nrf_802154_transmit_metadata_t metadata = {
            .frame_props = NRF_802154_TRANSMITTED_FRAME_PROPS_DEFAULT_INIT,
            .cca = true,  /* set based on what you want */
        };
    
    //#warning DEBUG PRINT
    //printk("TX FCF bytes: %02x %02x\r\n", tx_frame_buf[1], tx_frame_buf[2]);
    //uint8_t len = tx_frame_buf[0];
    //uint16_t fcf = (uint16_t)tx_frame_buf[1] | ((uint16_t)tx_frame_buf[2] << 8);
    //uint8_t seq = tx_frame_buf[3];
    //bool ackreq = (fcf & (1u << 5)) != 0;   // 802.15.4 ACK request bit is bit 5 of FCF
    //printk("TX: len=%u fcf=0x%04x seq=%u ackreq(bit)=%u\r\n", len, fcf, seq, ackreq);
    
    
        ok = nrf_802154_transmit_raw(tx_frame_buf, &metadata);
    
        if (!ok) {
            /* Could not even start TX: complete as failure */
            tx_fail_reason = TX_FAIL_START;
            k_work_submit_to_queue(&rf_wq, &tx_fail_work);
        }
    }
    
    
    static void tx_done_handler(struct k_work *work)
    {
        radio_interrupt(false, true, NULL, 0, 0, 0, 0);
        m_tx_complete = true;
        atomic_clear(&rf_tx_in_flight);
    }
    
    static void tx_fail_handler(struct k_work *work)
    {
        radio_interrupt(false, true, NULL, 0, 0, 0, 1);
        m_tx_complete = true;
        atomic_clear(&rf_tx_in_flight);
        if (tx_fail_reason == TX_FAIL_START)
        {
          printk("\r\nFAILURE - Failed to start transmit\r\n");
        }
        else if (tx_fail_reason == TX_FAIL_ON_AIR)
        {
          //printk("\r\nFAILURE - Transmit failed on-air\r\n");      
        }
    }
    #if(0)
    void nrf_802154_tx_ack_started(const uint8_t *p_data)
    {
        printk("AUTOACK started\r\n");
    }
    
    void nrf_802154_tx_ack_finished(const uint8_t *p_data)
    {
        printk("AUTOACK finished\r\n");
    }
    #endif
    //============================================================================
    //  Title   : radio
    //  Desc    : Driver for Nordic nRF52840 802.15.4 radio
    //  2022-9-12   Glen     Created
    //============================================================================
    
    //== Includes ================================================================
    #include <zephyr/kernel.h>
    #include <zephyr/device.h>
    #include <zephyr/devicetree.h>
    #include <zephyr/sys/byteorder.h>
    #include <zephyr/sys/printk.h>
    #include <string.h> // memset()
    #include "iomapping.h"
    #include "radioAddress.h" // addr for radio
    #include "M_uart.h"
    #include "M_timer.h"
    #include "crc.h"
    #include "radio.h"
    #include <stdbool.h>
    #include "radio_interface.h"
    #include "tls.h"
    
    //== Definitions and Types ===================================================
    #define TRANSMIT_BUFFER_SIZE     256
    
    //== Global Variables ========================================================
    extern unsigned char flag_reset_radio;
    
    RADIO_STATUS volatile RadioStatus;                         // radio state
    uint8_t volatile RXBuffer[PACKET_BUFFERS][RX_BUFFER_SIZE]; // rx packet buffers
    PACKET Tx;                                                 // structures describing transmitted and received packets
    PACKET Rx;                                                 // structures describing transmitted and received packets
    uint8_t m_transmit_buffer[TRANSMIT_BUFFER_SIZE];
    
    //== Function prototypes =====================================================
    
    //============================================================================
    
    //----------------------------------------------------------------------------
    // Name  : timerTicksSince
    // Desc  : Return elapsed time in 256us increments
    //----------------------------------------------------------------------------
    uint64_t timerTicksSince(uint64_t savedTimer)
    {
       return (m_timer_get_radio_timer() - savedTimer);  
    }
    
    //----------------------------------------------------------------------------
    // Name  : readBytes
    // Desc  : This combines memcpy with incrementing the source point.  It copies 
    //         bytewise, allowing it to copy to/from unaligned addresses
    //----------------------------------------------------------------------------
    unsigned char *readBytes(unsigned char *dstPtr, unsigned char *srcPtr, unsigned int count)
    {
       while (count--)
       {
          *dstPtr++ = *srcPtr++;
       }
    
       return srcPtr;
    }
    
    //----------------------------------------------------------------------------
    // Name  : toTXfifo
    // Desc  : Writes count consecutive bytes from source into consecutive FIFO slots starting at "register".  
    // Ins   : none
    // Outs  : Returns next empty register #.
    //----------------------------------------------------------------------------
    uint8_t toTXfifo(uint16_t reg, uint8_t *source, uint8_t count)
    {
       uint8_t *ptr;
       size_t buf_size = sizeof(m_transmit_buffer);
    
       ptr = &m_transmit_buffer[0];
       ptr += reg;
    
       if ((size_t)reg > buf_size) 
       {
          printk("toTXfifo: reg=%u out of range (buf=%zu)\n", reg, buf_size);
       }
    
       if ((size_t)reg + (size_t)count > buf_size) 
       {
          printk("toTXfifo: overflow reg=%u count=%u (buf=%zu)\n", reg, count, buf_size);
       }
    
       if (source == NULL) 
       {
          printk("toTXfifo: source NULL (reg=%u, count=%u)\n", reg, count);
          return reg;
       }
    
       memcpy(ptr, source, count);
       return (count + reg);
    }
    
    //----------------------------------------------------------------------------
    // Name  : initRADIO
    // Desc  : Warm start radio hardware, tunes to Channel.  Takes about 0.37 ms on PIC32 at 20 MHz, 10 MHz SPI hardware clock 
    // Ins   : none
    // Outs  : On return, 0=no radio hardare, 1=radio is reset
    //----------------------------------------------------------------------------
    uint8_t initRADIO(void)
    {
       m_timer_init_radio_timer();
       RadioStatus.LastTXTriggerTick = 0;
       RadioStatus.ResetCount++;
    
       RadioStatus.TX_FAIL = 0;
       RadioStatus.TX_PENDING_ACK = 0; // not pending an ack after reset
       RadioStatus.SLEEPING = 0;       // radio is not sleeping
    
       // Reset the radio
       radio_interface_config();
    
       RadioSetAddress(RadioStatus.MyShortAddress, RadioStatus.MyLongAddress, RadioStatus.MyPANID);
       RadioSetChannel(RadioStatus.Channel); // tune to current radio channel
    
       RadioSetSleep(0);
    
       return 1;
    }
    
    //----------------------------------------------------------------------------
    // Name  : RadioInit
    // Desc  : Initialize the RadioStatus structure and then call the lower level radio init function
    // Ins   : none
    // Outs  : on return, 1=radio is setup, 0=there is no radio
    //----------------------------------------------------------------------------
    bool RadioInit(unsigned char chan, unsigned long address, unsigned int panid) // cold start radio init
    {
       bool radio;
    
       memset((void *)&RadioStatus, 0, sizeof(RadioStatus));
    
       RadioStatus.MyPANID = panid;
       RadioStatus.MyShortAddress = 0x0001; //???
       RadioStatus.MyLongAddress = address;
    
       RadioStatus.Channel = chan; // Channel 25 should not interfere with Wifi
    
       radio = initRADIO(); // init radio hardware, tune to RadioStatus.Channel
    
       return 0;
    }
    
    //----------------------------------------------------------------------------
    // Name  : RadioSetAddress
    // Desc  : Set short address and PANID
    // Ins   : Short 16-bit address, long 64-bit address, and PAN ID
    // Outs  : none
    //----------------------------------------------------------------------------
    void RadioSetAddress(uint16_t shortAddress, uint64_t longAddress, uint16_t panID)
    {
       nrf_802154_pan_id_set((const uint8_t *)(&panID));
       nrf_802154_extended_address_set((const uint8_t *)(&longAddress));
       nrf_802154_short_address_set((const uint8_t *)(&shortAddress));
    }
    
    //----------------------------------------------------------------------------
    // Name  : RadioSetChannel
    // Desc  : Set radio channel.  
    // Ins   : none
    // Outs  : Returns with success/fail flag.
    //----------------------------------------------------------------------------
    bool RadioSetChannel(uint8_t channel)
    {
       if (channel < 11 || channel > 26)
       {
          return false;
       }
    
    #if defined(ENABLE_PA_LNA) // Permitted band is 2400 to 2483.5 MHz.
       if (channel == 26)
       {                // Center Freq. is 2405+5(k-11) MHz, for k=channel 11 to 26
          return false; // max output is 100mW (USA)
       }
    #endif // rolloff is not steep enough to avoid 2483.5 from channel 26 center of 2480 MHz at full MB power
    
       RadioStatus.Channel = channel;
    
       // Push the channel to the radio
       radio_interface_channel_set(channel);
    
       // Start receive
       //radio_interface_receive();     // This happens in RadioSetSleep(0), which is called next
    
       return true;
    }
    
    //----------------------------------------------------------------------------
    // Name  : RadioSetSleep
    // Desc  : Put the RF transceiver into sleep or wake it up
    // Radio power, RADIOMB - ENABLE_PA_LNA on:
    //	RX:  	28.4 mA
    //	TX: 	65.8 mA (as fast as I can xmit; nominal peak 130 mA)
    //	Sleep:	0.245 mA (spec is 5 uA with 'sleep clock disabled'; setting register 0x211 to 0x01 doesn't seem to help)
    // Note that you can in practice turn the radio power off completely for short periods (with a MOSFET) and then do a warm start.
    // Ins   : 1 to put the radio to sleep, 0 to wake it up
    // Outs  : none
    //----------------------------------------------------------------------------
    void RadioSetSleep(uint8_t powerState)
    {
       if (powerState)
       {
          RadioStatus.SLEEPING = 1; // radio is sleeping
          nrf_802154_sleep();       // Put the radio to sleep
       }
       else
       {
          RadioStatus.SLEEPING = 0;
          nrf_802154_receive(); // Wake up the radio and put it back in receive mode
       }
    }
    
    //----------------------------------------------------------------------------
    // Name  : RadioEnergyDetect
    // Desc  : Do a single (128 us) energy scan on current channel.  Return RSSI.
    // Ins   : none
    // Outs  : none
    //----------------------------------------------------------------------------
    uint8_t RadioEnergyDetect(void)
    {   
       // This function is not currently called from anywhere, so it has not been updated
       // This can be done using the following functions: nrf_802154_rssi_measure_begin, then nrf_802154_rssi_last_get
       // Info here: https://infocenter.nordicsemi.com/index.jsp?topic=%2Fsdk_tz_v4.1.0%2Fgroup__nrf__802154__rssi.html
    #if (0)
       uint8_t RSSIcheck;
    
       RSSIcheck = lowRead(READ_BBREG6); // Read RSSIRDY
       while ((RSSIcheck & 0x01) != 0x01)
       {                                    // Wait until RSSIRDY goes to 1; this indicates result is ready
          RSSIcheck = lowRead(READ_BBREG6); // this takes max 8 symbol periods (16 uS each = 128 uS)
       }
    
       RSSIcheck = highRead(0x210); // read the RSSI
    
       lowWrite(WRITE_BBREG6, 0x40); // enable RSSI on received packets again after energy scan is finished
    
    #if defined(ENABLE_PA_LNA)
       lowWrite(WRITE_GPIO, 0);
       lowWrite(WRITE_GPIODIR, 0x00); // Set GPIO direction to INPUT
       highWrite(TESTMODE, 0x0F);     // setup for automatic PA/LNA control
    #endif
    
       return RSSIcheck;
    #else
       return 0;
    #endif
    }
    
    //----------------------------------------------------------------------------
    // Name  : RadioTXRaw
    // Desc  : Radio transmit function
    // Ins   : none
    // Outs  : none
    //----------------------------------------------------------------------------
    void RadioTXRaw(void)
    {
       uint8_t wReg; // radio write register (into TX FIFO starting at long addr 0)
       uint16_t startRegGoodPayload;
       uint8_t zeroval = 0;
    
       wReg = toTXfifo(0, &zeroval, 1);                                // Frame length goes here - inserted at the end
       wReg = toTXfifo(1, BYTEPTR(Tx.frameControl), 2);
       wReg = toTXfifo(wReg, BYTEPTR(Tx.frameNumber), 1);
    
       if (Tx.frameControl.dstAddrMode == SHORT_ADDR_FIELD)
       {                                                  // if a short dest addr is present
          wReg = toTXfifo(wReg, BYTEPTR(Tx.dstPANID), 2); // write dstPANID
          wReg = toTXfifo(wReg, BYTEPTR(Tx.dstAddr), 2);  // write short address
       }
       else if (Tx.frameControl.dstAddrMode == LONG_ADDR_FIELD)
       {                                                  // if a long dest addr is present
          wReg = toTXfifo(wReg, BYTEPTR(Tx.dstPANID), 2); // write dstPANID
          wReg = toTXfifo(wReg, BYTEPTR(Tx.dstAddr), 8);  // long addr
       }
    
       // now wReg is at start of source PANID (if present)
    
       if (Tx.frameControl.srcAddrMode != NO_ADDR_FIELD && // if source present
           Tx.frameControl.dstAddrMode != NO_ADDR_FIELD && // and dest present
           !Tx.frameControl.panIDcomp)
       {                                                  // and no PANID compression
          wReg = toTXfifo(wReg, BYTEPTR(Tx.srcPANID), 2); // then write src PANID
       }
    
       if (Tx.frameControl.srcAddrMode == SHORT_ADDR_FIELD)
       { // if a short src addr is present
          wReg = toTXfifo(wReg, BYTEPTR(Tx.srcAddr), 2);
       }
       else if (Tx.frameControl.srcAddrMode == LONG_ADDR_FIELD)
       { // if a long src addr is present
          wReg = toTXfifo(wReg, BYTEPTR(Tx.srcAddr), 8);
       }
    
       // now wReg is pointing to first wReg after header (m)
       startRegGoodPayload = wReg;
    
       wReg = toTXfifo(wReg, Tx.payload, Tx.payloadLength);
    
       if (tls_encrypt(wReg - 4, startRegGoodPayload, m_transmit_buffer) == FAIL)
       { // encrypts payload (but not CRC), 'wreg - 4' corresponds to CRC start address
          return;
       }
    
       // The Nordic radio wants the full frame length to be in the first byte of the packet
       wReg++;  // Include the length
       toTXfifo(0, &wReg, 1); 
    
       printk("Sending packet - ACK requested=%d\r\n", Tx.frameControl.ackRequest);
    
       radio_interface_transmit_packet(m_transmit_buffer);
    
       RadioStatus.TX_PENDING_ACK = Tx.frameControl.ackRequest;
       RadioStatus.LastTXTriggerTick = m_timer_get_radio_timer(); // record time (used to check for locked-up radio or PLL loss)
    }
    
    //----------------------------------------------------------------------------
    // Name  : RadioTXPacket
    // Desc  : Sends next packet from Tx.  Blocks for up to RADIO_TIMEOUT_TICKS if transmitter is
    // not ready (RadioStatus.TX_BUSY).  If you don't want to be blocked, don't call
    // ths until RadioStatus.TX_BUSY == 0.
    // This automatically sets frame number and source address for you
    // Ins   : none
    // Outs  : none
    //----------------------------------------------------------------------------
    void RadioTXPacket(void)
    {
       if (Tx.frameControl.srcAddrMode == SHORT_ADDR_FIELD)
       {
          Tx.srcAddr = RadioStatus.MyShortAddress;
       }
       else if (Tx.frameControl.srcAddrMode == LONG_ADDR_FIELD)
       {
          Tx.srcAddr = RadioStatus.MyLongAddress;
       }
    
       Tx.frameNumber = RadioStatus.IEEESeqNum++;
    
       while (radio_interface_get_tx_busy())
       { // If TX is busy, wait for it to clear (for a resaonable time)
          if (timerTicksSince(RadioStatus.LastTXTriggerTick) > RADIO_TIMEOUT_TICKS)
          {
             // if not ready in a resonable time
             initRADIO(); // reset radio hardware (stay on same channel) -- JS Removed this
          }
       }
    
       RadioTXRaw();
    }
    
    //----------------------------------------------------------------------------
    // Name  : RadioTXResult
    // Desc  : Returns status of last transmitted packe
    // Ins   : none
    // Outs  : TX_SUCCESS (1), TX_FAILED (2), or 0 = no result yet because TX busy
    //----------------------------------------------------------------------------
    uint8_t RadioTXResult(void)
    {
       if (radio_interface_get_tx_busy())
       { // if TX not done yet
          return TX_RESULT_BUSY;
       }
    
       return TX_RESULT_SUCCESS + RadioStatus.TX_FAIL; // 1=success, 2=fail
    }
    
    //----------------------------------------------------------------------------
    // Name  : RadioWaitTXResult
    // Desc  : Wait for the transmit to complete
    // Ins   : none
    // Outs  : Returns TX_RESULT_SUCCESS or TX_RESULT_FAILED.  Waits up to RADIO_TIMEOUT_TICKS.
    //----------------------------------------------------------------------------
    uint8_t RadioWaitTXResult(void)
    {
       while (radio_interface_get_tx_busy())
       { // If TX is busy, wait for it to clear (for a resaonable time)
          clear_watchdog();
          if (timerTicksSince(RadioStatus.LastTXTriggerTick) > RADIO_TIMEOUT_TICKS)
          {               // if not ready in a resonable time
             initRADIO(); // reset radio hardware (stay on same channel)
          }
       }
       return TX_RESULT_SUCCESS + RadioStatus.TX_FAIL; // 1=success, 2=fail
    }
    
    //----------------------------------------------------------------------------
    // Name  : RadioRXPacket
    // Desc  : RX side - what goes in RXBuffer (from RADIO datasheet figure 3-9)
    //	Size	Offset
    //	1		0		Frame length (m+n+2 = header + 102 + FCS)
    //	1		1		LSB of Frame Control (bits)
    //	1		2		MSB of Frame Control (type)
    //	1		3		Sequence number
    //	20		23		Addressing fields, worst case (PANIDx2 = 4, MACx2=16 total =20)
    //	103		126		Payload
    //	1		127		LQI
    //	1		128		RSSI
    
    // Returns count of received packets waiting to be processed & discarded.  Next packet to process is in "Rx".
    // Note this gives you ALL received packets (not just ones addressed to you).   Check the addressing yourself if you care.
    // Also be aware that sucessive identical packets (same frame number) will be received if the far-end misses your ACK (it
    // will re-transmit).  Check for that if you care.
    //----------------------------------------------------------------------------
    uint8_t RadioRXPacket(void)
    {
       if (!RadioStatus.RXPacketCount)
       {
          return 0; // no packets to process
       }
    
       uint8_t *readPoint = (uint8_t *)RXBuffer[RadioStatus.RXReadBuffer]; // recieved packet read point
    
       if (radio_interface_get_tx_busy())
       { // time out and reset radio if we missed interrupts for a long time
          if (timerTicksSince(RadioStatus.LastTXTriggerTick) > RADIO_TIMEOUT_TICKS)
          {
             initRADIO(); // reset radio hardware (stay on same channel)
          }
       }
    
       readPoint = readBytes(BYTEPTR(Rx.frameLength), readPoint, 1); // copy frame length (1), frame control (2), frame number (1), PANID (2)
       readPoint = readBytes(BYTEPTR(Rx.frameControl.w), readPoint, 2);
       readPoint = readBytes(BYTEPTR(Rx.frameNumber), readPoint, 1);
       readPoint = readBytes(BYTEPTR(Rx.dstPANID), readPoint, 2);
    
       // If security is enabled for this packet, toss it (not supported). Also reject Beacon packets as we don't use them
       if ( (Rx.frameControl.securityEnabled) || (Rx.frameControl.frameType == PACKET_TYPE_BEACON) ) 
       {
          RadioStatus.RXSecurityEnabled++; // log error
          RadioDiscardPacket();
          return 0; // avoid recursion
       }
    
       if (Rx.frameControl.frameType == PACKET_TYPE_ACK)
       { // no PANID present on ACK frames [802.15.4 weakness: No way to know if this ACK is really for you]
          readPoint -= 2;
       }
    
       // readPoint now just after first PANID field
    
       if (Rx.frameControl.dstAddrMode == SHORT_ADDR_FIELD) // if a short dest addr is present
          readPoint = readBytes(BYTEPTR(Rx.dstAddr), readPoint, 2);
       else if (Rx.frameControl.dstAddrMode == LONG_ADDR_FIELD) // if a long dest addr is present
          readPoint = readBytes(BYTEPTR(Rx.dstAddr), readPoint, 8);
    
       Rx.srcPANID = Rx.dstPANID; // copy first PANID because we don't know if it's src or dst yet
       Rx.srcAddr = Rx.dstAddr;   // ditto for address
    
       // now readPoint is at start of source PANID (if present)
    
       if (Rx.frameControl.srcAddrMode != NO_ADDR_FIELD &&           // if source present
           Rx.frameControl.dstAddrMode != NO_ADDR_FIELD &&           // and dest present
           !Rx.frameControl.panIDcomp)                               // and no PANID compression
          readPoint = readBytes(BYTEPTR(Rx.srcPANID), readPoint, 2); // then read src PANID
    
       if (Rx.frameControl.srcAddrMode == SHORT_ADDR_FIELD)
       { // if a short src addr is present
          readPoint = readBytes(BYTEPTR(Rx.srcAddr), readPoint, 2);
       }
       else if (Rx.frameControl.srcAddrMode == LONG_ADDR_FIELD)
       { // if a long src addr is present
          readPoint = readBytes(BYTEPTR(Rx.srcAddr), readPoint, 8);
       }
    
       Rx.payload = readPoint; // now readPoint points at the start of the payload
    
       if (tls_decrypt() == FAIL)
       { // if the decryption was bad
          printk("Decryption failed. Src: %llu, Dst: %llu, PanID: %u\r\n", Rx.srcAddr, Rx.dstAddr, Rx.srcPANID);
          RadioDiscardPacket();
          // return RadioRXPacket(); //recursion may be allowed above with securityEnabled check, but am worried this causes occasional crash
          return 0; // avoid recursion
       }
    
       Rx.payloadLength = Rx.frameLength - (readPoint - RXBuffer[RadioStatus.RXReadBuffer]) + 1;
    
       readBytes(BYTEPTR(Rx.lqi), readPoint + Rx.payloadLength, 1);
       readBytes(BYTEPTR(Rx.rssi), readPoint + Rx.payloadLength + 1, 1);
    
       //uint32_t src, dst;
       //src = (uint32_t)(Rx.srcAddr);
       //dst = (uint32_t)(Rx.dstAddr);
       //printk("\r\nGood pkt. ");
       //printk("Src: %lu", src);
       //printk(", Dst: %lu", dst);
       //printk(", PanID: %u", Rx.srcPANID);   
       //printk(", RSSI: %u", Rx.rssi);   
       //printk(", LQI: %u\r\n", Rx.lqi);   
    
       return RadioStatus.RXPacketCount;
    }
    
    //----------------------------------------------------------------------------
    // Name  : RadioDiscardPacket
    // Desc  : If our RX buffer is overrunning, discard a packet
    //----------------------------------------------------------------------------
    void RadioDiscardPacket(void)
    {
       if (RadioStatus.RXPacketCount)
       { // just in case we get called more than we ought
          RadioStatus.RXPacketCount--;
          RadioStatus.RXReadBuffer = (RadioStatus.RXReadBuffer + 1) & (PACKET_BUFFERS - 1);
       }
       else
       {
          RadioStatus.RadioExtraDiscard++;
       }
    }
    
    //----------------------------------------------------------------------------
    // Name  : radio_interrupt
    // Desc  : Called from the radio packet TX/RX interrupt service routine to handle 
    //         the rx data or TX event
    // Ins   : read_interrupt is true if this was a read, false if this was a write
    // Outs  : none
    //----------------------------------------------------------------------------
    void radio_interrupt(bool rx_interrupt, bool tx_interrupt, uint8_t *data, uint16_t length, int8_t rssi, uint8_t lqi, uint8_t tx_fail)
    {
       if (rx_interrupt == true)
       { // RX int?
          uint8_t i;
    
          if (length > RX_BUFFER_SIZE)
          { // if too big for the RX buffer
             RadioStatus.RXPacketTooBig++;
             length = RX_BUFFER_SIZE; // truncate to fit
          }
    
          RXBuffer[RadioStatus.RXWriteBuffer][0] = length; // store length of packet (not counting length byte, FCS, LQI and RSSI)
    
          for (i = 1; i <= length; i++)
          { // copy data from the FIFO into the RX buffer, plus RSSI and LQI
             RXBuffer[RadioStatus.RXWriteBuffer][i] = *data++;
          }
    
          // Insert the RSSI and LQI at the end of the received data
          RXBuffer[RadioStatus.RXWriteBuffer][i++] = lqi;
    
          RXBuffer[RadioStatus.RXWriteBuffer][i++] = rssi;
    
          RadioStatus.RXPacketCount++;
          RadioStatus.RXWriteBuffer = (RadioStatus.RXWriteBuffer + 1) & (PACKET_BUFFERS - 1); // mod PACKET_BUFFERS
    
          if ((RadioStatus.RXPacketCount > PACKET_BUFFERS) || (RadioStatus.RXWriteBuffer == RadioStatus.RXReadBuffer))
          {
             RadioStatus.RXBufferOverruns++;
             //printk("RADIO PACKET OVERRUN\r\n");
             //Throw out two of the oldest unprocessed RX packet to keep from getting overrun
             RadioDiscardPacket();
             RadioDiscardPacket();
          }
       }
    
       if (tx_interrupt == true)
       {                           // TX int?  If so, this means TX is no longer busy, and the result (if any) of the ACK request is in
          if (RadioStatus.TX_PENDING_ACK)
          {                                                      // if we were waiting for an ACK
             RadioStatus.TX_FAIL = tx_fail;                      // 1 for a failure, 0 for success
             RadioStatus.TX_RETRIES = 0;                         // read TXNRETRY, number of retries of last sent packet (0..3)
             RadioStatus.TX_CCAFAIL = 0;                         // read CCAFAIL
             RadioStatus.TX_PENDING_ACK = 0;                     // TX finished, clear that I am pending an ACK, already got it (if I was gonna get it)
          }
       }
    }
    

  • It looks like at this point most of the errors I'm seeing are a result of better debug prints in my newer code and not due to the SDK upgrade. This seems like a network traffic issue that wasn't caught before.

    To hopefully further reduce errors, I ended up adding two k_work_delayable elements to help control the rate of back to back radio traffic a little. I'm not sure how much they are helping, but it seems like a reasonable thing to do.

    So now when my app code wants to transmit a packet I have it delay 500us before putting the outgoing data in the workqueue to transmit to give a little more turnaround between receives and transmits. Then when I hit either nrf_802154_transmitted_raw or nrf_802154_transmit_failed I schedule a 1ms delay before I clear my "tx complete" flag, which is what the higher level app is waiting on to know when it's OK to transmit again.

    Here are some new test results from my network containing the gateway I'm working on and 3 wireless nodes, one of which is running my older SDK 2.3 code that did not have the workqueue to serialize radio operations or the new delays my current code does. 

    In approximately 3h there were a bit over 900 messages and 50 errors.

    • Error 1: NRF_802154_TX_ERROR_BUSY_CHANNEL, qty 3
    • Error 5: NRF_802154_TX_ERROR_NO_ACK, qty 40
    • Error 6: NRF_802154_TX_ERROR_ABORTED, qty 3
    • TX request while busy 3 (This looks like my app trying to send a packet before the last one has finished going out... I thought this would be impossible with my added delays)
    • Failed to start transmit 1 (nrf_802154_transmit_raw() returned false)

    What do you think about the number and quantity of errors I'm getting for this system?

    Thanks,

    Glen

  • Hi again Glen, and thanks for the patience on this,

    It is hard to say that there is nothing that can be changed in the code, but the fact that it worked well in a previous NCS version still makes me not immediately think that this is the problem.

    Glen M said:

    To hopefully further reduce errors, I ended up adding two k_work_delayable elements to help control the rate of back to back radio traffic a little. I'm not sure how much they are helping, but it seems like a reasonable thing to do.

    I agree with you there, and with my assumption of what the issue could've come from, it should've helped. And while on the topic, if nrf_802154_received_raw() was just being started from an IRQ (?) it looks like there is still too much happening there. 

    Note btw that you need to initialize the work with k_work_init_delayable, you use k_work_init here which is only for non-delayable work. You can see if changing that makes a difference. 

    Glen M said:

    It looks like at this point most of the errors I'm seeing are a result of better debug prints in my newer code and not due to the SDK upgrade. This seems like a network traffic issue that wasn't caught before.

    The code you used should otherwise be the same in the two versions right? And does that count for what situations they are tested in? If let's say the NCS 3.1 version is also tested in a situation with more traffic (and would be in particular need of threads working well together) then it might make sense that it just starts breaking now.

    Glen M said:

    What do you think about the number and quantity of errors I'm getting for this system?

    50/900 doesn't sound like a lot, but I wouldn't simply accept it, of course. You had zero errors in the previous NCS version, right?

    MPSL timing requirements being stricter still sound like a potential reason to me.

    The relevant R&D team doesn't have a clear response to this yet.

    Do you remember if your debug-clean-up removed the same type of errors, or if that was a completely different thing?

    Could you show me how you are doing these delays, just to make sure that you are doing as little as possible in an IRQ?

    Regards,

    Elfving

  • Hi Elfving,

    I'm going to mark this issue as closed - I have it working pretty well now and I think the errors I'm having are within the realm of normal.

    One thing I found out about that was very helpful is that Nordic has an 802.15.4 sniffer I was able to program onto one of my nRF52840 devkits - really helped to see the traffic.

    Thanks for the help!

    Glen

Related