Updating to nrfxlib 2.4.1 causes nrf_modem_ init() to timeout (errno 116)

RE: Updating to nrfxlib 2.4.1 causes nrf_modem_ init() to timeout (errno 116)

dkhayes117 — Mon, 07 Aug 2023 14:26:47 GMT

Marking this as solved. It turns out that the nrf_modem_os_sem_take() function has a timeout parameter that is in milliseconds.

Our match on timeout calls nrf_modem_busy_wait() with a delay value of 1 and decrements the timeout value by 1 when an integer timeout value is passed in.

It turns out that the busywait function takes microseconds, not milliseconds; therefore, the timeout would happen 1000 times sooner than expected!

RE: Updating to nrfxlib 2.4.1 causes nrf_modem_ init() to timeout (errno 116)

dkhayes117 — Thu, 03 Aug 2023 01:01:36 GMT

I made another discovery, I was fooled that it kept working by not refreshing my cargo.lock file after changes. I've found the root of the problem, although I don't understand the implication. In our nrf_modem_os_sem_take() function we match the timeout parameter to whether it should return EAGAIN, busywait forever, or busywait for the timeout value. increasing the delay when a timeout int is provided fixes the issue.

pub extern "C" fn nrf_modem_os_sem_take(
    sem: *mut core::ffi::c_void,
    mut timeout: core::ffi::c_int,
) -> core::ffi::c_int {
    unsafe {
        if sem.is_null() {
            return -(nrfxlib_sys::NRF_EAGAIN as i32);
        }

        if nrfxlib_sys::nrf_modem_os_is_in_isr() {
            timeout = nrfxlib_sys::NRF_MODEM_OS_NO_WAIT as i32;
        }

        loop {
            if (*(sem as *mut Semaphore))
                .current_value
                .fetch_update(Ordering::SeqCst, Ordering::SeqCst, |val| {
                    if val > 0 {
                        Some(val - 1)
                    } else {
                        None
                    }
                })
                .is_ok()
            {
                return 0;
            }

            match timeout {
                0 => return -(nrfxlib_sys::NRF_EAGAIN as i32),
                nrfxlib_sys::NRF_MODEM_OS_FOREVER => {
                    nrf_modem_os_busywait(1);
                }
                _ => {
                    timeout -= 1;
                    nrf_modem_os_busywait(2); // changing this from 1 to 2 fixes the timeout error
                }
            }
        }
    }
}

RE: Updating to nrfxlib 2.4.1 causes nrf_modem_ init() to timeout (errno 116)

dkhayes117 — Wed, 02 Aug 2023 16:25:23 GMT

temporarily changing the nrf_modem_os_busywait() function to take longer than needed fixed the modem init timeout error. Later changing it back to the original delay, and it works just fine now! Definitely something strange is going on, seeing as someone else had the same issue with the nrfconnect sdk.

devzone.nordicsemi.com/.../437742

RE: Updating to nrfxlib 2.4.1 causes nrf_modem_ init() to timeout (errno 116)

dkhayes117 — Fri, 28 Jul 2023 03:16:43 GMT

Thanks for pointing that change out. I updated our IPC handler function, but it still didn't work, so I added RTT logging to every single FFI function we had defined and it started working!

It seems to have been some sort of timing issue. Inside our nrf_modem_os_timedwait(), we can't use K_FOREVER, so it loops over a nrf_modem_os_busywait() function that does nothing but uses assembly to create a delay. The calculation was usec * 64, so I changed it to times 200 and removed the trace logging and it still worked. I'm not sure why that would cause a timeout error, but this is progress 😃

cortex_m::asm::delay((usec as u32) * 200);

RE: Updating to nrfxlib 2.4.1 causes nrf_modem_ init() to timeout (errno 116)

Didrik Rokhaug — Thu, 27 Jul 2023 15:08:32 GMT

It's been very long since I last worked with Rust, so I was hoping I wouldn't have to figure out how to set up the project.

Looking at nrf-modem, do I understand it correctly that you have re-implemented the nrfx drivers used by the modem library, rather than using our implementation?

With nrf_modem 2.4.0, the nrfx version it was using was also increased.

Could you try to add this part to the nrfx_ipc_irq_handler, and see if that solves your problem?

RE: Updating to nrfxlib 2.4.1 causes nrf_modem_ init() to timeout (errno 116)

dkhayes117 — Thu, 27 Jul 2023 12:11:47 GMT

Thank you Didrik,

Here is the function that we use to init the modem.

Here is our memory.x definition for linking

Those should be the only relevant pieces to this puzzle, I think.

Just in case, here is the link to nrfxlib-sys being used to test the update to 2.4.1.

also, I'm using mfw 1.3.5.

-Daniel

RE: Updating to nrfxlib 2.4.1 causes nrf_modem_ init() to timeout (errno 116)

Didrik Rokhaug — Thu, 27 Jul 2023 08:45:36 GMT

Hi,

The hardfault seems to come from an assert (unwrap()) checking the return value of nrf_modem_init(), so the hardfault should disappear when you are able to initialize the modem.

Can you share your project (or enough of it to reproduce your problem), so I can look at how it is set up?

We don't officially support Rust, but I can at least try to have a look.

Also, due to summer vacations, we are a bit short on staff, so responses might be slower than normal.

Best regards,

Didrik