Beware that this post is related to an SDK in maintenance mode
More Info: Consider nRF Connect SDK for new designs
This post is older than 2 years and might not be relevant anymore
More Info: Consider searching for newer posts

FATFS/SD Card Write Stuck in wait_func when called from app_timer handler

I'm developing an application on an nRF52832 revAA that logs data coming from the app_uart module. It logs this data to an SD card using the fatfs library; the code is more-or-less exactly the same as the fatfs example in the SDK v15.2. The issue is, calls to f_write work fine during initialization, but if it's called from an app_timer instance handler, it hangs forever.

Here's a simplified version of what my application looks like:

int main()
{
    log_init();
    app_timer_init();
    power_management_init();
    
    app_timer_create(&m_state_machine_timer, APP_TIMER_MODE_REPEATED, state_machine_handler);
    
    
    /* This initializes the sensor of interest, which itself
       just sets up an app_uart instance that writes to a char buffer.
      
       When a full message is received, it sets the value of "sensor_message_ready"
       to true and, when the message is read, sets it back to false */
    sensor_init();
    
    if (sdcard_mount() && sdcard_create_file("log.txt", sizeof("log.txt")))
	{
	    // *** This call works fine, I can see "test line" written to the SD card ***
	    sdcard_write_line("log.txt", "test line", sizeof("test line"));
	    
	    app_timer_start(m_state_machine_timer, APP_TIMER_TICKS(50), NULL);
	    
		start_state_machine();
	}
	
	for(;;)
	{
	    idle_state_handle();
	}
}

static void idle_state_handle(void)
{
    if (NRF_LOG_PROCESS() == false)
    {
        nrf_pwr_mgmt_run();
    }
}

static void state_machine_handler(void * p)
{
    // States omitted for clarity, this is one state inside a larger switch/case statement
    
    if(sensor_message_ready)
    {
        size_t msg_size = 0;
        
        // This function returns the sensor message and sets the value of msg_size
        // to the length of that string
        char *msg = get_sensor_message(&msg_size);
        
        // *** This call fails, and never returns ***
        sdcard_write_line(msg, msg_size);
    }
}

bool sdcard_mount(void)
{
	//uint32_t bytes_written;
    FRESULT ff_result;
    DSTATUS disk_state = STA_NOINIT;
	
    // Initialize FATFS disk I/O interface by providing the block device.
    static diskio_blkdev_t drives[] =
    {
            DISKIO_BLOCKDEV_CONFIG(NRF_BLOCKDEV_BASE_ADDR(m_block_dev_sdc, block_dev), NULL)
    };

    diskio_blockdev_register(drives, ARRAY_SIZE(drives));

    for (uint32_t retries = 3; retries && disk_state; --retries)
    {
        disk_state = disk_initialize(0);
    }
    if (disk_state)
    {
        return false;
    }

    uint32_t blocks_per_mb = (1024uL * 1024uL) / m_block_dev_sdc.block_dev.p_ops->geometry(&m_block_dev_sdc.block_dev)->blk_size;
    uint32_t capacity = m_block_dev_sdc.block_dev.p_ops->geometry(&m_block_dev_sdc.block_dev)->blk_count / blocks_per_mb;

    ff_result = f_mount(&fs, "", 1);
    if (ff_result)
    {
        return false;
    }

    return true;
}

bool sdcard_create_file(char * filename, size_t fn_length)
{
	FRESULT ff_result;
    ff_result = f_open(&file, filename, FA_READ | FA_WRITE | FA_OPEN_APPEND);
    if (ff_result != FR_OK)
    {
        return false;
    }
    
	f_close(&file);
	
    return true;
}

void sdcard_write_line(char * filename, char * to_write, size_t size)
{
    FRESULT ff_result;
    uint32_t bytes_written;

    ff_result = f_open(&file, filename, FA_WRITE);
    
    if (ff_result != FR_OK)
    {
        return;
    }
    
    // *** The code hangs within this function. Specifically within diskio_blkdev.c:84 in the default_wait_func()
	ff_result = f_write(&file, to_write, size - 1, (UINT *) &bytes_written);

    if (ff_result != FR_OK)
    {
        NRF_LOG_INFO("Write failed\r\n.");
    }

    f_close(&file); 
}

As highlighted in the code, within the main context I can mount the SD card, create a new file, and write a line to it. After that point, I run the state machine timer and go to sleep. Whenever the state machine timer event triggers, it checks whether a message is ready and, if so, tries to write it to the SD card. However, all calls to sdcard_write_line hang indefinitely when called from the timer handler. Specifically, they hang in diskio_blkdev.c:84   which is the default wait function for the drive.

I put a logic analyzer on the SD card lines and found that all activity on those lines stops (CS low, SCK idle, etc.) once it calls __WFE within that default wait function, so naturally it will never wake up (but maybe I'm confused about the flow of things within the SD card module/handler?).

I've tried a few different things:

  • Slowed down the state machine timer to a glacial pace to see if it was interrupt bound
  • Changed interrupt priority of SPI/SPIM to higher than that of app_timer (2 and 6, respectively)
  • Changed interrupt priority of SPI/SPIM to lower than that of app_timer (6 and 2, respectively)

None of these had any effect, it seems like it always goes to sleep and then never wakes up. I'm going to guess it has something to do with reentrant calls to __WFE (via. the idle_state_handle and within the diskio_blkdev stuff), but I'm not sure how to avoid that given the structure of my application. Does this make any sense? And if so, is there a good workaround so I can make calls to write to the SD card from within my state machine?

Parents
  • If I understand your design correctly, I don't think you can do things the way you're trying to do them.

    If the timer callback works the way I think it does, then it's essentially an interrupt service routine: when the timer interval expires, you get an interrupt, and you can use that to trigger other things. But there is a limit to what you can reasonably do in an interrupt handler.

    When you issue an f_write(), the underlying code will need to perform SPI transactions and it has to wait for the transactions to complete. You can detect completion either by waiting for an interrupt from the SPI controller or polling a status register. The problem is, these are both basically "blocking" operations, and you don't want to perform a blocking operation inside an interrupt service routine. ISRs should be short and should complete in a well bounded amount of time.

    I think what you may need to do is to have the timer handler update a variable that indicates the timer has expired. You need to find a way to have your idle_state_handler() function wake up due to the interrupt and test the variable, and if it indicates the timer has fired, then check for data and write it to the SD card from the application context instead of from interrupt context.

    -Bill

Reply
  • If I understand your design correctly, I don't think you can do things the way you're trying to do them.

    If the timer callback works the way I think it does, then it's essentially an interrupt service routine: when the timer interval expires, you get an interrupt, and you can use that to trigger other things. But there is a limit to what you can reasonably do in an interrupt handler.

    When you issue an f_write(), the underlying code will need to perform SPI transactions and it has to wait for the transactions to complete. You can detect completion either by waiting for an interrupt from the SPI controller or polling a status register. The problem is, these are both basically "blocking" operations, and you don't want to perform a blocking operation inside an interrupt service routine. ISRs should be short and should complete in a well bounded amount of time.

    I think what you may need to do is to have the timer handler update a variable that indicates the timer has expired. You need to find a way to have your idle_state_handler() function wake up due to the interrupt and test the variable, and if it indicates the timer has fired, then check for data and write it to the SD card from the application context instead of from interrupt context.

    -Bill

Children
Related