nrf5340 qspi flash dma

[szhaulai]

Dear nordic engineers:

The only example I can find about flash is under zephyr/samples/drivers/spi_flash. However, the flash writing is pretty slow, and the MCU is halting while waiting for writing result. 

If dma buffer is available, I can send the command to write dma data to flash and do other time consuming task. The status of the flash writing can be checked in the next MCU loop.

As I understand there is no dma available in any of the zephyr flash api(flash.h). Do you have examples from nordic internal?  I found a post here https://devzone.nordicsemi.com/f/nordic-q-a/72045/flash-filing-system-for-nrf-connect-sdk  , at least someone in nordic has aware of the useful feature (in the marked answer the config file include "help
Enable support for nrfx QSPI driver with EasyDMA.") 

If there is no available example, I would like to implement it but need some guide on how to change nrf_qspi_nor.c and nrfx_qspi and make it accessible from my main.c.  Please provide some hint how I can proceed. 

[szhaulai]

Thanks for reply, can you give example on how to use different thread? I am testing with external memory chip mx25R6435 on nRF5340DK. 

[ovrebekk]

Hi

Making your own threads in Zephyr is very easy, and there is also a sample available here showing you how you can do it. 

Essentially you can create a new thread with a single line of code:

// The thread function should be void(void)
void thread_func(void)
{

}

// Make a thread with priority 5 and stacksize 512
#define MY_THREAD_PRIORITY 5
#define MY_THREAD_STACKSIZE 512
K_THREAD_DEFINE(blink0_id, MY_THREAD_STACKSIZE, thread_func, NULL, NULL, NULL, MY_THREAD_PRIORITY, 0, 0);

If you exit the thread function the thread will be disabled, so normally you would have an infinite loop in the thread function to make it run forever. 

In order for the thread not to occupy the CPU all the time you can use a sleep function, like k_msleep(), or yield the thread by calling k_yield(), whenever you don't have anything to do in the thread.  

Best regards
Torbjørn

[szhaulai]

Hi 

I tried to implement the thread but experience some set back. I couldn't start the thread. Would you please check my file and let me know where I do wrong. Thanks for help! 

I use a timer to create a counter and save the number to a fifo. Then I tried to use a thread that handling the flash, in which, I plan to get the data from fifo and write to flash for every 128 samples. However, the tread is never started. 

Fullscreen 3386.main.c Download

/*
 * Copyright (c) 2016 Intel Corporation.
 *
 * SPDX-License-Identifier: Apache-2.0
 */

#include <zephyr.h>
#include <drivers/flash.h>
#include <device.h>
#include <devicetree.h>
#include <stdio.h>
#include <string.h>
#include <kernel.h>
#include <helpers/nrfx_gppi.h>
#include <nrfx_dppi.h>
#include <nrfx_timer.h>
#include <logging/log.h>
LOG_MODULE_REGISTER(spi_flash, LOG_LEVEL_INF);

nrfx_timer_t timer1 = NRFX_TIMER_INSTANCE(1);
nrfx_timer_t timer2 = NRFX_TIMER_INSTANCE(2);
#if (CONFIG_SPI_NOR - 0) ||				\
	DT_NODE_HAS_STATUS(DT_INST(0, jedec_spi_nor), okay)
#define FLASH_DEVICE DT_LABEL(DT_INST(0, jedec_spi_nor))
#define FLASH_NAME "JEDEC SPI-NOR"
#elif (CONFIG_NORDIC_QSPI_NOR - 0) || \
	DT_NODE_HAS_STATUS(DT_INST(0, nordic_qspi_nor), okay)
#define FLASH_DEVICE DT_LABEL(DT_INST(0, nordic_qspi_nor))
#define FLASH_NAME "JEDEC QSPI-NOR"
#elif DT_NODE_HAS_STATUS(DT_INST(0, st_stm32_qspi_nor), okay)
#define FLASH_DEVICE DT_LABEL(DT_INST(0, st_stm32_qspi_nor))
#define FLASH_NAME "JEDEC QSPI-NOR"
#else
#error Unsupported flash driver
#endif

#if defined(CONFIG_BOARD_ADAFRUIT_FEATHER_STM32F405)
#define FLASH_TEST_REGION_OFFSET 0xf000
#elif defined(CONFIG_BOARD_ARTY_A7_ARM_DESIGNSTART_M1) || \
	defined(CONFIG_BOARD_ARTY_A7_ARM_DESIGNSTART_M3)
/* The FPGA bitstream is stored in the lower 536 sectors of the flash. */
#define FLASH_TEST_REGION_OFFSET \
	DT_REG_SIZE(DT_NODE_BY_FIXED_PARTITION_LABEL(fpga_bitstream))
#else
#define FLASH_TEST_REGION_OFFSET 0x00000
#endif

// inhirit from Henrik's flash structure setup 
#define FLASH_BYTES_PER_PAGE 256
#define FLASH_PAGES_PER_SECTOR 16
#define FLASH_BYTES_PER_SECTOR (FLASH_BYTES_PER_PAGE*FLASH_PAGES_PER_SECTOR)
#define FLASH_CHIP_DENSITY	(0x17 /* 23 */)
#define FLASH_NUM_PAGES (1U << (FLASH_CHIP_DENSITY-8))
#define FLASH_PAGE_MASK (FLASH_NUM_PAGES - 1)
#define FLASH_FSYS_SECTOR 0 /* The sector number of the file system sector (containing file pointers) */
#define FLASH_KVAL_SECTOR 1 /* The sector number of the special key-value sector */
#define FLASH_BULK_SECTOR 2 /* The first sector of the space dedicated to files */

#define FLASH_FSYS_START_PAGE (FLASH_FSYS_SECTOR*FLASH_PAGES_PER_SECTOR)
#define FLASH_KVAL_START_PAGE (FLASH_KVAL_SECTOR*FLASH_PAGES_PER_SECTOR)
#define FLASH_BULK_START_PAGE (FLASH_BULK_SECTOR*FLASH_PAGES_PER_SECTOR)

#define FLASH_FSYS_START_ADDRESS (FLASH_FSYS_SECTOR*FLASH_BYTES_PER_SECTOR)
#define FLASH_KVAL_START_ADDRESS (FLASH_KVAL_SECTOR*FLASH_BYTES_PER_SECTOR)
#define FLASH_BULK_START_ADDRESS (FLASH_BULK_SECTOR*FLASH_BYTES_PER_SECTOR)

// if sampling time is too quick ex 1ms, the write_flash will delay the process, and the next page will start from 141 instead of 127

#define SAMPLE_TIME_ms 		100  // have tried 10us sampling with 100 pages, all good
#define READ_TIME_ms 		10 
#define ANALOG_NUM_BUF_pages 2  // no problem to set 100 pages of buffer


uint8_t outputbuf[ANALOG_NUM_BUF_pages << 8]; 
uint8_t outputbuf1[256];
uint16_t count;

int st,ed;

const struct device *flash_dev;

K_FIFO_DEFINE(flash_write_fifo);
struct data_item_t {
    void *fifo_reserved;   /* 1st word reserved for use by fifo */
    uint16_t counter;
};


void timer1_evt_handler(nrf_timer_event_t event_type, void * p_context)
{	
	switch(event_type)
	{	
		case NRF_TIMER_EVENT_COMPARE0: ;
			struct data_item_t tx_data = { .counter=++count };
			k_fifo_put(&flash_write_fifo, &tx_data);
			printk("count: %d\n",count);
			break;
		
		default:
			//Do nothing.
			break;
	}    
}

void timer_init(void)
{
	nrfx_err_t err = NRFX_SUCCESS;
	nrfx_timer_config_t timer_cfg = NRFX_TIMER_DEFAULT_CONFIG;
	timer_cfg.bit_width = NRF_TIMER_BIT_WIDTH_32;

	err = nrfx_timer_init(&timer1, &timer_cfg, timer1_evt_handler);
	if(err != NRFX_SUCCESS)
	{
		LOG_ERR("timer 1 Error! Could not initialize TIMER1: %d\n", err);
	}
	uint32_t time_ticks = nrfx_timer_ms_to_ticks(&timer1, SAMPLE_TIME_ms);
	nrfx_timer_extended_compare(&timer1, NRF_TIMER_CC_CHANNEL0, time_ticks, NRF_TIMER_SHORT_COMPARE0_CLEAR_MASK, true);	
	nrfx_timer_enable(&timer1);
	IRQ_DIRECT_CONNECT(TIMER1_IRQn, 0, nrfx_timer_1_irq_handler, 0);
	
}
void timer_deinit(void){
	nrfx_timer_disable(&timer1);
	irq_disable(TIMER1_IRQn);
}
// static inline bool caughtUpWithWriting() {
// 	return page_address_read >= page_address_write; //&&
// 		   //(!filled_flash || page_address_read < starting_page);
// }

static void flash_write_thread(void *a1, void *a2, void *a3 ){
	size_t siz, wbsiz;
	int rc;
	uint32_t st,ed;
	
	const struct device *flash_dev;
	
	printf("\n" FLASH_NAME " SPI flash testing\n");
	printf("==========================\n");

	flash_dev = device_get_binding(FLASH_DEVICE);

	if (!flash_dev) {
		printf("SPI flash driver %s was not found!\n",
		       FLASH_DEVICE);
		return;
	}
	
	siz=flash_get_page_count(flash_dev);
	wbsiz=flash_get_write_block_size(flash_dev);

	
   // erase is needed before write, otherwise there may be error between write and read
	printf("\nTest 1: Flash erase\n");
	st=k_cycle_get_32();
	rc = flash_erase(flash_dev, FLASH_TEST_REGION_OFFSET,
			 (ANALOG_NUM_BUF_pages/FLASH_PAGES_PER_SECTOR+1)*FLASH_BYTES_PER_SECTOR); // erase take integer sectors of bytes
	ed=k_cycle_get_32();
	if (rc != 0) {
		printf("Flash erase failed! %d\n", rc);
	} else {
		printf("Flash erase succeeded! %d , %d \n",st,ed);
	}

	struct data_item_t  *rx_data;
	uint8_t tembuf[FLASH_BYTES_PER_PAGE];
	int reads, page_address_write;
	reads=0;
	
	page_address_write=0;
	

	while(1){
		uint8_t *tmptr=tembuf+2*reads;
		rx_data = k_fifo_get(&flash_write_fifo, K_FOREVER);
		*(uint16_t*)tmptr=rx_data->counter;
		printk(" in write thread %d \n", rx_data->counter);
			if(++reads>=128){

				//printk("addr offset %x \n", FLASH_TEST_REGION_OFFSET+page_address_write*FLASH_BYTES_PER_PAGE);
				// st = k_cycle_get_32()/32;
				 rc = flash_write(flash_dev, FLASH_TEST_REGION_OFFSET+page_address_write*FLASH_BYTES_PER_PAGE, tembuf, FLASH_BYTES_PER_PAGE);
				// ed=k_cycle_get_32()/32;
				 if (rc != 0) {
				 	printf("Flash write failed! %d\n", rc);
				 	return;
				 }else{
				 	printk("page_address_write %d, count %d, write %d, %d \n",page_address_write ,count, st,ed);
				 }
				reads = 0;
				if(++page_address_write >= ANALOG_NUM_BUF_pages) {
					timer_deinit();
					printk("reach page %d\n",ANALOG_NUM_BUF_pages);
				
					
				}
			}	
		
	}
}
// K_THREAD_STACK_DEFINE(my_stack_area, 1024);
// struct k_thread my_thread_data;

// k_tid_t my_tid = k_thread_create(&my_thread_data, my_stack_area,
//                                  K_THREAD_STACK_SIZEOF(my_stack_area),
//                                  flash_write_thread,
//                                  NULL, NULL, NULL,
//                                  7, 0, K_NO_WAIT);

 K_THREAD_DEFINE(write_to_flash_id, 1024, flash_write_thread, NULL, NULL, NULL,
		7, 0, 0);
void main(void)
{	
	timer_init();
	k_thread_start(write_to_flash_id);
	while(1){
	}
		
}

0243.prj.conf

[ovrebekk]

Hi

There is no need to run k_thread_start(..) manually. It will start automatically as long as you use the K_THREAD_DEFINE(..) macro. 

I think the problem here is that you have a while(1){} loop in your main function. This will make the main thread run 100%, potentially blocking other threads from running. 

Could you try either removing the loop (with Zephyr it is fine to exit the main() function), or adding a delay function inside the while(1) loop to allow other threads to run?

Just changing the loop like this should be sufficient: 

while(1) {
  k_msleep(1000);
}

Best regards
Torbjørn

[szhaulai]

Hi 

Thanks for the reply. I've been able to let the thread run by making k_msleep. But I still have a bunch of question about thread, which will be deviate from my origin question about flash writing time.  Please suggest if I should raise a new ticket. 

The setback I have experienced is the thread is somehow locked. In the code, it uses __ASSERT_NO_MSG, I have no idea how to debug with the message. I tried to take __ASSERT_NO_MSG away and get some zephyr message about faulting during interrupt handling but I  have no idea how to debug it. 

Fullscreen 0827.main.c Download

/*
 * Copyright (c) 2016 Intel Corporation.
 *
 * SPDX-License-Identifier: Apache-2.0
 */

#include <zephyr.h>
#include <drivers/flash.h>
#include <device.h>
#include <devicetree.h>
#include <stdio.h>
#include <string.h>
#include <kernel.h>
#include <helpers/nrfx_gppi.h>
#include <nrfx_dppi.h>
#include <nrfx_timer.h>
#include <logging/log.h>
LOG_MODULE_REGISTER(spi_flash, LOG_LEVEL_INF);

nrfx_timer_t timer1 = NRFX_TIMER_INSTANCE(1);
nrfx_timer_t timer2 = NRFX_TIMER_INSTANCE(2);
#if (CONFIG_SPI_NOR - 0) ||				\
	DT_NODE_HAS_STATUS(DT_INST(0, jedec_spi_nor), okay)
#define FLASH_DEVICE DT_LABEL(DT_INST(0, jedec_spi_nor))
#define FLASH_NAME "JEDEC SPI-NOR"
#elif (CONFIG_NORDIC_QSPI_NOR - 0) || \
	DT_NODE_HAS_STATUS(DT_INST(0, nordic_qspi_nor), okay)
#define FLASH_DEVICE DT_LABEL(DT_INST(0, nordic_qspi_nor))
#define FLASH_NAME "JEDEC QSPI-NOR"
#elif DT_NODE_HAS_STATUS(DT_INST(0, st_stm32_qspi_nor), okay)
#define FLASH_DEVICE DT_LABEL(DT_INST(0, st_stm32_qspi_nor))
#define FLASH_NAME "JEDEC QSPI-NOR"
#else
#error Unsupported flash driver
#endif

#if defined(CONFIG_BOARD_ADAFRUIT_FEATHER_STM32F405)
#define FLASH_TEST_REGION_OFFSET 0xf000
#elif defined(CONFIG_BOARD_ARTY_A7_ARM_DESIGNSTART_M1) || \
	defined(CONFIG_BOARD_ARTY_A7_ARM_DESIGNSTART_M3)
/* The FPGA bitstream is stored in the lower 536 sectors of the flash. */
#define FLASH_TEST_REGION_OFFSET \
	DT_REG_SIZE(DT_NODE_BY_FIXED_PARTITION_LABEL(fpga_bitstream))
#else
#define FLASH_TEST_REGION_OFFSET 0x00000
#endif

// inhirit from Henrik's flash structure setup 
#define FLASH_BYTES_PER_PAGE 256
#define FLASH_PAGES_PER_SECTOR 16
#define FLASH_BYTES_PER_SECTOR (FLASH_BYTES_PER_PAGE*FLASH_PAGES_PER_SECTOR)
#define FLASH_CHIP_DENSITY	(0x17 /* 23 */)
#define FLASH_NUM_PAGES (1U << (FLASH_CHIP_DENSITY-8))
#define FLASH_PAGE_MASK (FLASH_NUM_PAGES - 1)
#define FLASH_FSYS_SECTOR 0 /* The sector number of the file system sector (containing file pointers) */
#define FLASH_KVAL_SECTOR 1 /* The sector number of the special key-value sector */
#define FLASH_BULK_SECTOR 2 /* The first sector of the space dedicated to files */

#define FLASH_FSYS_START_PAGE (FLASH_FSYS_SECTOR*FLASH_PAGES_PER_SECTOR)
#define FLASH_KVAL_START_PAGE (FLASH_KVAL_SECTOR*FLASH_PAGES_PER_SECTOR)
#define FLASH_BULK_START_PAGE (FLASH_BULK_SECTOR*FLASH_PAGES_PER_SECTOR)

#define FLASH_FSYS_START_ADDRESS (FLASH_FSYS_SECTOR*FLASH_BYTES_PER_SECTOR)
#define FLASH_KVAL_START_ADDRESS (FLASH_KVAL_SECTOR*FLASH_BYTES_PER_SECTOR)
#define FLASH_BULK_START_ADDRESS (FLASH_BULK_SECTOR*FLASH_BYTES_PER_SECTOR)

// if sampling time is too quick ex 1ms, the write_flash will delay the process, and the next page will start from 141 instead of 127

#define SAMPLE_TIME_ms 		2  // have tried 10us sampling with 100 pages, all good
#define ANALOG_NUM_BUF_pages 5  // no problem to set 100 pages of buffer

uint16_t count;

int st,ed;

const struct device *flash_dev;

K_FIFO_DEFINE(flash_write_fifo);
struct data_item_t {
    void *fifo_reserved;   /* 1st word reserved for use by fifo */
    uint16_t counter;
};


void timer1_evt_handler(nrf_timer_event_t event_type, void * p_context)
{	
	switch(event_type)
	{	
		case NRF_TIMER_EVENT_COMPARE0: ;
			struct data_item_t tx_data = { .counter=count };
			size_t size = sizeof(struct data_item_t);
			char *mem_ptr = k_malloc(size);
			__ASSERT_NO_MSG(mem_ptr != 0);

			memcpy(mem_ptr, &tx_data, size);

			k_fifo_put(&flash_write_fifo, &tx_data);
			count++;
			//printk("count: %d\n",count);
			break;
		
		default:
			//Do nothing.
			break;
	}    
}

void timer_init(void)
{
	nrfx_err_t err = NRFX_SUCCESS;
	nrfx_timer_config_t timer_cfg = NRFX_TIMER_DEFAULT_CONFIG;
	timer_cfg.bit_width = NRF_TIMER_BIT_WIDTH_32;

	err = nrfx_timer_init(&timer1, &timer_cfg, timer1_evt_handler);
	if(err != NRFX_SUCCESS)
	{
		LOG_ERR("timer 1 Error! Could not initialize TIMER1: %d\n", err);
	}
	uint32_t time_ticks = nrfx_timer_ms_to_ticks(&timer1, SAMPLE_TIME_ms);
	nrfx_timer_extended_compare(&timer1, NRF_TIMER_CC_CHANNEL0, time_ticks, NRF_TIMER_SHORT_COMPARE0_CLEAR_MASK, true);	
	nrfx_timer_enable(&timer1);
	IRQ_DIRECT_CONNECT(TIMER1_IRQn, 0, nrfx_timer_1_irq_handler, 0);
	
}
void timer_deinit(void){
	nrfx_timer_disable(&timer1);
	irq_disable(TIMER1_IRQn);
}


static void flash_write_thread(void *a1, void *a2, void *a3 ){
	size_t siz, wbsiz;
	int rc;
	uint32_t st,ed;
	
	const struct device *flash_dev;
	
	printf("\n" FLASH_NAME " SPI flash testing\n");
	printf("==========================\n");

	flash_dev = device_get_binding(FLASH_DEVICE);

	if (!flash_dev) {
		printf("SPI flash driver %s was not found!\n",
		       FLASH_DEVICE);
		return;
	}
	
	siz=flash_get_page_count(flash_dev);
	wbsiz=flash_get_write_block_size(flash_dev);

	
   // erase is needed before write, otherwise there may be error between write and read
	printf("\nTest 1: Flash erase\n");
	st=k_cycle_get_32();
	rc = flash_erase(flash_dev, FLASH_TEST_REGION_OFFSET,
			 (ANALOG_NUM_BUF_pages/FLASH_PAGES_PER_SECTOR+1)*FLASH_BYTES_PER_SECTOR); // erase take integer sectors of bytes
	ed=k_cycle_get_32();
	if (rc != 0) {
		printf("Flash erase failed! %d\n", rc);
	} else {
		printf("Flash erase succeeded! %d , %d \n",st,ed);
	}
	uint8_t tembuf[FLASH_BYTES_PER_PAGE];
	int reads, page_address_write;
	reads=0;
	
	page_address_write=0;
	

	while(1){
		
		if(!k_fifo_is_empty(&flash_write_fifo)){
			uint8_t *tmptr=tembuf+2*reads;
			struct data_item_t  *rx_data = k_fifo_get(&flash_write_fifo, K_FOREVER);
			*(uint16_t*)tmptr=rx_data->counter;
			
			printk(" in write thread %d \n", *tmptr);
				if(++reads>=128){
					//printk("addr offset %x \n", FLASH_TEST_REGION_OFFSET+page_address_write*FLASH_BYTES_PER_PAGE);
					st = k_cycle_get_32()/32;
					rc = flash_write(flash_dev, FLASH_TEST_REGION_OFFSET+page_address_write*FLASH_BYTES_PER_PAGE, tembuf, FLASH_BYTES_PER_PAGE);
					ed=k_cycle_get_32()/32;
					if (rc != 0) {
						printf("Flash write failed! %d\n", rc);
						return;
					}else{
						printk("page_address_write %d, count %d, write %d, %d \n",page_address_write ,count, st,ed);
					}
					reads = 0;
					if(++page_address_write >= ANALOG_NUM_BUF_pages) {
						timer_deinit();
						printk("reach page %d\n",ANALOG_NUM_BUF_pages);
					
						
					}
				}	
			//k_free(rx_data);
		}
	}
}

 K_THREAD_DEFINE(write_to_flash_id, 2048, flash_write_thread, NULL, NULL, NULL,
		7, 0, 0);
void main(void)
{	
	count=0;
	timer_init();
	//k_thread_start(write_to_flash_id);
	while(1){
		k_msleep(15);
	}		
}

Please give me some indication how I can debug thread and interrupt. I've tried with breakpoint but it doesn't seems to help, the thread just suddenly halted. 

Another question I have about thread is about the sleep time that you suggested. 

question 1. if I have a thread that needs a long time to process (assume extreme case 100ms), is it a "o.k" practice to have a sleep period less than the process time (assume 1ms). 

question 2: assume I have many threads that do multiple signal processings, writing to flash. How do I know which thread will be run during the sleep. Should I plan a specific time for the main thread to sleep and activate a specific thread to run during the sleep time of main thread?

I hope there are example that can explain this better. 

Thanks for help

 

[ovrebekk]

Hi 

Maybe yo could try just to print something to the log if mem_ptr != 0 ?

You should skip the remaining code in the timer event handler if this occurs, to avoid writing to the flash_write_fifo if the malloc fails. 

Also, unless you don't use k_malloc anywhere else it is not safe to call this function from an interrupt handler. It is best to only use this method from a thread or a work item (or even better to use statically defined memory structures, rather than dynamic memory allocation).

szhaulai said:

question 1. if I have a thread that needs a long time to process (assume extreme case 100ms), is it a "o.k" practice to have a sleep period less than the process time (assume 1ms). 

If the thread has a positive priority number it can be interrupted by other, more important threads as long as they have a higher interrupt priority (lower number). 

As an example, if you give your time consuming but less critical thread a priority of 5, you can put other more time critical threads at priority 4 or below, and they will be able to interrupt the slow running thread when they have some work to do. 

szhaulai said:

question 2: assume I have many threads that do multiple signal processings, writing to flash. How do I know which thread will be run during the sleep. Should I plan a specific time for the main thread to sleep and activate a specific thread to run during the sleep time of main thread?

You don't really know when the different threads are going to run, this is all controlled by the scheduler. 

If you feel that you need to know this information then it sounds like you need a better way to synchronize the different activities in your program. 

As an example, if you have multiple processes that need to access the flash at different times you can use a FIFO to allow the different threads to schedule flash operations, and execute the operations (read the FIFO) from a single thread so that you avoid conflicts. 

Best regards
Torbjørn