TWIM usage in non secure application core

[GiulianoFranchetto]

Hi,

We are currently porting our code to the newest nRf5340 chip.

Everything goes fine when the target is secure (not ending with "ns" and not defining "CONFIG_TRUSTED_EXECUTION_NONSECURE".

However, when changing the target to be "nonsecure", the I2C stops working, with every i2c_xxx function returning -EBUSY.

What do I need to do to enable i2c when using the non-secure target?

Our device tree is using the same philosophy as nrf5340dk, with a common dts file with 

&i2c2 {
	compatible = "nordic,nrf-twim";
	status = "okay";
	sda-pin = <34>;
	scl-pin = <4>;
	clock-frequency = <I2C_BITRATE_STANDARD>;
	zephyr,concat-buf-size = <520>;
	
	....
};

Regards

Giuliano

[Carl Richard]

Hi!

Just to confirm: you're using the nRF5340DK v.0.11? It should be sufficient to rename your overlay file to nrf5340dk_nrf5340_cpuappns.overlay. In addition you must make sure to add the _NS postfix (like this NRF_TWIM2_NS) when referencing the device directly.

Best regards,
Carl Richard

[GiulianoFranchetto]

Hi!

We are using our own custom board. 

The i2c device is found without problem (device_get_binding() is not NULL).

The issue here is not that the bus is not referenced, but is said to be "busy".

My guts tell me that the I2C periph. is not properly configured once entering the NS domain, and one (or both) SDA/SCL line is not well configured.

Once again, this issue is not present if the program is built on the secure target

Regards

[Carl Richard]

Hi again!

I had another look at your device tree file. The P1.x pins of the nRF5340 are considered secure by default by the secure partition manager, this includes pin 34. Could you try to replace the code of <ncs_root>\nrf\subsys\spm\spm.c with the following: 

/*
 * Copyright (c) 2019 Nordic Semiconductor ASA
 *
 * SPDX-License-Identifier: LicenseRef-BSD-5-Clause-Nordic
 */

#include <zephyr.h>
#include <sys/printk.h>
#include <sys/util.h>
#include <linker/linker-defs.h>
#include <device.h>
#include <drivers/gpio.h>
#include <hal/nrf_spu.h>
#include <arch/arm/aarch32/irq.h>
#include "spm_internal.h"

#if !defined(CONFIG_ARM_SECURE_FIRMWARE)
#error "Module requires compiling for Secure ARM Firmware"
#endif

/* Include required APIs for TrustZone-M */
#include <arm_cmse.h>
#include <cortex_m/tz.h>

#include <nrfx.h>

#if USE_PARTITION_MANAGER
#include <pm_config.h>
#define NON_SECURE_APP_ADDRESS PM_APP_ADDRESS
#define NON_SECURE_RAM_OFFSET PM_SRAM_SECURE_SIZE
#else
#include <storage/flash_map.h>
#define NON_SECURE_APP_ADDRESS FLASH_AREA_ID(image_0_nonsecure)
/* This reflects the configuration in DTS. */
#define NON_SECURE_RAM_OFFSET 0x10000
#endif /* USE_PARTITION_MANAGER */

#define NON_SECURE_FLASH_REGION_INDEX \
	((NON_SECURE_APP_ADDRESS) / (FLASH_SECURE_ATTRIBUTION_REGION_SIZE))
#define NON_SECURE_RAM_REGION_INDEX \
	((NON_SECURE_RAM_OFFSET) / (RAM_SECURE_ATTRIBUTION_REGION_SIZE))

/*
 *  * The security configuration for depends on where the non secure app
 *  * is placed. All flash regions before the region which contains the
 *  * non secure app is configured as Secure.
 *
 *                FLASH
 *  1 MB  |---------------------|
 *        |                     |
 *        |                     |
 *        |                     |
 *        |                     |
 *        |                     |
 *        |     Non-Secure      |
 *        |       Flash         |
 *        |                     |
 *  X kB  |---------------------|
 *        |                     |
 *        |     Secure          |
 *        |      Flash          |
 *  0 kB  |---------------------|
 *
 *  * The SRAM configuration is given by the partition manager.
 *  * To see the current configuration, run the 'pm_report' target.
 *  * E.g. 'ninja pm_report, in your build folder. All partitions
 *  * within the 'secure_ram' span is configured as secure by the SPM.
 *
 */

extern irq_target_state_t irq_target_state_set(unsigned int irq,
	irq_target_state_t irq_target_state);
extern int irq_target_state_is_secure(unsigned int irq);

/* printk wrapper, to turn off logs when booting silently */
#define PRINT(...)                                                             \
	do {                                                                   \
		if (!IS_ENABLED(CONFIG_SPM_BOOT_SILENTLY)) {                   \
			printk(__VA_ARGS__);                                   \
		}                                                              \
	} while (0)

/* Local convenience macro to extract the peripheral
 * ID from the base address.
 */
#define NRFX_PERIPHERAL_ID_GET(base_addr) \
	(uint8_t)((uint32_t)(base_addr) >> 12)

#ifdef CONFIG_SPM_BOOT_SILENTLY
#define PERIPH(name, reg, config)                                              \
	{                                                                      \
		.id = NRFX_PERIPHERAL_ID_GET(reg), IS_ENABLED(config)          \
	}
#else
#define PERIPH(name, reg, config)                                              \
	{                                                                      \
		name, .id = NRFX_PERIPHERAL_ID_GET(reg), IS_ENABLED(config)    \
	}
#endif

/* Check if configuration exceeds the number of
 * DPPI Channels available on device.
 */
#if (CONFIG_SPM_NRF_DPPIC_PERM_MASK >= (1 << DPPI_CH_NUM))
#error "SPM_NRF_DPPIC_PERM_MASK exceeds number of available DPPI channels"
#endif

#if defined(CONFIG_ARM_FIRMWARE_HAS_SECURE_ENTRY_FUNCS)

static void spm_config_nsc_flash(void)
{
	/* Configure a single region in Secure Flash as Non-Secure Callable
	 * (NSC) area.
	 *
	 * Area to configure is dynamically decided with help from linker code.
	 *
	 * Note: Any Secure Entry functions, exposing secure services to the
	 * Non-Secure firmware, shall be located inside this NSC area.
	 *
	 * If the start address of the NSC area is hard-coded, it must follow
	 * the HW restrictions: The size must be a power of 2 between 32 and
	 * 4096, and the end address must fall on a SPU region boundary.
	 */
	uint32_t nsc_size = FLASH_NSC_SIZE_FROM_ADDR(__sg_start);

	__ASSERT((uint32_t)__sg_size <= nsc_size,
		"The Non-Secure Callable region is overflowed by %d byte(s).\n",
		(uint32_t)__sg_size - nsc_size);

	nrf_spu_flashnsc_set(NRF_SPU, 0, FLASH_NSC_SIZE_REG(nsc_size),
			FLASH_NSC_REGION_FROM_ADDR(__sg_start), false);

	PRINT("Non-secure callable region 0 placed in flash region %d with size %d.\n",
		NRF_SPU->FLASHNSC[0].REGION, NRF_SPU->FLASHNSC[0].SIZE << 5);
}
#endif /* CONFIG_ARM_FIRMWARE_HAS_SECURE_ENTRY_FUNCS */


static void config_regions(bool ram, size_t start, size_t end, uint32_t perm)
{
	const size_t region_size = ram ? RAM_SECURE_ATTRIBUTION_REGION_SIZE
					: FLASH_SECURE_ATTRIBUTION_REGION_SIZE;

	for (size_t i = start; i < end; i++) {
		if (ram) {
			NRF_SPU->RAMREGION[i].PERM = perm;
		} else {
			NRF_SPU->FLASHREGION[i].PERM = perm;
		}
	}

	PRINT("%02u %02u 0x%05x 0x%05x \t", start, end - 1,
				region_size * start, region_size * end);
	PRINT("%s", perm & (ram ? SRAM_SECURE : FLASH_SECURE) ? "Secure\t\t" :
								"Non-Secure\t");
	PRINT("%c", perm & (ram ? SRAM_READ : FLASH_READ)  ? 'r' : '-');
	PRINT("%c", perm & (ram ? SRAM_WRITE : FLASH_WRITE) ? 'w' : '-');
	PRINT("%c", perm & (ram ? SRAM_EXEC : FLASH_EXEC)  ? 'x' : '-');
	PRINT("%c", perm & (ram ? SRAM_LOCK : FLASH_LOCK)  ? 'l' : '-');
	PRINT("\n");
}


static void spm_config_flash(void)
{
	/* Regions of flash up to and including SPM are configured as Secure.
	 * The rest of flash is configured as Non-Secure.
	 */
	const uint32_t secure_flash_perm = FLASH_READ | FLASH_WRITE | FLASH_EXEC
			| FLASH_LOCK | FLASH_SECURE;
	const uint32_t nonsecure_flash_perm = FLASH_READ | FLASH_WRITE | FLASH_EXEC
			| FLASH_LOCK | FLASH_NONSEC;

	PRINT("Flash regions\t\tDomain\t\tPermissions\n");

	config_regions(false, 0, NON_SECURE_FLASH_REGION_INDEX,
			secure_flash_perm);
	config_regions(false, NON_SECURE_FLASH_REGION_INDEX,
			NUM_FLASH_SECURE_ATTRIBUTION_REGIONS,
			nonsecure_flash_perm);
	PRINT("\n");

#if defined(CONFIG_ARM_FIRMWARE_HAS_SECURE_ENTRY_FUNCS)
	spm_config_nsc_flash();
	PRINT("\n");

#if defined(CONFIG_SPM_SECURE_SERVICES)
	int err = spm_secure_services_init();

	if (err != 0) {
		PRINT("Could not initialize secure services (err %d).\n", err);
	}
#endif
#endif /* CONFIG_ARM_FIRMWARE_HAS_SECURE_ENTRY_FUNCS */
}

static void spm_config_sram(void)
{
	/* Lower 64 kB of SRAM is allocated to the Secure firmware image.
	 * The rest of SRAM is allocated to Non-Secure firmware image.
	 */

	const uint32_t secure_ram_perm = SRAM_READ | SRAM_WRITE | SRAM_EXEC
		| SRAM_LOCK | SRAM_SECURE;
	const uint32_t nonsecure_ram_perm = SRAM_READ | SRAM_WRITE | SRAM_EXEC
		| SRAM_LOCK | SRAM_NONSEC;

	PRINT("SRAM region\t\tDomain\t\tPermissions\n");

	/* Configuration for Secure RAM Regions (0 - 64 kB) */
	config_regions(true, 0, NON_SECURE_RAM_REGION_INDEX,
			secure_ram_perm);
	/* Configuration for Non-Secure RAM Regions (64 kb - end) */
	config_regions(true, NON_SECURE_RAM_REGION_INDEX,
			NUM_RAM_SECURE_ATTRIBUTION_REGIONS,
			nonsecure_ram_perm);
	PRINT("\n");
}

static bool usel_or_split(uint8_t id)
{
	const uint32_t perm = NRF_SPU->PERIPHID[id].PERM;

	/* NRF_GPIOTE1_NS needs special handling as its
	 * peripheral ID for non-secure han incorrect properties
	 * in the NRF_SPM->PERIPHID[id].perm register.
	 */
	if (id == NRFX_PERIPHERAL_ID_GET(NRF_GPIOTE1_NS)) {
		return true;
	}

	bool present = (perm & SPU_PERIPHID_PERM_PRESENT_Msk) ==
		       SPU_PERIPHID_PERM_PRESENT_Msk;

	/* User-selectable attribution */
	bool usel = (perm & SPU_PERIPHID_PERM_SECUREMAPPING_Msk) ==
		    SPU_PERIPHID_PERM_SECUREMAPPING_UserSelectable;

	/* Split attribution */
	bool split = (perm & SPU_PERIPHID_PERM_SECUREMAPPING_Msk) ==
		     SPU_PERIPHID_PERM_SECUREMAPPING_Split;

	return present && (usel || split);
}

static int spm_config_peripheral(uint8_t id, bool dma_present)
{
	/* Set a peripheral to Non-Secure state, if
	 * - it is present
	 * - has UserSelectable/Split attribution.
	 *
	 * Assign DMA capabilities and lock down the attribution.
	 *
	 * Note: the function assumes that the peripheral ID matches
	 * the IRQ line.
	 */
	NVIC_DisableIRQ(id);

	if (usel_or_split(id)) {
		NRF_SPU->PERIPHID[id].PERM = PERIPH_PRESENT | PERIPH_NONSEC |
			(dma_present ? PERIPH_DMA_NOSEP : 0) |
			PERIPH_LOCK;
	}

	/* Even for non-present peripherals we force IRQs to be routed
	 * to Non-Secure state.
	 */
	irq_target_state_set(id, IRQ_TARGET_STATE_NON_SECURE);
	return 0;
}

static void spm_dppi_configure(uint32_t mask)
{
	NRF_SPU->DPPI[0].PERM = mask;
}

static void spm_config_peripherals(void)
{
	struct periph_cfg {
#ifndef CONFIG_SPM_BOOT_SILENTLY
		char *name;
#endif
		uint8_t id;
		uint8_t nonsecure;
	};

	/* - All user peripherals are allocated to the Non-Secure domain.
	 * - All GPIOs are allocated to the Non-Secure domain.
	 */
	static const struct periph_cfg periph[] = {
#ifdef NRF_P0
		PERIPH("NRF_P0", NRF_P0, CONFIG_SPM_NRF_P0_NS),
#endif
#ifdef NRF_CLOCK
		PERIPH("NRF_CLOCK", NRF_CLOCK, CONFIG_SPM_NRF_CLOCK_NS),
#endif
#ifdef NRF_RTC0
		PERIPH("NRF_RTC0", NRF_RTC0, CONFIG_SPM_NRF_RTC0_NS),
#endif
#ifdef NRF_RTC1
		PERIPH("NRF_RTC1", NRF_RTC1, CONFIG_SPM_NRF_RTC1_NS),
#endif
#ifdef NRF_NFCT
		PERIPH("NRF_NFCT", NRF_NFCT, CONFIG_SPM_NRF_NFCT_NS),
#endif
#ifdef NRF_NVMC
		PERIPH("NRF_NVMC", NRF_NVMC, CONFIG_SPM_NRF_NVMC_NS),
#endif
#ifdef NRF_UARTE1
		PERIPH("NRF_UARTE1", NRF_UARTE1, CONFIG_SPM_NRF_UARTE1_NS),
#endif
#ifdef NRF_UARTE2
		PERIPH("NRF_UARTE2", NRF_UARTE2, CONFIG_SPM_NRF_UARTE2_NS),
#endif
#ifdef NRF_TWIM2
		PERIPH("NRF_TWIM2", NRF_TWIM2, CONFIG_SPM_NRF_TWIM2_NS),
#endif
#ifdef NRF_SPIM3
		PERIPH("NRF_SPIM3", NRF_SPIM3, CONFIG_SPM_NRF_SPIM3_NS),
#endif
#ifdef NRF_TIMER0
		PERIPH("NRF_TIMER0", NRF_TIMER0, CONFIG_SPM_NRF_TIMER0_NS),
#endif
#ifdef NRF_TIMER1
		PERIPH("NRF_TIMER1", NRF_TIMER1, CONFIG_SPM_NRF_TIMER1_NS),
#endif
#ifdef NRF_TIMER2
		PERIPH("NRF_TIMER2", NRF_TIMER2, CONFIG_SPM_NRF_TIMER2_NS),
#endif
#ifdef NRF_SAADC
		PERIPH("NRF_SAADC", NRF_SAADC, CONFIG_SPM_NRF_SAADC_NS),
#endif
#ifdef NRF_PWM0
		PERIPH("NRF_PWM0", NRF_PWM0, CONFIG_SPM_NRF_PWM0_NS),
#endif
#ifdef NRF_PWM1
		PERIPH("NRF_PWM1", NRF_PWM1, CONFIG_SPM_NRF_PWM1_NS),
#endif
#ifdef NRF_PWM2
		PERIPH("NRF_PWM2", NRF_PWM2, CONFIG_SPM_NRF_PWM2_NS),
#endif
#ifdef NRF_PWM3
		PERIPH("NRF_PWM3", NRF_PWM3, CONFIG_SPM_NRF_PWM3_NS),
#endif
#ifdef NRF_WDT
		PERIPH("NRF_WDT", NRF_WDT, CONFIG_SPM_NRF_WDT_NS),
#endif
		/* There is no DTS node for the peripherals below,
		 * so address them using nrfx macros directly.
		 */
		PERIPH("NRF_IPC", NRF_IPC_S, CONFIG_SPM_NRF_IPC_NS),
		PERIPH("NRF_VMC", NRF_VMC_S, CONFIG_SPM_NRF_VMC_NS),
		PERIPH("NRF_FPU", NRF_FPU_S, CONFIG_SPM_NRF_FPU_NS),
		PERIPH("NRF_EGU1", NRF_EGU1_S, CONFIG_SPM_NRF_EGU1_NS),
		PERIPH("NRF_EGU2", NRF_EGU2_S, CONFIG_SPM_NRF_EGU2_NS),
		PERIPH("NRF_DPPIC", NRF_DPPIC_S, CONFIG_SPM_NRF_DPPIC_NS),

		PERIPH("NRF_GPIOTE1", NRF_GPIOTE1_NS,
				      CONFIG_SPM_NRF_GPIOTE1_NS),
		PERIPH("NRF_REGULATORS", NRF_REGULATORS_S,
				      CONFIG_SPM_NRF_REGULATORS_NS),
	};

	if (IS_ENABLED(CONFIG_SPM_NRF_DPPIC_NS)) {
		spm_dppi_configure(CONFIG_SPM_NRF_DPPIC_PERM_MASK);
	}

	PRINT("Peripheral\t\tDomain\t\tStatus\n");

	if (IS_ENABLED(CONFIG_SPM_NRF_P0_NS)) {
		/* Configure GPIO pins to be Non-Secure */
		NRF_SPU->GPIOPORT[0].PERM = 0;
		NRF_SPU->GPIOPORT[1].PERM = 0;
	}

	for (size_t i = 0; i < ARRAY_SIZE(periph); i++) {
		int err;

#ifndef CONFIG_SPM_BOOT_SILENTLY
		PRINT("%02u %-21s%s", i, periph[i].name,
		      periph[i].nonsecure ? "Non-Secure" : "Secure\t");
#endif

		if (!periph[i].nonsecure) {
			PRINT("\tSKIP\n");
			continue;
		}

		err = spm_config_peripheral(periph[i].id, false);
		if (err) {
			PRINT("\tERROR\n");
		} else {
			PRINT("\tOK\n");
		}
	}
	PRINT("\n");
}


static void spm_configure_ns(const tz_nonsecure_setup_conf_t
	*spm_ns_conf)
{
	/* Configure core register block for Non-Secure state. */
	tz_nonsecure_state_setup(spm_ns_conf);
	/* Prioritize Secure exceptions over Non-Secure */
	tz_nonsecure_exception_prio_config(1);
	/* Set non-banked exceptions to target Non-Secure */
	tz_nbanked_exception_target_state_set(0);
	/* Configure if Non-Secure firmware should be allowed to issue System
	 * reset. If not it could be enabled through a secure service.
	 */
	tz_nonsecure_system_reset_req_block(
		IS_ENABLED(CONFIG_SPM_BLOCK_NON_SECURE_RESET)
	);
	/* Allow SPU to have precedence over (non-existing) ARMv8-M SAU. */
	tz_sau_configure(0, 1);

#if defined(CONFIG_ARMV7_M_ARMV8_M_FP) && defined(CONFIG_SPM_NRF_FPU_NS)
	/* Allow Non-Secure firmware to use the FPU */
	tz_nonsecure_fpu_access_enable();
#endif /* CONFIG_ARMV7_M_ARMV8_M_FP */
}

void spm_jump(void)
{
	/* Extract initial MSP of the Non-Secure firmware image.
	 * The assumption is that the MSP is located at VTOR_NS[0].
	 */
	uint32_t *vtor_ns = (uint32_t *)NON_SECURE_APP_ADDRESS;

	PRINT("SPM: NS image at 0x%x\n", (uint32_t)vtor_ns);
	PRINT("SPM: NS MSP at 0x%x\n", vtor_ns[0]);
	PRINT("SPM: NS reset vector at 0x%x\n", vtor_ns[1]);

	/* Configure Non-Secure stack */
	tz_nonsecure_setup_conf_t spm_ns_conf = {
		.vtor_ns = (uint32_t)vtor_ns,
		.msp_ns = vtor_ns[0],
		.psp_ns = 0,
		.control_ns.npriv = 0, /* Privileged mode*/
		.control_ns.spsel = 0 /* Use MSP in Thread mode */
	};

	spm_configure_ns(&spm_ns_conf);

	/* Generate function pointer for Non-Secure function call. */
	TZ_NONSECURE_FUNC_PTR_DECLARE(reset_ns);
	reset_ns = TZ_NONSECURE_FUNC_PTR_CREATE(vtor_ns[1]);

	if (TZ_NONSECURE_FUNC_PTR_IS_NS(reset_ns)) {
		PRINT("SPM: prepare to jump to Non-Secure image.\n");

		/* Note: Move UARTE0 before jumping, if it is
		 * to be used on the Non-Secure domain.
		 */

		/* Configure UARTE0 as non-secure */
		spm_config_peripheral(
			NRFX_PERIPHERAL_ID_GET(NRF_UARTE0), 0);

		__DSB();
		__ISB();

		/* Jump to Non-Secure firmware */
		reset_ns();

		CODE_UNREACHABLE;

	} else {
		PRINT("SPM: wrong pointer type: 0x%x\n",
		      (uint32_t)reset_ns);
	}
}

void spm_config(void)
{
	spm_config_flash();
	spm_config_sram();
	spm_config_peripherals();
}

This enables P1.x pins for non-secure use, which should fix the problem.

Best regards,
Carl Richard

[GiulianoFranchetto]

Hi,

I just tried it. I now have a crash on startup:

[00:00:00.002,685] <err> os: ***** BUS FAULT *****
[00:00:00.002,685] <err> os:   Precise data bus error
[00:00:00.002,685] <err> os:   BFAR Address: 0x4002b524
[00:00:00.002,685] <err> os: r0/a1:  0x2000f370  r1/a2:  0x0001f82f  r2/a3:  0x200083b8
[00:00:00.002,685] <err> os: r3/a4:  0x4002b000 r12/ip:  0x00000012 r14/lr:  0x0000001
[00:00:00.002,716] <err> os:  xpsr:  0x81000000
[00:00:00.002,716] <err> os: Faulting instruction address (r15/pc): 0x000184ec
[00:00:00.002,716] <err> os: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0
[00:00:00.002,716] <err> os: Current thread: 0x200091b8 (unknown)
[00:00:00.657,623] <err> fatal_error: Resetting system[0m

[GiulianoFranchetto]

Apparently, this is not related to TWIM issue, but QSPI.

A crash is happenig here: 

nrf_qspi_pins_set(NRF_QSPI, p_config);
    000184CA   LDR          R3, [PC, #0xBC]            
    000184CC   STR.W        LR, [R3, #0x524]
p_reg->PSEL.CSN = NRF_QSPI_PIN_VAL(p_pins->csn_pin);

I've tried to add in smp.c

#ifdef NRF_QSPI
		PERIPH("NRF_QSPI", NRF_QSPI, CONFIG_SPM_NRF_QSPI_NS),
#endif

but no luck 

When I disable QSPI, the crash disapears, and the TWI is indeed working So P1 not being accessible from not secure was indeed the issue.

But apparently, this issue is also the same with QSPI. Correct?