Cannot loop send() when trying to send large data in https_client sample on nRF9160

[OYAMA YUKIO]

Hi everyone,

I want to send large data with https protocol.

I edited the https_client sample split the data and looped send() for sending the large data.

But when I did send() once, I got error 128. Did the socket close by itself?

After I run send() once, I run close() -> tls_setup() ->connect() again. Then I can do send() again.

This is very time consuming, in a single loop, reconnecting takes 2 or 3 seconds in my environment. So I need a different solution.

Is it possible to loop send() continuously with the socket open? In the past, there have been similar case with the http protocol, but it seems that closing the socket with every send() is the solution.

https://devzone.nordicsemi.com/f/nordic-q-a/46086/nrf9160-dk-http-post-to-my-webserver

Is this still not improving? If I can send quickly even if I close the socket, it may be a solution.

HW:nRF9160DK

FW:modem v1.3.1

SDK v1.9.1

My LTE connection is LTE-M.

Best Regards,

Yukio Oyama

[Einarh]

Hello

Looking at the https client sample, it looks like the code is supposed to support sending multiple packets before closing the socket.

I assume the changes you've made is to limit how many bytes you're allowed to send in each packet? Can you show me the exact changes you've made to the sample?

Best regards,

Einar

[OYAMA YUKIO]

Hello Einar-san,

This is the minimum set with errors. (Some APN information was hidden.)

To find the cause, I searched for the smallest change where the error occurred. When sending twice (not loop), an error will occur on the second sending.

This may be due to server behavior rather than device, but I'm not familiar with either.

If a socket has been a timeout from open or access, can it be controlled by a parameter in the code?

4263.prj.conf

Fullscreen 5734.main.c Download

/*
 * Copyright (c) 2020 Nordic Semiconductor ASA
 *
 * SPDX-License-Identifier: LicenseRef-Nordic-5-Clause
 */

#include <string.h>
#include <zephyr.h>
#include <stdlib.h>
#include <net/socket.h>
#include <modem/nrf_modem_lib.h>
#include <net/tls_credentials.h>
#include <modem/lte_lc.h>
#include <modem/modem_key_mgmt.h>

#define _HTTPS_POST_      1	// Send https to the test server.(1:enable)
#define _SIM_SOFTBANK_EN_ 1 // Use softbank sim.(1:enable)
#define _SECOND_SEND_     1 // Second transmission.(1:enable)

#define HTTPS_PORT 443

#if _HTTPS_POST_
#define HTTPS_HOSTNAME "httpbin.org"
#else
#define HTTPS_HOSTNAME "example.com"
#endif

#if _HTTPS_POST_
#define HTTP_HEAD                      \
	"POST /post HTTP/1.1\r\n"              \
	"Host: " HTTPS_HOSTNAME ":443\r\n" \
	"Content-Type: text/html\r\n"	   \
	"Content-Length: 5\r\n"            \
	"\r\nhello\r\n"
#else
#define HTTP_HEAD                                                              \
	"HEAD / HTTP/1.1\r\n"                                                  \
	"Host: " HTTPS_HOSTNAME ":443\r\n"                                     \
	"Connection: close\r\n\r\n"
#endif


#define HTTP_HEAD_LEN (sizeof(HTTP_HEAD) - 1)

#define HTTP_HDR_END "\r\n\r\n"

#define RECV_BUF_SIZE 2048
#define TLS_SEC_TAG 42

static const char send_buf[] = HTTP_HEAD;
static char recv_buf[RECV_BUF_SIZE];

/* Certificate for `example.com` */
static const char cert[] = {
#if _HTTPS_POST_
	#include "../cert/startfield_class2.pem"	// "httpbin.org"
#else
	#include "../cert/DigiCertGlobalRootCA.pem"
#endif
};

BUILD_ASSERT(sizeof(cert) < KB(4), "Certificate too large");

#if _SIM_SOFTBANK_EN_
#define CMD_CGDCONT "AT+CGDCONT=0,\"IPV4V6\",\"APN_name\""
#define CMD_CGAUTH  "AT+CGAUTH=0,1,\"username\",\"password\""
#define AT_MAX_CMD_LEN          2048
static uint8_t at_buf[AT_MAX_CMD_LEN];
#endif

/* Provision certificate to modem */
int cert_provision(void)
{
	int err;
	bool exists;
	int mismatch;

	/* It may be sufficient for you application to check whether the correct
	 * certificate is provisioned with a given tag directly using modem_key_mgmt_cmp().
	 * Here, for the sake of the completeness, we check that a certificate exists
	 * before comparing it with what we expect it to be.
	 */
	err = modem_key_mgmt_exists(TLS_SEC_TAG, MODEM_KEY_MGMT_CRED_TYPE_CA_CHAIN, &exists);
	if (err) {
		printk("Failed to check for certificates err %d\n", err);
		return err;
	}

	if (exists) {
		mismatch = modem_key_mgmt_cmp(TLS_SEC_TAG,
					      MODEM_KEY_MGMT_CRED_TYPE_CA_CHAIN,
					      cert, strlen(cert));
		if (!mismatch) {
			printk("Certificate match\n");
			return 0;
		}

		printk("Certificate mismatch\n");
		err = modem_key_mgmt_delete(TLS_SEC_TAG, MODEM_KEY_MGMT_CRED_TYPE_CA_CHAIN);
		if (err) {
			printk("Failed to delete existing certificate, err %d\n", err);
		}
	}

	printk("Provisioning certificate\n");

	/*  Provision certificate to the modem */
	err = modem_key_mgmt_write(TLS_SEC_TAG,
				   MODEM_KEY_MGMT_CRED_TYPE_CA_CHAIN,
				   cert, sizeof(cert) - 1);
	if (err) {
		printk("Failed to provision certificate, err %d\n", err);
		return err;
	}

	return 0;
}

/* Setup TLS options on a given socket */
int tls_setup(int fd)
{
	int err;
	int verify;

	/* Security tag that we have provisioned the certificate with */
	const sec_tag_t tls_sec_tag[] = {
		TLS_SEC_TAG,
	};

#if defined(CONFIG_SAMPLE_TFM_MBEDTLS)
	err = tls_credential_add(tls_sec_tag[0], TLS_CREDENTIAL_CA_CERTIFICATE, cert, sizeof(cert));
	if (err) {
		return err;
	}
#endif

	/* Set up TLS peer verification */
	enum {
		NONE = 0,
		OPTIONAL = 1,
		REQUIRED = 2,
	};

//	verify = REQUIRED;
	verify = NONE;

	err = setsockopt(fd, SOL_TLS, TLS_PEER_VERIFY, &verify, sizeof(verify));
	if (err) {
		printk("Failed to setup peer verification, err %d\n", errno);
		return err;
	}

	/* Associate the socket with the security tag
	 * we have provisioned the certificate with.
	 */
	err = setsockopt(fd, SOL_TLS, TLS_SEC_TAG_LIST, tls_sec_tag,
			 sizeof(tls_sec_tag));
	if (err) {
		printk("Failed to setup TLS sec tag, err %d\n", errno);
		return err;
	}

	err = setsockopt(fd, SOL_TLS, TLS_HOSTNAME, HTTPS_HOSTNAME, sizeof(HTTPS_HOSTNAME) - 1);
	if (err) {
		printk("Failed to setup TLS hostname, err %d\n", errno);
		return err;
	}
	return 0;
}

void main(void)
{
	int err;
	int fd;
	char *p;
	int bytes;
	size_t off;
	struct addrinfo *res;
	struct addrinfo hints = {
		.ai_family = AF_INET,
		.ai_socktype = SOCK_STREAM,
	};

	printk("HTTPS client sample started\n\r");

#if _SIM_SOFTBANK_EN_
	err = nrf_modem_at_cmd(recv_buf, sizeof(recv_buf), "%s", CMD_CGDCONT);
	printk("AT+CGDCONT err:%x, %s\n\r",err, recv_buf);
	err = nrf_modem_at_cmd(recv_buf, sizeof(recv_buf), "%s", CMD_CGAUTH);
	printk("AT+CGDCONT err:%x, %s\n\r",err, recv_buf);
#endif	//_SIM_SOFTBANK_EN_

#if !defined(CONFIG_SAMPLE_TFM_MBEDTLS)
	/* Provision certificates before connecting to the LTE network */
	err = cert_provision();
	if (err) {
		return;
	}
#endif

	printk("Waiting for network.. ");
	err = lte_lc_init_and_connect();
	if (err) {
		printk("Failed to connect to the LTE network, err %d\n", err);
		return;
	}
	printk("OK\n");

	err = getaddrinfo(HTTPS_HOSTNAME, NULL, &hints, &res);
	if (err) {
		printk("getaddrinfo() failed, err %d\n", errno);
		return;
	}

	((struct sockaddr_in *)res->ai_addr)->sin_port = htons(HTTPS_PORT);

	if (IS_ENABLED(CONFIG_SAMPLE_TFM_MBEDTLS)) {
		fd = socket(AF_INET, SOCK_STREAM | SOCK_NATIVE_TLS, IPPROTO_TLS_1_2);
	} else {
		fd = socket(AF_INET, SOCK_STREAM, IPPROTO_TLS_1_2);
	}
	if (fd == -1) {
		printk("Failed to open socket!\n");
		goto clean_up;
	}

	/* Setup TLS socket options */
	err = tls_setup(fd);
	if (err) {
		goto clean_up;
	}

	printk("Connecting to %s\n", HTTPS_HOSTNAME);
	err = connect(fd, res->ai_addr, sizeof(struct sockaddr_in));
	if (err) {
		printk("connect() failed, err: %d\n", errno);
		goto clean_up;
	}

	off = 0;
	do {
		bytes = send(fd, &send_buf[off], HTTP_HEAD_LEN - off, 0);
		if (bytes < 0) {
			printk("send() failed, err %d\n", errno);
			goto clean_up;
		}
		off += bytes;
	} while (off < HTTP_HEAD_LEN);

	printk("Sent %d bytes\n", off);

	off = 0;
	do {
		bytes = recv(fd, &recv_buf[off], RECV_BUF_SIZE - off, 0);
		if (bytes < 0) {
			printk("recv() failed, err %d\n", errno);
			goto clean_up;
		}
		off += bytes;
	} while (bytes != 0 /* peer closed connection */);

	printk("Received %d bytes\n", off);

	/* Print HTTP response */
	p = strstr(recv_buf, "\r\n");
	if (p) {
		off = p - recv_buf;
		recv_buf[off + 1] = '\0';
		printk("\n>\t %s\n\n", recv_buf);
	}

#if _SECOND_SEND_
	off = 0;
	do {
		bytes = send(fd, &send_buf[off], HTTP_HEAD_LEN - off, 0);
		if (bytes < 0) {
			printk("send() failed, err %d\n", errno);
			goto clean_up;
		}
		off += bytes;
	} while (off < HTTP_HEAD_LEN);
	printk("Sent %d bytes\n", off);

	off = 0;
	do {
		bytes = recv(fd, &recv_buf[off], RECV_BUF_SIZE - off, 0);
		if (bytes < 0) {
			printk("recv() failed, err %d\n", errno);
			goto clean_up;
		}
		off += bytes;
	} while (bytes != 0 /* peer closed connection */);

	printk("Received %d bytes\n", off);

	p = strstr(recv_buf, "\r\n");
	if (p) {
		off = p - recv_buf;
		recv_buf[off + 1] = '\0';
		printk("\n>\t %s\n\n", recv_buf);
	}


#endif

	printk("Finished, closing socket.\n");

clean_up:
	freeaddrinfo(res);
	(void)close(fd);

	lte_lc_power_off();
}

In my experiment, send_buff [] is fixed and small in size. Finally, in my project I want to split the image file every 2KB (or less) and upload it by looping the send() .

Best Regards,

Yukio Oyama

[OYAMA YUKIO]

Hi Einar-san,

I haven't answered the previous question, but I have additional questions.

I was focusing on send() to send large data quickly, but I found that recv() also takes time.

When I run recv(), it takes 5-7 seconds to respond.

Looking at the receive buffer, the arrival of data by recv() is fast enough that I think the time extension to the disconnect operation.

I don't expect a disconnect with recv() because the server is keep-alive, but you said the disconnect after recv() was fine.

However, I feel that the time it takes to disconnect is long.(5-7 seconds)

Is this correct? Or is it extended with some error due to an unexpected disconnection?

If due to an error, is there a function that actively disconnects to avoid the error and reduce its time?

Best Regards,

Yukio Oyama

[Einarh]

My sending data size is not fixed, it is dangerous to set the waiting time to a fixed value.

One solution then would be to always wait as if you're sending the longest packet you'll be sending, but I agree this is not optimal.

Can I receive AT notifications after using send()?

I believe the send() function is actually sending AT-commands to the modem under the hood, which would mean the AT notifications are also there, you'd just have to catch them in your application.

does the chunk mode of AT#XHTTPCREQ deal with the buffer problem for repeated sending without any countermeasure?

Have you tried using this AT command? I thought these SLM-specific AT commands were specifically for the Serial LTE Modem application. Though if it works it works, never hurts to try.

Is there an example of AT command execution in chunk mode?

Not that I know of.

When I run recv(), it takes 5-7 seconds to respond.

Sounds strange, but I could imagine it being due to something like the client not expecting the server to be keep-alive, and waiting for it to terminate the connection?

Do you know exactly what function call is taking long? Is it recv()?

You can probably set the timeout to be shorter somehow, maybe with a flag in the recv command.

It might also be the case that you could set an option for the socket to prevent recv from waiting for a message if no message is available:

If no messages are available at the socket and O_NONBLOCK is not set on the socket's file descriptor, recv() shall block until a message arrives.

Best regards,

Einar

[OYAMA YUKIO]

Hi Einar-san,

I haven't successfully changed the code to receive AT notifications, but I'm still trying it.

1, According to your link,

setsockopt () has a SO_KEEPALIVE option whose initial value seems to be off.

Do I need to turn this option on to keep keep-alive in http protocol?

2. Native TLS does not support session resumption according to this thread and this URL.

I don't understand the configuration of the Config file, so in the https_client sample I can't understand whether TLS is offloaded to the modem or using native TLS. I'm based on the https_client sample, will TLS be offloaded to the modem? I haven't made any TLS related changes in the cfg file. The TLS settings in the cfg file are the same as the https_client sample settings.

3,Today, I used the Modem shell App and iperf3 to check the effective speed of the uplink in my environment.

It was 100K ~ 180Kbps.

Fullscreen 220523_nRF9160_ModemShell_iPerf3.txt Download

*** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***
Flash regions		Domain		Permissions
00 01 0x00000 0x10000 	Secure		rwxl
02 31 0x10000 0x100000 	Non-Secure	rwxl

Non-secure callable region 0 placed in flash region 1 with size 32.

SRAM region		Domain		Permissions
00 07 0x00000 0x10000 	Secure		rwxl
08 31 0x10000 0x40000 	Non-Secure	rwxl

Peripheral		Domain		Status
00 NRF_P0               Non-Secure	OK
01 NRF_CLOCK            Non-Secure	OK
02 NRF_RTC0             Non-Secure	OK
03 NRF_RTC1             Non-Secure	OK
04 NRF_NVMC             Non-Secure	OK
05 NRF_UARTE1           Non-Secure	OK
06 NRF_UARTE2           Secure		SKIP
07 NRF_TWIM2            Non-Secure	OK
08 NRF_SPIM3            Non-Secure	OK
09 NRF_TIMER0           Non-Secure	OK
10 NRF_TIMER1           Non-Secure	OK
11 NRF_TIMER2           Non-Secure	OK
12 NRF_SAADC            Non-Secure	OK
13 NRF_PWM0             Non-Secure	OK
14 NRF_PWM1             Non-Secure	OK
15 NRF_PWM2             Non-Secure	OK
16 NRF_PWM3             Non-Secure	OK
17 NRF_WDT              Non-Secure	OK
18 NRF_IPC              Non-Secure	OK
19 NRF_VMC              Non-Secure	OK
20 NRF_FPU              Non-Secure	OK
21 NRF_EGU1             Non-Secure	OK
22 NRF_EGU2             Non-Secure	OK
23 NRF_DPPIC            Non-Secure	OK
24 NRF_REGULATORS       Non-Secure	OK
25 NRF_PDM              Non-Secure	OK
26 NRF_I2S              Non-Secure	OK
27 NRF_GPIOTE1          Non-Secure	OK

SPM: NS image at 0x10000
SPM: NS MSP at 0x200294f0
SPM: NS reset vector at 0x24c69
SPM: prepare to jump to Non-Secure image.
*** Booting Zephyr OS build v2.6.0-rc1-ncs1  ***

MOSH version:       v1.6.0
MOSH build id:      custom
MOSH build variant: dev

Initializing modemlib...


mosh:~$ Initialized modemlib
Network registration status: searching
mosh:~$ LTE cell changed: Cell ID: 77815387, Tracking area: 9252
mosh:~$ Currently active system mode: LTE-M
mosh:~$ RRC mode: Connected
mosh:~$ RRC mode: Idle
mosh:~$ at at+cfun?
+CFUN: 1
OK
mosh:~$ at at+cfun=0
Network registration status: not registered
OK
mosh:~$ at at+cfun=1
OK
mosh:~$ at at+cereg?
+CEREG: 0,1,"2424","04A35E5B",7
OK
mosh:~$ at at+cgatt?
+CGATT: 1
OK
mosh:~$ at at+cgact?
+CGACT: 0,1
OK
mosh:~$ at at+cops?
+COPS: 0,2,"44020",7
OK
mosh:~$ iperf3 -c speedtest.serverius.net -p 5002 -l 705 -t 30
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Socket is not connected
iperf3: error - unable to send control message: Socket is not connected
iperf Failed.
mosh:~$ sock connect -a speedtest.serverius.net -p 5002
Socket open and connect family=1, type=1, port=5002, bind_port=0, pdn_cid=0, address=speedtest.serverius.net
Socket created socket_id=0, fd=4
mosh:~$ Received data for socket socket_id=0, buffer_size=1:
mosh:~$ 	�
mosh:~$ Socket id=0 (fd=4) disconnected so closing.
mosh:~$ sock list
There are no open sockets
mosh:~$ sock listsock connect -a speedtest.serverius.net -p 5002
Socket open and connect family=1, type=1, port=5002, bind_port=0, pdn_cid=0, address=speedtest.serverius.net
Socket created socket_id=0, fd=4
mosh:~$ Received data for socket socket_id=0, buffer_size=1:
mosh:~$ 	�
mosh:~$ Socket id=0 (fd=4) disconnected so closing.
mosh:~$ sock connect -a speedtest.serverius.net -p 5002
Socket open and connect family=1, type=1, port=5002, bind_port=0, pdn_cid=0, address=speedtest.serverius.net
Socket created socket_id=0, fd=4
mosh:~$ Received data for socket socket_id=0, buffer_size=1:
mosh:~$ 	�
mosh:~$ Socket id=0 (fd=4) disconnected so closing.
mosh:~$ sock connect -a speedtest.serverius.net -p 5002sock listsock connect -a speedtest.serverius.net -p 5002iperf3 -c speedtest.serverius.net -p 5002 -l 705 -t 30
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Connection reset by peer
iperf3: error - unable to send control message: Connection reset by peer
iperf Failed.
mosh:~$ iperf3 -c speedtest.serverius.net -p 5002 -l 705 -t 30
iperf3: error - the server is busy running a test. try again later
iperf3: error - the server is busy running a test. try again later
iperf Failed.
mosh:~$ iperf3 -c speedtest.serverius.net -p 5002 -l 705 -t 30
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Socket is not connected
iperf3: error - unable to send control message: Socket is not connected
iperf Failed.
mosh:~$ iperf3 -c speedtest.serverius.net -p 5002 -l 705 -t 301024
iperf3: error - the server is busy running a test. try again later
iperf3: error - the server is busy running a test. try again later
iperf Failed.
mosh:~$ iperf3 -c speedtest.serverius.net -p 5002 -l 1024 -l 1024 -l 1024 -l 10242 -l 10240 -l 10241 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024.net -p 5201 -l 1024i.net -p 5201 -l 1024p.net -p 5201 -l 1024e.net -p 5201 -l 1024r.net -p 5201 -l 1024f.net -p 5201 -l 1024..net -p 5201 -l 1024h.net -p 5201 -l 1024e.net -p 5201 -l 1024
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Bad file number
iperf3: error - unable to send control message: Bad file number
iperf Failed.
mosh:~$ iperf3 -c iperf.he.net -p 5201 -l 1024
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Bad file number
iperf3: error - unable to send control message: Bad file number
iperf Failed.
mosh:~$ iperf3 -c iperf.he.net -p 5201 -l 1024iperf.he.net -p 5201 -l 1024iperf.he.net -p 5201 -l 1024iperf.he.net -p 5201 -l 1024
mosh:~$ iperf3 iperf.biznetnetworks.com -l 1024-iperf.biznetnetworks.com -l 1024ciperf.biznetnetworks.com -l 1024 iperf.biznetnetworks.com -l 1024
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Connection reset by peer
iperf3: error - unable to send control message: Connection reset by peer
iperf Failed.
mosh:~$ iperf3 -c iperf.biznetnetworks.com -l 1024
Connecting to host iperf.biznetnetworks.com, port 5201
[  5] local 0.0.0.0 port 0 connected to 117.102.109.186 port 5201
stream [0]: socket: 5 read: 0 write: 1
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec  15.2 KBytes   123 Kbits/sec                  
[  5]   1.01-2.08   sec  22.1 KBytes   171 Kbits/sec                  
[  5]   2.08-3.10   sec  23.5 KBytes   188 Kbits/sec                  
[  5]   3.10-4.06   sec  22.1 KBytes   188 Kbits/sec                  
[  5]   4.06-5.10   sec  22.4 KBytes   178 Kbits/sec                  
[  5]   5.10-6.11   sec  23.5 KBytes   190 Kbits/sec                  
[  5]   6.11-7.11   sec  23.5 KBytes   193 Kbits/sec                  
[  5]   7.11-8.05   sec  22.1 KBytes   193 Kbits/sec                  
[  5]   8.05-9.01   sec  23.5 KBytes   200 Kbits/sec                  
[  5]   9.01-10.13  sec  24.2 KBytes   178 Kbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.13  sec   222 KBytes   180 Kbits/sec                  sender
[  5]   0.00-10.13  sec   222 KBytes   180 Kbits/sec                  receiver
iperf Done.
mosh:~$ iperf3 -c iperf.biznetnetworks.com -l 1024 -R
Connecting to host iperf.biznetnetworks.com, port 5201
Reverse mode, remote host iperf.biznetnetworks.com is sending
[  5] local 0.0.0.0 port 0 connected to 117.102.109.186 port 5201
stream [0]: socket: 5 read: 1 write: 0
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  21.4 KBytes   175 Kbits/sec                  
[  5]   1.00-2.00   sec  22.1 KBytes   181 Kbits/sec                  
[  5]   2.00-3.00   sec  24.2 KBytes   198 Kbits/sec                  
[  5]   3.00-4.00   sec  22.8 KBytes   187 Kbits/sec                  
[  5]   4.00-5.00   sec  22.1 KBytes   181 Kbits/sec                  
[  5]   5.00-6.00   sec  21.4 KBytes   176 Kbits/sec                  
[  5]   6.00-7.00   sec  24.2 KBytes   198 Kbits/sec                  
[  5]   7.00-8.00   sec  23.5 KBytes   193 Kbits/sec                  
[  5]   8.00-9.00   sec  22.1 KBytes   181 Kbits/sec                  
[  5]   9.00-10.00  sec  23.5 KBytes   193 Kbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   260 KBytes   213 Kbits/sec    0             sender
[  5]   0.00-10.00  sec   227 KBytes   186 Kbits/sec                  receiver
iperf Done.
mosh:~$ iperf3 -c iperf.biznetnetworks.com -l 1024 -Riperf3 -c iperf.biznetnetworks.com -l 1024
iperf3: error - the server is busy running a test. try again later
iperf3: error - the server is busy running a test. try again later
iperf Failed.
mosh:~$ iperf3 -c iperf.biznetnetworks.com -l 1024
iperf3: error - the server is busy running a test. try again later
iperf3: error - the server is busy running a test. try again later
iperf Failed.
mosh:~$ iperf3 -c iperf.biznetnetworks.com -l 1024
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Socket is not connected
iperf3: error - unable to send control message: Socket is not connected
iperf Failed.
mosh:~$ iperf3 -c iperf.biznetnetworks.com -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024i -l 1024p -l 1024e -l 1024r -l 1024f -l 1024. -l 1024i -l 1024t -l 1024- -l 1024n -l 1024o -l 1024r -l 1024t -l 1024h -l 1024. -l 1024n -l 1024e -l 1024t -l 1024
getaddrinfo failed with error code -11 EAI_SYSTEM
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Bad file number
iperf3: error - unable to send control message: Bad file number
iperf Failed.
mosh:~$ iperf3 -c iperf.it-north.net -l 1024
getaddrinfo failed with error code -11 EAI_SYSTEM
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Bad file number
iperf3: error - unable to send control message: Bad file number
iperf Failed.
mosh:~$ iperf3 -c speedtest.serverius.net -p 5002 -l 705 -t 30
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Socket is not connected
iperf3: error - unable to send control message: Socket is not connected
iperf Failed.
mosh:~$ iperf3 -c speedtest.serverius.net -p 5002 -l 705 -t 30
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Socket is not connected
iperf3: error - unable to send control message: Socket is not connected
iperf Failed.
mosh:~$ iperf3 -c speedtest.serverius.net -p 5002 -l 705 -t 30iperf3 -c iperf.it-north.net -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024 -l 1024s -l 1024p -l 1024e -l 1024e -l 1024d -l 1024t -l 1024e -l 1024s -l 1024t -l 1024- -l 1024i -l 1024p -l 1024e -l 1024r -l 1024f -l 1024- -l 1024a -l 1024k -l 1024l -l 1024. -l 1024v -l 1024e -l 1024t -l 1024t -l 1024a -l 1024. -l 1024o -l 1024n -l 1024l -l 1024i -l 1024n -l 1024e -l 1024
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Bad file number
iperf3: error - unable to send control message: Bad file number
iperf Failed.
mosh:~$ iperf3 -c speedtest-iperf-akl.vetta.online -l 1024
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Bad file number
iperf3: error - unable to send control message: Bad file number
iperf Failed.
mosh:~$ iperf3 -c speedtest-iperf-akl.vetta.online -l 1024iperf3 -c speedtest.serverius.net -p 5002 -l 705 -t 30iperf3 -c iperf.it-north.net -l 1024iperf3 -c iperf.biznetnetworks.com -l 1024
iperf3: error - the server is busy running a test. try again later
iperf3: error - the server is busy running a test. try again later
iperf Failed.
mosh:~$ 
mosh:~$ iperf3 -c iperf.biznetnetworks.com -l 1024
Connecting to host iperf.biznetnetworks.com, port 5201
[  5] local 0.0.0.0 port 0 connected to 117.102.109.186 port 5201
stream [0]: socket: 5 read: 0 write: 1
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec  7.61 KBytes  61.8 Kbits/sec                  
[  5]   1.01-2.00   sec  18.0 KBytes   149 Kbits/sec                  
[  5]   2.00-3.00   sec  23.1 KBytes   189 Kbits/sec                  
[  5]   3.00-4.00   sec  22.5 KBytes   184 Kbits/sec                  
[  5]   4.00-5.00   sec  22.1 KBytes   181 Kbits/sec                  
[  5]   5.00-6.00   sec  23.5 KBytes   193 Kbits/sec                  
[  5]   6.00-7.00   sec  23.5 KBytes   193 Kbits/sec                  
[  5]   7.00-8.00   sec  23.5 KBytes   193 Kbits/sec                  
[  5]   8.00-9.00   sec  23.5 KBytes   193 Kbits/sec                  
[  5]   9.00-10.00  sec  22.7 KBytes   186 Kbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec   210 KBytes   172 Kbits/sec                  sender
[  5]   0.00-10.00  sec   210 KBytes   172 Kbits/sec                  receiver
iperf Done.
mosh:~$ 
mosh:~$ iperf3 -c iperf.biznetnetworks.com -l 1024iperf3 -c speedtest-iperf-akl.vetta.online -l 1024iperf3 -c speedtest.serverius.net -p 5002 -l 705 -t 30
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Socket is not connected
iperf3: error - unable to send control message: Socket is not connected
iperf Failed.
mosh:~$ 
mosh:~$ iperf3 --client ping.online.net --port 5202 -l 3540 --time 30 -V -R
iperf 3.9-nrf
nrf9160
Control connection MSS: using modem default
iperf_client_end: iperf_set_send_state failed
iperf3: error - unable to send control message: Socket is not connected
iperf3: error - unable to send control message: Socket is not connected
iperf Failed.
mosh:~$ iperf3 --client ping.online.net --port 5202 -l 3540 --time 30 -V -R
iperf 3.9-nrf
nrf9160
Control connection MSS: using modem default
Time: 22/05/23,06:07:06+36
Connecting to host ping.online.net, port 5202
Reverse mode, remote host ping.online.net is sending
      Cookie: nrf9160.6490.844024.6db505597d157d8d
      TCP MSS: 0 (default)
[  5] local 0.0.0.0 port 0 connected to 62.210.18.40 port 5202
stream [0]: socket: 5 read: 1 write: 0
Starting Test: protocol: TCP, 1 streams, 3540 byte blocks, omitting 0 seconds, 30 second test, tos 0
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec  2.77 KBytes  22.4 Kbits/sec                  
[  5]   1.01-2.01   sec  15.2 KBytes   125 Kbits/sec                  
[  5]   2.01-3.01   sec  17.3 KBytes   142 Kbits/sec                  
[  5]   3.01-4.01   sec  15.2 KBytes   125 Kbits/sec                  
[  5]   4.01-5.01   sec  16.6 KBytes   136 Kbits/sec                  
[  5]   5.01-6.01   sec  17.3 KBytes   142 Kbits/sec                  
[  5]   6.01-7.01   sec  15.2 KBytes   125 Kbits/sec                  
[  5]   7.01-8.01   sec  15.9 KBytes   130 Kbits/sec                  
[  5]   8.01-9.01   sec  14.5 KBytes   119 Kbits/sec                  
[  5]   9.01-10.01  sec  16.6 KBytes   136 Kbits/sec                  
[  5]  10.01-11.01  sec  14.5 KBytes   119 Kbits/sec                  
[  5]  11.01-12.01  sec  17.3 KBytes   141 Kbits/sec                  
[  5]  12.01-13.01  sec  13.8 KBytes   113 Kbits/sec                  
[  5]  13.01-14.01  sec  16.6 KBytes   136 Kbits/sec                  
[  5]  14.01-15.01  sec  17.3 KBytes   142 Kbits/sec                  
[  5]  15.01-16.01  sec  15.9 KBytes   130 Kbits/sec                  
[  5]  16.01-17.01  sec  15.9 KBytes   130 Kbits/sec                  
[  5]  17.01-18.01  sec  16.6 KBytes   136 Kbits/sec                  
[  5]  18.01-19.01  sec  16.6 KBytes   136 Kbits/sec                  
[  5]  19.01-20.01  sec  15.9 KBytes   130 Kbits/sec                  
[  5]  20.01-21.01  sec  15.9 KBytes   130 Kbits/sec                  
[  5]  21.01-22.01  sec  16.6 KBytes   136 Kbits/sec                  
[  5]  22.01-23.01  sec  15.9 KBytes   130 Kbits/sec                  
[  5]  23.01-24.01  sec  18.0 KBytes   147 Kbits/sec                  
[  5]  24.01-25.01  sec  15.9 KBytes   130 Kbits/sec                  
[  5]  25.01-26.01  sec  16.6 KBytes   136 Kbits/sec                  
[  5]  26.01-27.01  sec  18.0 KBytes   147 Kbits/sec                  
[  5]  27.01-28.01  sec  16.6 KBytes   136 Kbits/sec                  
[  5]  28.01-29.01  sec  16.6 KBytes   136 Kbits/sec                  
[  5]  29.01-30.01  sec  11.8 KBytes  96.3 Kbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-30.01  sec   532 KBytes   145 Kbits/sec    0             sender
[  5]   0.00-30.01  sec   469 KBytes   128 Kbits/sec                  receiver
iperf Done.
mosh:~$ iperf3 --client ping.online.net --port 5202 -l 3540 --time 30 -V -R
iperf 3.9-nrf
nrf9160
Control connection MSS: using modem default
Time: 22/05/23,06:07:51+36
Connecting to host ping.online.net, port 5202
      Cookie: nrf9160.6535.922882.71dfed4345cb977b
      TCP MSS: 0 (default)
[  5] local 0.0.0.0 port 0 connected to 62.210.18.40 port 5202
stream [0]: socket: 5 read: 0 write: 1
Starting Test: protocol: TCP, 1 streams, 3540 byte blocks, omitting 0 seconds, 30 second test, tos 0
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec  7.61 KBytes  61.7 Kbits/sec                  
[  5]   1.01-2.01   sec  12.4 KBytes   102 Kbits/sec                  
[  5]   2.01-3.01   sec  13.8 KBytes   113 Kbits/sec                  
[  5]   3.01-4.15   sec  13.8 KBytes  99.2 Kbits/sec                  
[  5]   4.15-5.11   sec  13.8 KBytes   118 Kbits/sec                  
[  5]   5.11-6.04   sec  15.2 KBytes   134 Kbits/sec                  
[  5]   6.04-7.03   sec  16.6 KBytes   138 Kbits/sec                  
[  5]   7.03-8.04   sec  13.8 KBytes   112 Kbits/sec                  
[  5]   8.04-9.03   sec  16.6 KBytes   138 Kbits/sec                  
[  5]   9.03-10.01  sec  13.8 KBytes   115 Kbits/sec                  
[  5]  10.01-11.05  sec  15.2 KBytes   120 Kbits/sec                  
[  5]  11.05-12.02  sec  12.4 KBytes   106 Kbits/sec                  
[  5]  12.02-13.04  sec  13.8 KBytes   111 Kbits/sec                  
[  5]  13.04-14.07  sec  13.8 KBytes   111 Kbits/sec                  
[  5]  14.07-15.01  sec  13.8 KBytes   120 Kbits/sec                  
[  5]  15.01-16.07  sec  13.8 KBytes   107 Kbits/sec                  
[  5]  16.07-17.02  sec  12.4 KBytes   108 Kbits/sec                  
[  5]  17.02-18.16  sec  16.6 KBytes   119 Kbits/sec                  
[  5]  18.16-19.13  sec  13.8 KBytes   117 Kbits/sec                  
[  5]  19.13-20.04  sec  13.8 KBytes   124 Kbits/sec                  
[  5]  20.04-21.14  sec  15.2 KBytes   114 Kbits/sec                  
[  5]  21.14-22.01  sec  13.8 KBytes   129 Kbits/sec                  
[  5]  22.01-23.03  sec  15.2 KBytes   122 Kbits/sec                  
[  5]  23.03-24.12  sec  15.2 KBytes   115 Kbits/sec                  
[  5]  24.12-25.04  sec  13.8 KBytes   123 Kbits/sec                  
[  5]  25.04-26.03  sec  13.8 KBytes   114 Kbits/sec                  
[  5]  26.03-27.09  sec  13.8 KBytes   107 Kbits/sec                  
[  5]  27.09-28.01  sec  13.8 KBytes   123 Kbits/sec                  
[  5]  28.01-29.05  sec  12.4 KBytes  98.3 Kbits/sec                  
[  5]  29.05-30.19  sec  15.2 KBytes   109 Kbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-30.19  sec   420 KBytes   114 Kbits/sec                  sender
[  5]   0.00-30.19  sec   420 KBytes   114 Kbits/sec                  receiver
CPU Utilization: not supported
iperf Done.
mosh:~$ 
mosh:~$ 

Now, I take about 35 seconds to upload a 30KB file based on the https_client sample.

Although there is header information of http and tls, data redundancy due to encryption, and waiting time after send() that I am adding, I think that it takes a long time compared to the measurement result of iperf. With the measured effective speed, I think that 30KB of data can be transmitted within 5 seconds.

Best Regards,

Yukio Oyama

[Einarh]

Hi!

1. This doesn't seem like a bad idea, but looking at the Zephyr documentation it looks like this option is not implemented.

some options are dummy and provided to ease porting of existing code

2. Looking in the sample documentation:

Instead of offloading the TLS sockets into the modem, you can use the Mbed TLS library from Zephyr. Using the Zephyr Mbed TLS, you can still use the offloaded sockets. Mbed TLS offers more configuration options than using the offloaded TLS handling

It looks like the sample uses offloaded sockets by default.

I also see that the sample has a file called overlay-tfm_mbedtls.conf, containing what I assume are the options required to use mbed TLS instead of offloaded sockets, and the Kconfig file defines a SAMPLE_TFM_MBEDTLS option, so yes by default the sample definitely uses offloaded sockets.

3. Right, so what takes the most time in your application right now? Is it the wait after recv?

-Einar

[OYAMA YUKIO]

Hi Einar-san,

 

I built in receiving AT notifications below and checked if there were AT notifications.

AT_MONITOR(at_notify, ANY, notification_handler);

There was no AT notification after send(). I couldn't know the end of the send() action through AT notifications.

 

I set the timeout of recv() below. Recv() was speed up due to the timeout.

setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, (const char*)&tv, sizeof tv);

I set the timeout time to 1 second. However, it is based on several experiments and is unfounded.

 

In the question of what function calls take a long time, if what you mean is the time each function call takes, it was recv().
However, recv() is a bit faster with the timeout setting, and it only runs once for each file I want to send. Therefore, the impact of recv() is small.

When sending one file, it is the send() loop that takes the longest time as a whole.

I have set a wait time of 200ms after send().

When sending a 30KB file, send() loops around 10 times (I have tried various sending sizes, so the number of loops varies. But it is usually around 10 times.)

The waiting time of 200ms is about 2 seconds in about 10 loops.  In my code, it takes about 35 seconds to send a 30KB file, so I consider the send() calling to take longer than the 200ms wait time.

Best Regards,

Yukio Oyama

[OYAMA YUKIO]

Hello Einarh-san,

I've been paying attention to whether the send was completed in order for the send() loop to succeed.

However, in the thread below, Didrick-san said in response to #2 that if the socket could not be assigned to the send queue, it would return an ENOMEM (or ETIMEOUT) error.
Unfortunately I don't know what function call he said in which sample.
In code based on my https_client sample, send () returns error 95 (EOPNOTSUPP), so I suspect he's referring to a different sample.

Do you know what sample it is based on?

Or is the same error not occurring in send() in the https_client sample I'm trying because the situation isn't happening?

I think it could be based on "\zephyr\samples\net\sockets\http_client".

Is this any difference compared to "\nrf\samples\nrf9160\https_client"? Didrik-san says Zephyr's http_client sample can also handle TLS.

Best Regards,

Yukio Oyama

[Einarh]

OYAMA YUKIO said:

In my code, it takes about 35 seconds to send a 30KB file, so I consider the send() calling to take longer than the 200ms wait time.

OYAMA YUKIO said:

Today, I used the Modem shell App and iperf3 to check the effective speed of the uplink in my environment.

It was 100K ~ 180Kbps

This seems strange to me, it would be interesting to know what in send() is taking so much time, is it waiting for an answer from the server? or a timeout?

OYAMA YUKIO said:

I think it could be based on "\zephyr\samples\net\sockets\http_client".

Is this any difference compared to "\nrf\samples\nrf9160\https_client"?

Yes they are different, Zephyr's sample uses Zephyr's HTTP Client API, while the nRF9160 sample doesn't. From what Didrik is saying it seems like Zephyr's library is a more complete implementation, so maybe you'll have more luck with that.

I would also recommend having a look at Zephyr's CoAP API. It has a very similar use case to HTTP, and you might find it easier to use with the nRF9160.

Best regards,

Einar