RE: Need the nrf9160 example to transfer large data by TCP.

Luther — Mon, 09 Sep 2019 15:33:18 GMT

Hi Martin,

I had look into tcp example, but it seem not what I want.
That example is for receiving data. But it has a "blocking_send" function.
So I had modify that tcp example to do the following test for sending large tcp data.
The send buffer size is about 20KB
Test 1: Just using "blocking_send" to send the data.
Result:

***** Booting Zephyr OS v1.14.99-ncs2 *****
LTE Link Connecting ...
LTE Link Connected!
TCP example getaddrinfo err: 0
client_fd: 3
connect err: 0
Send total data size: 20479
Test 1: blocking_send -1

Actually it will fail to send data. The return code is -1.

Test 2: I write a new function slice the buffer, every time just send 512 bytes. but without delay.
Result:

***** Booting Zephyr OS v1.14.99-ncs2 *****
LTE Link Connecting ...
LTE Link Connected!
TCP example getaddrinfo err: 0
client_fd: 3
connect err: 0
Send total data size: 20479
Send piece NoDelay: 512
Send piece NoDelay: 512
Send piece NoDelay: 512
Send piece NoDelay: 512
Send piece NoDelay: 512
Send piece NoDelay: 512
Send piece NoDelay: 512
Error: send large data, c:-1, ix:3584
Test 2: sendLargeData 3584

As you can see, the data send about 7 piece then got the error code -1.

Test 3: Slice the buffer, every time send 512 bytes and add delay 200ms.
Result:

***** Booting Zephyr OS v1.14.99-ncs2 *****
LTE Link Connecting ...
LTE Link Connected!
TCP example getaddrinfo err: 0
client_fd: 3
connect err: 0
Send total data size: 20479
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 512
Send piece delay 200ms: 511
Test 2: sendLargeData 20479

Yes, this time. The data had successful be send. But I wish I could do the optimum about the delay time.

BTW, I am testing the NBIoT.

Here is my prj.conf and main.c

prj.conf

CONFIG_BSD_LIBRARY=y
CONFIG_GPIO=n
CONFIG_SERIAL=y
CONFIG_STDOUT_CONSOLE=y
CONFIG_UART_INTERRUPT_DRIVEN=y
CONFIG_TEST_RANDOM_GENERATOR=y
CONFIG_NETWORKING=y
CONFIG_NET_BUF_USER_DATA_SIZE=1
CONFIG_NET_SOCKETS_OFFLOAD=y
CONFIG_NET_SOCKETS=y
CONFIG_NET_SOCKETS_POSIX_NAMES=y
CONFIG_NET_RAW_MODE=y
CONFIG_TRUSTED_EXECUTION_NONSECURE=y
CONFIG_LOG=n
CONFIG_LOG_DEFAULT_LEVEL=4
CONFIG_HEAP_MEM_POOL_SIZE=1024
# LTE link control
CONFIG_LTE_LINK_CONTROL=y
CONFIG_LTE_AUTO_INIT_AND_CONNECT=n
CONFIG_LTE_NETWORK_MODE_NBIOT=y
# TYPE "4":E-UTRAN (WB-S1 mode), "5" E-UTRAN (NB-S1 mode)
CONFIG_LTE_EDRX_REQ_ACTT_TYPE="5"
CONFIG_MAIN_STACK_SIZE=4096

Remember to modify the #define TCP_HOST and TCP_PORT to your test server.

/*
 * Copyright (c) 2018 Nordic Semiconductor ASA
 *
 * SPDX-License-Identifier: LicenseRef-BSD-5-Clause-Nordic
 */

#include <zephyr.h>
#include <net/socket.h>
#include <stdio.h>
#include <uart.h>
#include <string.h>
#include <lte_lc.h>

#define TEST_SEND_WITH_DELAY
//#define DO_TEST_1

#define TCP_HOST "test.com"
#define TCP_PORT "8080"
#define RECV_BUF_SIZE ((80 * 24) + 1)
#define MAX_SOCKET_SEND_SIZE  512
#define SEND_BUF_SIZE (20 * 1024)

char recv_buf[RECV_BUF_SIZE];
char send_buf[SEND_BUF_SIZE];

u32_t inet_addr(u8_t a, u8_t b, u8_t c, u8_t d)
{
	u32_t value = 0;

	value |= (u32_t)((((u8_t)(d)) << 24) & 0xFF000000);
	value |= (u32_t)((((u8_t)(c)) << 16) & 0x00FF0000);
	value |= (u32_t)((((u8_t)(b)) << 8) & 0x0000FF00);
	value |= (u32_t)((((u8_t)(a)) << 0) & 0x000000FF);

	return value;
}

int blocking_recv(int fd, u8_t *buf, u32_t size, u32_t flags)
{
	int err;

	do {
		err = recv(fd, buf, size, flags);
	} while (err < 0 && errno == EAGAIN);

	return err;
}

int blocking_send(int fd, u8_t *buf, u32_t size, u32_t flags)
{
	int err;

	do {
		err = send(fd, buf, size, flags);
	} while (err < 0 && errno == EAGAIN);

	return err;
}

int blocking_connect(int fd, struct sockaddr *local_addr, socklen_t len)
{
	int err;

	do {
		err = connect(fd, local_addr, len);
	} while (err < 0 && errno == EAGAIN);

	return err;
}

int sendLargeData(int sock, uint8_t *pSrc, int size)
{
  int sendCount = 0;
  int sendIx = 0;
  int err = 0;
  int remainSize = size;

  while (remainSize > 0)
  {
    if (remainSize > MAX_SOCKET_SEND_SIZE)
    {
      sendCount = MAX_SOCKET_SEND_SIZE;
    }
    else
    {
      sendCount = remainSize;
    }

    err = blocking_send(sock, (pSrc + sendIx), sendCount, 0);
    if (err != sendCount)
    {
      printk("Error: send large data, c:%d, ix:%d\n", err, sendIx);
      break;
    }
    remainSize -= sendCount;
    sendIx += sendCount;
#ifdef TEST_SEND_WITH_DELAY
    printk("Send piece delay 200ms: %d\n", sendCount);
    if (remainSize > 0)
    {
      k_sleep(200);
    }
#else
    printk("Send piece NoDelay: %d\n", sendCount);
#endif //TEST_SEND_WITH_DELAY

  }

  return sendIx;
}

void app_tcp_socket_start(void)
{
	struct addrinfo *res;
  static struct addrinfo hints;
  
	printk("TCP example ");

  hints.ai_family = AF_INET;
  hints.ai_socktype = SOCK_STREAM;
	int err = getaddrinfo(TCP_HOST, TCP_PORT, &hints, &res);

	printk("getaddrinfo err: %d\n\r", err);
	//((struct sockaddr_in *)res->ai_addr)->sin_port = htons(TCP_PORT);
	//struct sockaddr_in local_addr;

	//local_addr.sin_family = AF_INET;
	//local_addr.sin_port = htons(0);
	//local_addr.sin_addr.s_addr = 0;

	int client_fd = socket(AF_INET, SOCK_STREAM, 0);

	printk("client_fd: %d\n\r", client_fd);
	//err = bind(client_fd, (struct sockaddr *)&local_addr,
	//	   sizeof(local_addr));
	//printk("bind err: %d\n\r", err);
	err = blocking_connect(client_fd, (struct sockaddr *)res->ai_addr,
			       sizeof(struct sockaddr_in));
	printk("connect err: %d\n\r", errno);

	size_t ret;
  printk("Send total data size: %d\n", sizeof(send_buf) - 1);
#ifdef DO_TEST_1
  /* Luther add TEST 1: using blocking_send in one time */
  ret = blocking_send(client_fd, send_buf, sizeof(send_buf) - 1, 0);
  printk("Test 1: blocking_send %d", ret);
#else	
  /* Luther add TEST 2: slice small piece send large data. */
  ret = sendLargeData(client_fd, send_buf, sizeof(send_buf) - 1);
  printk("Test 2: sendLargeData %d", ret);
#endif //DO_TEST_1

	do {
		ret = blocking_recv(client_fd, recv_buf, sizeof(recv_buf) - 1,
				    0);
		if (ret > 0 && ret < RECV_BUF_SIZE) {
			/* Ensure that string is zero terminated */
			recv_buf[ret] = 0;
			printk("%s", recv_buf);
		}
	} while (ret > 0);

	printk("\n\rfin");
	freeaddrinfo(res);
	err = close(client_fd);
}

/* initial the send buffer, fill 1 to 255*/
void init_send_buf()
{
  int size = sizeof(send_buf);
  int i = 0;
  for (i=0;i<size;i++)
  {
    send_buf[i] = (char)i;
  }
}

/**@brief Configures modem to provide LTE link. Blocks until link is
 * successfully established.
 */
static void modem_configure(void)
{
#if defined(CONFIG_LTE_LINK_CONTROL)
  if (IS_ENABLED(CONFIG_LTE_AUTO_INIT_AND_CONNECT)) {
    /* Do nothing, modem is already turned on
     * and connected.
     */
  } else {
    int err;
    printk("LTE Link Connecting ...\n");
    err = lte_lc_init_and_connect();
    __ASSERT(err == 0, "LTE link could not be established.");
    printk("LTE Link Connected!\n");
  }
#endif
}

int main(void)
{
  /* Enable NBIoT */
  modem_configure();
  
  init_send_buf();
	app_tcp_socket_start();
  
  return 0;
}

RE: Need the nrf9160 example to transfer large data by TCP.

Martin Lesund — Tue, 27 Aug 2019 11:47:24 GMT

Please take a look at this TCP sample and use that as reference.

(similar thread)

RE: Need the nrf9160 example to transfer large data by TCP.

Luther — Mon, 26 Aug 2019 20:58:54 GMT

Hi Martin,

Thanks for your reply.
I already using the modem firmware v1.0.0.
After I update the NCS, and update the two libbsd_nrf9160_xxaa.a files.
It seems better now, it already running 10 hours and success to send the data to the server without error.
I will continue testing.

Thanks a lot.

By the way, can you suggest a smart way to send the large data?
Here is my send data function.

#define MAX_SOCKET_SEND_SIZE  512

int sendLargeData(int sock, uint8_t *pSrc, int size)
{
  int sendCount = 0;
  int sendIx = 0;
  int err = 0;
  int remainSize = size;

  while (remainSize > 0)
  {
    if (remainSize > MAX_SOCKET_SEND_SIZE)
    {
      sendCount = MAX_SOCKET_SEND_SIZE;
    }
    else
    {
      sendCount = remainSize;
    }

    err = blocking_send(sock, (pSrc + sendIx), sendCount, 0);
    if (err != sendCount)
    {
      DBG_PRINT("Error: send large data, c:%d, ix:%d\n", err, sendIx);
      break;
    }
    remainSize -= sendCount;
    sendIx += sendCount;
    if (remainSize > 0)
    {
      k_sleep(200);
    }
  }

  return sendIx;
}

I slice the total data to small pieces, and add a delay k_sleep(200) after send each piece.
It seems after call the send() function, the data just keep in the buffer. Not finished the sending data.
I don't know how to check the buffer is clear or not. So I just add a waiting time.

RE: Need the nrf9160 example to transfer large data by TCP.

Martin Lesund — Fri, 23 Aug 2019 11:15:24 GMT

Hi Luther,
Please make sure you have the latest modem firmware .

And please add this fix into your environment: https://github.com/NordicPlayground/nrfxlib/pull/77

Also please share any log output or even a modem trace would be beneficial.