nRF9160 goes into cpu idle after sending data

[nWre]

Hello nordic.

I'm using your FTP client. The client can only send 4 kB of data at once so I send data numerous times and append to a file.

The first send I do is 160kB which requires 40 packets to be sent. If I then send another file it halts after sending around 70 packets, I get an error -116 and the device goes into idle and I have to reboot.
If I wait a 2-3 minutes from the first file before sending the second file, there is no problem.

Thus there must be a problem with a timer or a buffer overflowing or some such thing. The FTP client has a heartbeat timer but changing the value of this made no difference. I have deduced that it does not matter if I wait 10 seconds or 50 seconds, the second file always breaks at the same packet. This makes me think that the problem should be with a buffer that clears after a set amount of time.

Edit: I should add that I do ftp_uninit() when done with transfer and ftp_open() and ftp_login() before starting transfer.

Edit2: I accidentally verified the answer of the old thread and could not revert it, so please discard the other thread.

The error says connection timed out but if I try reconnecting it says the ftp connection is still up. Also, the timing doesn't matter, if its within the range of say, one minute, the transfer stops at the same place, indicating a set amount of data causes it to halt. I have gotten connection timed out before when it actually times out, then the program continues and does NOT halt. So there must be some sort of cpu panic going on here.

Looking forward to your reply

Kind regards

[Einarh]

Hello again!

Could you please share the most basic setup that causes the issue, so I can try to replicate?

Also, have you looked into whether the issue could be server side, since your error message is connection related?

Best regards,

Einar

[nWre]

Hi! Here is a test version:

Main file

#include "ftptest.c"

void main(void)
{
    tfirstinit();
	tftpinit();
	for (int i = 0;i<120;i++){
		tftptest();
	}
}

ftptest.c

#include <zephyr.h>
#include <net/socket.h>
#include <stdio.h>
#include <drivers/uart.h>
#include <string.h>
#include <logging/log.h>
#include <ctype.h>
#include <net/tls_credentials.h>
#include <net/ftp_client.h>
#include <fs/fs.h>
#include <ff.h>
#include <device.h>
#include <drivers/spi.h>
#include <sys/printk.h>
#include <disk/disk_access.h>
#include <logging/log.h>

int treturnValue;

K_SEM_DEFINE(tmy_sem, 0, 1);
uint64_t testData[500] = {0};

void tftp_ctrl_callback(const uint8_t *msg, uint16_t len)
{
    printk("%s", msg);
    /* Release semaphore when receiving answer from server */
    k_sem_give(&my_sem);
}

void tftp_data_callback(const uint8_t *msg, uint16_t len)
{
    printk("%s", msg);
}

int tfirstinit(void){
    treturnValue = ftp_init(tftp_ctrl_callback, tftp_data_callback);
    return treturnValue;
}

int tftpinit(void)
{
    treturnValue = ftp_open("MYSERVER", 21, -1);
    treturnValue = ftp_login("ACCNAME", "ACCPASS");
    return treturnValue;
}

int tftptest()
{
    for (int i = 0;i<500;i++){
        testData[i] = 0;
    } 
    int retVal = 0;
    retVal = ftp_put("test.bin", testData, sizeof(testData), FTP_PUT_APPEND);
    printk("put procedure says: %i \n", retVal);
    return retVal;
}

int tftpuninit(void){
    treturnValue = ftp_uninit();
    return treturnValue;
}

and prjconf

CONFIG_SPI=y                            # SPI hardware bus
CONFIG_SPI_SLAVE=y                      # SPI slave support [EXP]

CONFIG_LTE_AUTO_INIT_AND_CONNECT=y
CONFIG_NCS_SAMPLES_DEFAULTS=y
# LTE link control
CONFIG_LTE_LINK_CONTROL=y
# Modem library
CONFIG_NRF_MODEM_LIB=y
# Modem library
CONFIG_NRF_MODEM_LIB_SYS_INIT=y
# FTP Client
CONFIG_FTP_CLIENT=y

CONFIG_DISK_DRIVER_SDMMC=y
CONFIG_MAIN_STACK_SIZE=4096
CONFIG_SPI_NRFX_RAM_BUFFER_SIZE=8
CONFIG_FILE_SYSTEM=y
CONFIG_FAT_FILESYSTEM_ELM=y

# General config
CONFIG_NEWLIB_LIBC=y
CONFIG_NEWLIB_LIBC_FLOAT_PRINTF=y

# Network
CONFIG_PRINTK=y
CONFIG_NETWORKING=y
CONFIG_NET_NATIVE=n
CONFIG_NET_SOCKETS=y
CONFIG_NET_SOCKETS_OFFLOAD=y

CONFIG_HEAP_MEM_POOL_SIZE=4096
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048
CONFIG_PRINTK=y
CONFIG_ASSERT=y
CONFIG_STDOUT_CONSOLE=y
CONFIG_UART_INTERRUPT_DRIVEN=n
CONFIG_NET_SOCKETS_POSIX_NAMES=y
CONFIG_NET_BUF_USER_DATA_SIZE=1000
CONFIG_LOG=y
CONFIG_SDMMC_LOG_LEVEL_DBG=n

As each 4kB segment sent increments the portnumber(?) as shown by the output, I'm thinking I might reach a maximum amounts of connections to the server. I'm not that familiar with the FTP protocol but it feels like I shouldn't be entering passive mode for each segment but only once and then just send data.

Thanks again!

[Einarh]

Hi

I'm looking through the program flow in ftp_client.c, and it looks like the file you send gets added to a workqueue as a single workitem.

I could imagine that the work thread might starve the other threads, including the heartbeat timer, if the workitem is sufficiently large, causing your issue.

Do you see the same behavior if you send the 70 packets right away without sending the 40 packets first?

How long does it take before you get the error when you start sending your big file? Is it a matter of seconds?

Also, from what I understand FTP keeps two separate connections for control messages and data transfer, which is why the connection would still be up after a timeout in the data connection.

Best regards,

Einar

[nWre]

Hi!

No it doesn't matter. If I send a "file" large enough it will halt. What I'm doing is reading from a bin-file that I have stored on a micro-sd card and then sending 4kB at a time from that with the append command.

Each time the "PASV" command is sent, a new port on the server is opened and my thinking was that maybe this is a limitation, that if you open enough ports, eventually the server will "lock".

If there is a way to use the same port, maybe by sending "PASS" instead of "PASV" or manually saving the port that was previously used this wouldn't happen. I've tried doing this but I get error bad sequence of commands.

After each segment sent, poll_task_done(); is called and I imagine the thread is deconstructed.

The error occurs after about 100 segments, I will check the exact amount. It is not based on time but in segments and I think opening 100 ports for passive connection might be the problem but I see no way around it.

Regards
nWre

[nWre]

Here is a screenshot of the output where it fails:



so it fails after 109 sends, as you can see, the port number is incremented each segment and I think this might be the cause of the problem.

[Einarh]

Are you slicing the file into 4kB parts yourself before sending?

Because it looks like the ftp code will do this for you, have you tried just sending the whole file?

And when you send the "PASV" command I believe you are telling the server to enter passive mode to prepare to receive data. "Pass" is related to sending password credentials.

You can read more about how FTP works here.

-Einar

[nWre]

If I try to send a buffer larger than 4kB I get "E: send data failed: -122" and 122 error code is "Message too long".

Edit: And yes, I slice it myself. I read 4kB from my file into a buffer, send via ftp, read next 4kB, send via ftp etc.

Edit2: I should mention that a previous thread about FTP on this forum said that 4kB was a restriction without a workaround, something with the modem firmware.

[Einarh]

Hi

Yes it's correct that the modem can't send more than 4kB at a time, I thought the "send" function might be able to figure that out but I guess not.

It does seem like the FTP put function automatically closes the data connection when the transfer is completed though, so it might be the case that you're not supposed to uninit and open between each consecutive transfer.

It could be that you're clogging the available connections with control message connections since you're opening many connections quickly?

Remember that when you issue a put command, the transfer is placed in a queue, and isn't executed immediately.

I would try just queueing your transfers without disconnecting between each one if you haven't already.

-Einar