Problem with nRF-connect-SDK http_client_req() function

[royalbee]

Hi,


I open ssl socket to the server (without certificate verification),
and then use an http_client_req() call, to perform the actual http request (with additional optional headers).

That works fine for both http GET and POST requests, that have a short response from the server.

However, when the server's response is a bit larger (say 2.5KB not including response headers and cookie), I always get "Socket was closed by remote" error (-104).
I tried increasing recv_buf_len from 1KB to 8KB, but it didn't help.

When debugging http_client_req(), I see that the call to socket's recv() just returns 0 bytes, which is considered socket close by remote.
On the server logs I see that HTTP request, but no special problems appear on aws elb logs.

Performing the same request on my PC (using Postman for example), works just fine, so the problem is not on the server side (at least not with standard http/tcp settings)/

thanks in advance..

[Albrecht Markus Schellenberger]

Hello again

our modem team went through your modem traces and has the following feedback:

“Then Client sends UL data with success but then first data receive fails, TLS stack returns error from data handling procedure to our TLS controller. To me looks that incoming data may not be TLS data format at all, also wireshark -tool does not recognize it as TLS data (like it does for example that Uplink data client sends just before that incoming data etc.). TLS stack also creates and sends Alert message immediately after that INVALID_LENGTH, which also means that there was some problem with TLS data (to inform server side about data failure). So there should be checked is server side working ok, and sending actually TLS related data.

So shortly TLS stack does not recognize incoming data as TLS (and neither wireshark) -> error.”

Could you please check with the server side here?

Regards,

Markus

[royalbee]

Thanks for the quick response Markus.

The application which logs and traces I sent you, actually did not connect to our server. It connected a public API mock server that I used for my tests (beeceptor.com). 
So the problem happened both on our servers and on a different (public) server (which is actually a stable server that companies use for their tests).

Also note that both servers (ours and beeceptor.com) worked perfectly fine with ssl API calls, as long as the server's response was short enough, so there seems to be no problem with those servers ssl implementation.
Also note that both servers had no problem with ssl requests that have a large response, as long as those requests did not come from the 9160 modem (when I used Postman for example, or just the browser, with https requests, I got valid long responses from the servers). So it doesn't seem like a problem on the servers side at all.

Maybe you can try to check this problem on your side as well? just connect to any server that supports ssl and returns response that is longer than 2.5K (I actually think that 2K will also be enough).
I will be happy to see a simple example that you tested and worked for you (if you like, you can use the https_client sample for that purpose, like I did).



BTW, my guess is that the problem is related to the "INVALID_LENGTH" error that you got there.
As you said, the socket buffer is limited to 2K in case of SSL. Maybe when the server's response is larger than the 2K buffer, somehow it doesn't fit into the buffer because of SSL overhead (or something like that..), and in practice the server tries to return a bit more than 2K and we get this invalid-length error.
I supposed that such response that is cropped to 2K will no longer be a valid TLS data, since part of its data is missing, so that may be the reason that Wireshark doesn't recognize it as a valid TLS data.

It is just a guess of course, but it does seem like something related to the 2K buffer limitation, and the invalid-length error..