Trouble receiving binary stream data with recv()

[-Richard-]

I've been writing this FOTA solution that would work together with an existing server we use together with Onomondo connectors.

And I've been running into this issue when I'm trying to download a firmware bin file, when I try to use the recv function to retrieve the data, I'll get a few bytes that make sense, the starter byte for example, and then in between there's a loop of `239 191 189` bytes just repeating hundreds of times.

When I've tested the output from the server through Postman or just by URL, it appears to work correctly and returns the file. I am however able to return the bytes in a string format and then convert them on the device, although it's a very slow operation.

I've also setup logging on the server to verify the data that's supposedly being returned to the nrf9160, and that also appears to be correct?

[Sigurd Hellesvik]

I try to use the recv function

Do you check the return value of the recv() function?
Does it return success both for the "good bytes" and the "bad bytes"?

I am however able to return the bytes in a string format and then convert them on the device, although it's a very slow operation.

Is this for the server or the nRF9160?

Regards,
Sigurd Hellesvik

[-Richard-]

Yes it returns success for both bytes, I recv chunks of 2048 bytes at a time, and in there I'll have a mix of both. 

I use a NodeJS express server, and if I return a binary buffer, that's when the NRF9160 gets the wrong bytes, however if I take that buffer and say turn it into a string or a hex string on the server before returning it, then I receive the data in string format correctly on the NRF9160.

Appears to only happen when I try to recv a binary buffer on the NRF9160 from the express server, as I've tried with other formats such as JSON, and that gets handled correctly.

[Sigurd Hellesvik]

Can you record and upload a modem trace?

This may give us more insight into the issue.

Regards,
Sigurd Hellesvik

[-Richard-]

8357.modem-trace.zip

There's a modem trace for 2 of the ports, as 2 of them seemed to read something and I wasn't sure which port exactly was the right one.

[Sigurd Hellesvik]

Hi,

Which modem firmware version do you use?

Regards,
Sigurd Hellesvik

[-Richard-]

The modem firmware version is mfw_nrf9160_1.3.3

[Sigurd Hellesvik]

I have still been unable to find what this could be, so I will keep asking for info.

Can you send the file you are trying to transfer?
Can you list the functions on the nRF9160 you use to receive the data?

From your modem trace, it looks like your server has the IP 1.2.3.4. This seems a bit odd to me. Does your server actually have the IP 1.2.3.4, or is something else happening here?

Since it works when you send the data as strings, can you list how the send function looks with and without the string formatting?
And can you take a modem trace with the string formatting? It could be useful to compare the good VS bad modem traces.

Regards,
Sigurd Hellesvik

[-Richard-]

Here's the bin file that's on the server

4722.app_update.bin

This is how I read the data from the socket on the nrf9160, both string formatted and binary.

The IP being 1.2.3.4 is correct, we use Onomondo connectors to send and receive our data, and we configure on their dashboard where the data will be sent. You can read more about it here Onomondo Connectors.

Here's how the nodejs express server returns the bin currently, where response is just an array of the bytes.

Binary version

Hex string version

When I test this in Postman or download it directly to my computer through the API it works with the binary version, but when I try to download on the NRF9160 I get the wrong bytes, although when I convert it to a string in a hex format I then receive it correctly.

Modem trace with string formatting.

trace-2023-02-27T21-26-23.459Z.bin

trace-2023-03-01T12-15-56.056Z.bin

[Sigurd Hellesvik]

To view your modem traces, I converted them to .pcapng using the Modem Trace tool.
Then I open .pcapng in Wireshark.

For the app_update.bin, I use the command-line tool hexyl to look at the data.

Here are two screenshots from Wireshark and hexyl:

The "broken" trace:

app_update.bin:

As you can see from the data, both start with "3d" and then differ slightly.
I suspect this is because you have sent me the wrong app_update.bin from the one used in your trace.
Either way, both have "02 00 00 00 00 00 etc" as well, so the start of both these look fine so far.

However, after a little bit, the trace repeats "ef bd bd ef bf bd" and so on.
When I inspect packets further down where the data start to repeat, I get all "ef bd db" repeating.
If you convert this to decimal, it is likely the data you get repeating.

I googled this and found a post on stack overflow: https://stackoverflow.com/questions/38018263/what-is-the-format-or-encoding-of-a-file-with-data-like-this.

From this, I suggest you have another look at how you encode the binary file when you transfer it.
I am not experienced in nodejs express server, but you put "utf-8" into the send function. Maybe do something else here?

From the working hex-string trace, I can read data which is not these "illegal utf-8" bytes from the pcapng.
I think this is because when you send a string, the string is "utf-8".

Does this help?

Regards,
Sigurd Hellesvik

[-Richard-]

It is in fact the same app_update.bin, the first 4 bytes when you compile the firmware in a sort of identifier, so 3D B8 F3 96 is an ID for MCUBoot Application Image, so I think even if the app_update.bin were different it should still start with that.

I tried every single encoding type that the express server supported and none of them worked, but I have been in touch with Onomondo's support, and although they don't think they're the cause of the issue either, I tried disabling the connector and just using regular HTTP/s requests and that actually works as expected!

So for now I'll just have to use regular requests and avoid the connector system when sending binary data. 

Thank you for the support!