Buffer JSON data and publish through MQTT nrf9160

RE: Buffer JSON data and publish through MQTT nrf9160

Adeel — Wed, 17 Feb 2021 16:18:38 GMT

Hi Didrik,

Thanks for the information regarding this. Helped me understand more about these allocations 🙂.

Regards,

Adeel.

RE: Buffer JSON data and publish through MQTT nrf9160

Didrik Rokhaug — Wed, 17 Feb 2021 16:01:26 GMT

[quote user="Adeel"]What can be the maximum value of these configurations ? and how do they effect the memory ?[/quote]

You are obviously limited by the amount of RAM in the system.

CONFIG_MAIN_STACK_SIZE sets the stack size for the main thread, but there are often other threads as well. The sum of all the stack sizes, and the heap size should not be more than the amount of RAM in the system.

However, you might be able to set higher numbers, as long as you don't use too much RAM. If a stack or the heap starts to grow into a different area, weird things can happen (but porbably you just get a crash. Regardless, don't do that).

When it comes to the heap, the memory allocation algorithm will also be a consideration. Depending on the algorithm, increasing the heap size might not let you allocate more items, just bigger items.

RE: Buffer JSON data and publish through MQTT nrf9160

Adeel — Wed, 17 Feb 2021 09:52:38 GMT

Hi Didrik,

Thanks for the suggestion. I tried it and it worked. I had just one more question to ask you regarding the 2 configurations in prj.conf file .

I have it like this in my sample.

CONFIG_MAIN_STACK_SIZE=32768

CONFIG_HEAP_MEM_POOL_SIZE=65536

What can be the maximum value of these configurations ? and how do they effect the memory ?

Regards,

Adeel.

RE: Buffer JSON data and publish through MQTT nrf9160

Didrik Rokhaug — Mon, 15 Feb 2021 09:03:02 GMT

You need to free sensor6 each time my_work_handler_6 runs, otherwise you will leak the memory allocated by cJSON_CreateObject().

If you move the call to cJSON_Delete() out of the for loop

RE: Buffer JSON data and publish through MQTT nrf9160

Adeel — Fri, 12 Feb 2021 13:49:02 GMT

Hi Didrik,

[quote userid="81181" url="~/f/nordic-q-a/71505/buffer-json-data-and-publish-through-mqtt-nrf9160/293830#293830"]The easiest way to save the data is probably to create a static array, either with pointers to heap-allocated strings (the allocation should happen in your work_handler, and the strings must be freed when you send them), or memcpy'ed strings directly into the array.[/quote]

Thanks for the advice on this and I moved along to create a static array to buffer the data. I was able to achieve my usage but I noticed that the code reboots when I try to publish my data for the 5th time. Its a bit strange and I tried to change the MQTT broker but the code reboots at exactly the same point again.

I am attaching the log file as well as the sample code I am using so you could reproduce it at your end.

devzone.nordicsemi.com/.../Rebooting-log.txt devzone.nordicsemi.com/.../mqtt_5F00_buffer_5F00_test.rar

To test it quickly, I have each timers working at 30 seconds for now, I try to get the data every 30 seconds and then send out the data when I have buffered 6 messages.

It always happens that the code works perfectly as I need it for the first 4 times while publishing every 180 seconds for now, but at the 5th time, the code reboots without any error update.

I tried to understand it based on the memory issue, and increased my memory space to :

# Main thread
CONFIG_MAIN_STACK_SIZE=32768

CONFIG_HEAP_MEM_POOL_SIZE=65536

and now the reboot happens after the 15th time.

I guess I am not freeing up my memory correctly that I need to, and I am not sure how to do that properly.

Could you try to reproduce this code at your end and see what the issue could be with this. It would be of great help 🙂.

Regards,

Adeel.

RE: Buffer JSON data and publish through MQTT nrf9160

Didrik Rokhaug — Wed, 10 Feb 2021 15:38:18 GMT

Hi,

Depending on your use case, it might be easier to store the raw data, than the encoded string.

Where do you allocate memory for out1?

The easiest way to save the data is probably to create a static array, either with pointers to heap-allocated strings (the allocation should happen in your work_handler, and the strings must be freed when you send them), or memcpy'ed strings directly into the array.

You would also need a counter somewhere to keep track of which index you should write the next data point to.

Alternatively, if you want a more fancy data structure, you can take a look at Zephyr's ring buffer or FIFO queue.

Best regards,

Didrik