nRF9160 issue in receiving MQTT publish messages larger than 2048 bytes over TLS

Question

Hi, 
 My Setup details: 
 nRF9160 DK Board: 0.8.3 nRF Connect SDK tag: 1.3.0 OS: Windows 10 Pro LTE Network: NB-IoT Modem Firmware: mfw_nrf9160_1.2.0 Cloud Platform: AWS IoT Core CONFIG_AWS_IOT_MQTT_PAYLOAD_BUFFER_LEN: 5120 
 Use case: 
 As part of device provisioning process we need to receive a MQTT message from AWS which contains certificate details, this message can't be split as we don't have control over it. Size of such messages are around 3KB. 
 Issue: 
 Whenever the incoming PUBLISH message size from AWS is more than 2048 bytes, MQTT connection is dropped. After some debugging the function "mqtt_read_and_parse_fixed_header" in MQTT library (mqtt_rx.c source code) returns -ENOTCONN. 
 In the release notes of modem firmware, a limitation is mentioned about TLS feature: "2kB secure socket buffer size." However after experimenting with incoming message sizes I noticed publish messages up to 2235 bytes are received successfully. Few bytes over it and the connection drops. In general with TCP protocol large packets can be received with small buffers with appropriate MTU/Window size. Is this issue linked to the above TLS limitation or something else? 
 Reproducing the issue is simple, use any cloud example to connect to AWS over TLS, subscribe to any topic and from AWS send 2KB+ sized messages. 
 Thanks, Ravikiran

Dominic · Answer

Just commenting on this because I ran into the same issue. The lambda which intercepts the CreateKeysAndCertificate response ("$aws/certificates/create/json/accepted") must then republish the certificate and key separately to two new topics. 
 There is some trickery going on in the AWS IoT backend because the policy document states that a device can subscribe to any subtopic of "$aws/certificates/create/*" but you will find that subscribing to a subtopic other than /accepted and /rejected causes the client to be disconnected from the broker. Also, the lambda cannot publish to a reserved topic denoted by the '$' prefix 
 The solution is to create two new topics, eg: "fleetprovision/certificate" and " fleetprovision /key" which the lambda can publish to. The final step is updating the fleet certificate policy document so that the device can also subscribe and receive messages from these topics using its fleet provisioning certificate.

Dominic · Answer

Following up because I finally got this to work, here are some tips to save the next person some time: 
 1. The provision-template request needs to be published and received by the same client which made the certificateRequest (EC2 instance or lambda depending on your setup). Trying to do so from the nrf9160 results in a 'InvalidCertificateId' error. 
 2. Using Zephyr's json parser library, newline characters are parsed as literal '\' and 'n' characters. I wrote a function which uses strtok to replace these characters with actual 0x0A newline chars. 
 static int copy_certificate(char *dest, char *src, int max_len)
{
 int copy_len = 0;
 char backslash[] = "\";
 char newline = '
';

 char* tok = strtok(src, backslash);

 while(true){
 copy_len += strlen(tok);
 if(copy_len >= max_len){
 LOG_ERR("Certificate dest not large enough");
 return -ENOMEM;
 }
 strcat(dest, tok);

 tok = strtok(NULL, backslash);
 if(tok == NULL) break;

 *tok = newline;
 }
 return 0;
} 
 
 3. modem_key_mgmt_delete and modem_key_mgmt_write only work when the LTE modem is off however they will still return a zero response code when the modem is on/connected, even though modem_key_mgmt_exists confirms that the certificates are not written to the modem. 
 So overall a slightly painful process of trial and error but saves a step at the factory and you don't need to expose the modem's UART to write the certificates.

nRF9160 issue in receiving MQTT publish messages larger than 2048 bytes over TLS

Top Replies