1NCE Onboarding SIM Card

RE: 1NCE Onboarding SIM Card

danielboe — Thu, 11 Feb 2021 15:22:25 GMT

Hi, here is some code for the basic cert querry and deploy them in a sec-tag to the device.

devzone.nordicsemi.com/.../1nce_5F00_deploy.zip

RE: 1NCE Onboarding SIM Card

danielboe — Thu, 11 Feb 2021 12:07:16 GMT

Hi Jan, thanks for your time. The fault in this case was on my side and is fixed now(forgot to clear the array). The process works fine and I was able to connect to aws cloud from scratch . Thanks for all the effort, all this stuff helps us a lot and shows us that you are a great partner for IoT networks! I will share the code for the onboarding later in this thread.

RE: 1NCE Onboarding SIM Card

Jan Sulaiman - 1NCE — Thu, 11 Feb 2021 12:00:08 GMT

Hi Daniel,

I asked our Engineers to check out the issue, but unfortunately, we are not able to replicate it.

We have tested ourselves with various devices which are used and they were working fine.

Can you maybe share the specific code and also the full HTTP Request you are sending?

Then it might be easier for us to figure out what is going wrong.

We are committed to help you and get it working.

Thanks a lot,

Jan

RE: 1NCE Onboarding SIM Card

Jan Sulaiman - 1NCE — Thu, 11 Feb 2021 07:11:14 GMT

Hi Daniel,

Thanks for letting us know. Let me investigate this.

I will update you as soon as I know more.

-Jan

RE: 1NCE Onboarding SIM Card

danielboe — Wed, 10 Feb 2021 19:04:42 GMT

Hi Jan,

for every part I get to the payload a string at the end

....,"iccid":"8988228066601885269"}4TxdtTcv\n3Fy9cgi8wRK4xQ9xQ6BF1zNCmupGdFDb

"4TxdtTcv\n3Fy9cgi8wRK4xQ9xQ6BF1zNCmupGdFDb"

You have any idea where this could come from?

RE: 1NCE Onboarding SIM Card

danielboe — Tue, 09 Feb 2021 15:37:25 GMT

Hi Jan, the Range Request works like it should and I am now able to get data from your endpoint. I will do the complete implementation the next days but this should be straight from this point. https://pastebin.com/BBnFu3gz

RE: 1NCE Onboarding SIM Card

Jan Sulaiman - 1NCE — Tue, 09 Feb 2021 11:41:40 GMT

Hi Daniel,

Sorry for the slight delay in response.
It took a week longer the originally planned to get this feature request implemented on our side.

But I am happy to announce that we are now offering response splitting using HTTP range requests for the Devie API/Onboarding call you have been using.

In the following, I will explain this new feature (final documentation on the 1NCE website is still outstanding). I hope you can test it and hopefully, this will provide you everything you need.

I would love to hear some feedback and further suggestions if you have any.

Thanks a lot,
Jan

Range requests - Documentation

Some of our API’s support splitting the response body into smaller chunks. This can be especially useful for devices with a small TLS buffer. This is done using a HTTP range request as defined by RFC-7233.

This documentation is applicable for all our API’s that contain an Accept-Ranges: bytes header in the response. This can be checked by executing a normal GET or HEAD request on this endpoint. It will also be explicitly stated in the API definition of that endpoint.

In general, only GET endpoints can support HTTP range requests.

If the endpoint supports range requests, the client can request this using specific headers. If the request is successful and would normally return status 200 OK, it will return status 206 Partial Content instead, with only the requested byte range in the response body.

Please note that errors do not have status 200 OK, so the range request will not be applied and the response body may be larger than expected!

There are a couple of known limitations. Please check the segment “Known limitations“ for more info.

Headers

Request

Name	Located in	Description	Required	Schema
Range	request header	The Range request HTTP header advertises which bytes the client wants to receive in the response body. Default: no range request Format: bytes=<range-start>-<range-end> range-start: An integer indicating the beginning of the request range. The index is 0-based. range-end: An optional integer indicating the end of the request range. The end is inclusive, so bytes=0-4 would return 5 bytes: 0, 1, 2, 3 and 4. If omitted, the end of the response body is taken as the end of the range.	No (Yes, for response splitting)	string

Name

Located in

Description

Required

Schema

Range

request header

The Range request HTTP header advertises which bytes the client wants to receive in the response body.

Default: no range request
Format: bytes=<range-start>-<range-end>
range-start: An integer indicating the beginning of the request range. The index is 0-based.
range-end: An optional integer indicating the end of the request range. The end is inclusive, so bytes=0-4 would return 5 bytes: 0, 1, 2, 3 and 4.
If omitted, the end of the response body is taken as the end of the range.

No
(Yes, for response splitting)

string

Response

Name	Located in	Description	Required	Schema
Content-Range	response header	The Content-Range response header advertises which bytes the server has sent in the response body. Format: bytes <range-start>-<range-end>/<size> range-start: An integer indicating the beginning of the request range. The index is 0-based. range-end: An integer indicating the end of the request range. The end is inclusive, so bytes 0-4/100 would return 5 bytes: 0, 1, 2, 3 and 4. size: The total byte length of the original response. Example: Content-Range: bytes 1024-2047/3629	Yes when status is 206; No otherwise	string
Content-Length	response header	In the context of range requests, the Content-Length response header always reflects the actual sent response body length in bytes. Example: The client request Range: bytes=0-1023. The server responds with Content-Length: 1024 and Content-Range: bytes 0-1023/3629	Yes	number
Accept-Ranges	response header	The Accept-Ranges response header advertises whether this endpoint accepts HTTP range requests. The value indicatates the supported unit to define a range. Only the unit bytes is supported. If the Accept-Ranges response header is present, but its value is none, range requests are not supported.	No	string

Name

Located in

Description

Required

Schema

Content-Range

response header

The Content-Range response header advertises which bytes the server has sent in the response body.

Format: bytes <range-start>-<range-end>/<size>
range-start: An integer indicating the beginning of the request range. The index is 0-based.
range-end: An integer indicating the end of the request range. The end is inclusive, so bytes 0-4/100 would return 5 bytes: 0, 1, 2, 3 and 4.
size: The total byte length of the original response.

Example: Content-Range: bytes 1024-2047/3629

Yes when status is 206;

No otherwise

string

Content-Length

response header

In the context of range requests, the Content-Length response header always reflects the actual sent response body length in bytes.

Example:
The client request Range: bytes=0-1023.
The server responds with Content-Length: 1024 and Content-Range: bytes 0-1023/3629

Yes

number

Accept-Ranges

response header

The Accept-Ranges response header advertises whether this endpoint accepts HTTP range requests. The value indicatates the supported unit to define a range.

Only the unit bytes is supported.

If the Accept-Ranges response header is present, but its value is none, range requests are not supported.

string

Examples

The following cURL command demonstrates how HTTP range requests could be used:

curl -i --request GET \
 --url device.dev.connectivity-suite.cloud/.../onboarding \
 --header 'Accept: application/json' \
 --header 'Range: bytes=0-1023'

This gives the following response:

HTTP/2 206
server: awselb/2.0
date: Mon, 08 Feb 2021 13:13:54 GMT
content-type: application/json; charset=utf-8
content-length: 1024
etag: W/"400-xWQvmq9+I96lV5hagdjlteB5l2o"
content-range: bytes 0-1023/3163
access-control-allow-origin: *
x-powered-by: Express

{"certificate":"-----BEGIN CERTIFICATE-----\nMIIDWjCCAkKgAwIBAgIVAOtPQsvJMiW4AZxB+JI/Y2msvE+OMA0GCSqGSIb3DQEB\nCwUAME0xSzBJBgNVBAsMQkFtYXpvbiBXZWIgU2VydmljZXMgTz1BbWF6b24uY29t\nIEluYy4gTD1TZWF0dGxlIFNUPVdhc2hpbmd0b24gQz1VUzAeFw0yMDA3MDMwNjM5\nMjJaFw00OTEyMzEyMzU5NTlaMB4xHDAaBgNVBAMME0FXUyBJb1QgQ2VydGlmaWNh\ndGUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDkTbzZIePmMoeD1QTj\nqgV0cueKMP1jui4aKF/f46bxOYmy8rfVfbngf/xBQKLWIa3RkgPzdZCOuKLUHU6q\nAOPL102tfkr2UA4EJ7Ddi8PFQESKpyoBrX8fDeabo5iriequvKi2skAUTgr9FlPp\nMLnWNmxEfjFGR7P9jb2EPqnqSkVfC5a8oWP+Y3qS/oI717lYU7rDfR8ZGmmxp+p6\nfgR1a/Y5Zp47mWQZ4J01psMqDCA3WKh3WC5IACuACiJzBdGCJcCpRL9FWoC6YNao\nOK6YpbkhQPEgGXb7Z+JfnelKF9CQdOKrodLBi0zBRuhrxAqAINwtSU7S+LW4wnb1\nyXE/AgMBAAGjYDBeMB8GA1UdIwQYMBaAFCr/xHqxbPxhtRGXodmKCMZ5aljyMB0G\nA1UdDgQWBBQTSiQQOpoWXQ6nZgsss8i1Q/XrBDAMBgNVHRMBAf8EAjAAMA4GA1Ud\nDwEB/wQEAwIHgDANBgkqhkiG9w0BAQsFAAOCAQEAmsh6AJFKknKSFTyqcKwPErQj\nbx1XVvzwH88EINU0/2A4cjG+7jqdiNAQPksZjqvr5gQHn/tBBYOXbfndKxgqNBGk\nM9DFuYV7ABryfYb3jB0BHpw6L3NfTCtBvpXupyqHFZwEnope3ApWHwk

Errors

Malformed Range header

If the request Range header is not in the correct format, the server will respond with a 400 Bad Request status and the message “Malformed Range header“.

curl --request GET \
 --url device.dev.connectivity-suite.cloud/.../onboarding \
 --header 'Accept: application/json' \
 --header 'Range: bytes=yikes'

Response:

400 Bad Request

{
 "statusCode": 400,
 "statusText": "Bad Request",
 "errors": [
 {
 "message": "Malformed Range header: yikes"
 }
 ]
}

Unsatisfiable Range header

If both the range-start and range-end values of the requestRange header are outside the available range, the server will respond with a 416 Range Not Satisfiable status and the message “Unsatisfiable Range header“.

curl --request GET \ --url device.dev.connectivity-suite.cloud/.../onboarding \ --header 'Accept: application/json' \ --header 'Range: bytes=10000-10999'Response:

416 Range Not Satisfiable

{
 "statusCode": 416,
 "statusText": "Range Not Satisfiable",
 "errors": [
 {
 "message": "Unsatistiable Range header: bytes=10000-10999"
 }
 ]
}

Known limitations

Multipart ranges

Multipart ranges are not supported and will return the original complete response.

Conditional range requests

Conditional range requests using an If-Range header is not supported and will be ignored.

HEAD requests

Currently, HEAD requests will always have a Content-Length response header of 0, so they can not be used to retrieve the total byte length of the response beforehand.

Workaround

As a workaround, the total response byte length can be retrieved from the response Content-Range header when doing an HTTP range request. The total byte length is the value after the slash.

For example: In the response header Content-Range: bytes 0-1023/4298 the value 4298 would be the total response byte length.

References

https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests

https://tools.ietf.org/html/rfc7233

RE: 1NCE Onboarding SIM Card

danielboe — Mon, 18 Jan 2021 18:48:57 GMT

Hi Jan,

thanks for your mail and your feedback. It´s great to hear that we can find a solution here. Let me know if I can give you feedback for the planned API.

I will post some code here if I get the deployment to work with our devices.

Best regards

Daniel

RE: 1NCE Onboarding SIM Card

Jan Sulaiman - 1NCE — Mon, 18 Jan 2021 13:05:11 GMT

Hi Daniel,

Thanks for sharing your experience. I am Jan, Product Manager IoT at 1NCE and responsible for the Connectivity Suite which you have been using.

We are aware of this constraint already for a while. Unfortunately at the moment, we do not have a direct workaround. The response for the Onboarding call is too big and will not fit into the 2kb which are supported by the device. Even when using txt/csv as response type it will be too big.

Therefore we have already planned an additional feature that will allow the onboarding call to be split into multiple responses. It will be a Query Parameter that will allow you to only request the individual fields in the response body. That way each response message is under 2kb and they can be used locally then. Since this has now been requested by a customer I will try to get the development done in our next Development sprint which would allow us to release this feature in the next 2 weeks.

I am hoping for your understanding. I will update this thread here with more details directly from the Development Team as soon as I get them.

We hope that we can count you in to be the first tester of this new feature as soon as available.

Thanks,

Jan

Product Manager IoT @1NCE

Mobile: +49 175 552 4825 | jan.sulaiman@1nce.com | www.1nce.com

RE: 1NCE Onboarding SIM Card

danielboe — Fri, 15 Jan 2021 16:02:09 GMT

okay, sounds good. I will talk to 1nce in this case. Also its still strange that also the HEAD request returns Content-length:0 also with 200 OK. Thanks for your help here!

RE: 1NCE Onboarding SIM Card

Didrik Rokhaug — Fri, 15 Jan 2021 15:52:13 GMT

[quote user="danielboe"]So if the range header is not supportet on the server side there is no way to get the Content? [/quote]

It is possible to run the TLS stack on the application side. In that case, you could set the TLS buffer size as large as you want.

This is supported in the serial_lte_modem if you want to see how it can be done.

But, this will come with a significant flash and RAM cost.

Another option would be to try to convince 1nce to add support for partial content requests or add more API calls to get each key by itself (assuming each individual key is small enough).

Other than those options, I don't know of any other way.

RE: 1NCE Onboarding SIM Card

danielboe — Fri, 15 Jan 2021 15:40:16 GMT

Okay the TLS Part I understand and makes completely sense to me. So if the range header is not supportet on the server side there is no way to get the Content?

RE: 1NCE Onboarding SIM Card

Didrik Rokhaug — Fri, 15 Jan 2021 15:36:41 GMT

The download_client asks for a specific range: "Range: bytes=%u-%u\r\n"

The start of the range is the current offset, while the end is MIN(offset + wanted fragment size, total file size).

[quote user="danielboe"]Okay but shouldn´t I get anything from the server also if the size is bigger?[/quote]

What happens is that the device will start downloading encrypted data, but it cannot decrypt it until the whole TLS fragment is downloaded. The encrypted data is therefore stored in a buffer of 2kB. But, as the TLS fragment is larger, the buffer cannot hold it all, and the fragment cannot be decrypted. It is therefore dropped.

RE: 1NCE Onboarding SIM Card

danielboe — Fri, 15 Jan 2021 15:21:26 GMT

If I do a HEAD request, I get a response. Also I get a 503 Error if I change the URL Path. So this seems to work. now is the question how to download the content.

HTTP/1.1 200 OK

Server: awselb/2.0

Date: Fri, 15 Jan 2021 15:15:58 GMT

Content-Type: text/csv; charset=utf-8

Content-Length: 0

Connection: keep-alive

etag: W/"10c3-H17P8ulLBfQxnRd+jSKR4RP6v3U"

access-control-allow-origin: *

x-powered-by: Express

RE: 1NCE Onboarding SIM Card

danielboe — Fri, 15 Jan 2021 15:01:32 GMT

Okay but shouldn´t I get anything from the server also if the size is bigger? I mean the first package at least and normaly the revc loops as long as there is content. So in the demo implementation from once in RTOS it does exactly this: https://pastebin.com/edA7HMLg also they use port 443 and TCP for the socket whats also strange for me

RE: 1NCE Onboarding SIM Card

Didrik Rokhaug — Fri, 15 Jan 2021 14:51:29 GMT

One thing you could try, is to use the content-range header field to see if you can download just a part of the certificates.

This is what we are doing in our download_client, to download DFU images.

RE: 1NCE Onboarding SIM Card

danielboe — Fri, 15 Jan 2021 14:13:30 GMT

response is a bit bigger (with certificates) but i dont think 2kb. What would be the solution here?

Edit: I think it is over 2KB limit because the single certs are even bigger.

HTTP/2 200

server: awselb/2.0

date: Mon, 22 Jun 2020 14:26:15 GMT

content-type: application/json; charset=utf-8

content-length: 3156

etag: W/“c54-l8hc4And5ODwW2GDRbihJtLzNrI“

access-control-allow-origin: *

{

„certificate“: „—–BEGIN CERTIFICATE—–\HERE_WOULD_BE_CERTIFICATE\n—–END CERTIFICATE—–\n“,

„privateKey“: „—–BEGIN RSA PRIVATE KEY—–\HERE_WOULD_BE_KEY\n—–END RSA PRIVATE KEY—–\n“,

„amazonRootCaUrl“: „www.amazontrust.com/.../AmazonRootCA1.pem“,

„iotCoreEndpointUrl“: „a259hu9tuXXXXX-ats.iot.eu-central-1.amazonaws.com“,

„ICCID“: „898828066600000XXXX”

}

RE: 1NCE Onboarding SIM Card

Didrik Rokhaug — Fri, 15 Jan 2021 14:10:51 GMT

How big is the response from the server?

The nRF9160 has a size limit for TLS fragments of 2kB.

RE: 1NCE Onboarding SIM Card

danielboe — Fri, 15 Jan 2021 13:52:35 GMT

Hi Didrik, thanks for the answer. I tried to set an empty tag space with "err = setsockopt(fd, SOL_TLS, TLS_SEC_TAG_LIST, 0, 0);" and set TLS for the socket. If I do so the answer is now empty from the server so there must be some other problem.

RE: 1NCE Onboarding SIM Card

Didrik Rokhaug — Fri, 15 Jan 2021 13:37:53 GMT

Hi,

In the attached code, you are opening a "normal" TCP socket, instead of a TLS socket.

When using TLS, you must use IPPROTO_TLS_1_2 as the socket protocol.

In addition, you have to at least set the sec_tag where the certificates to be used are stored (this gets done if you call tls_setup).

To get the CA certificate of the server, you can use tools such as openssl.

Best regards,

Didrik