FOTA support for authentication tokens

[Randall]

I've been attempting to use the FOTA module to download application updates.  We are using AWS and generating URLs on the fly that are received via MQTT messages.  The URLs come with authentication tokens which FOTA doens't seem to understand and mangles the URL in an attempt to break it in to hostname and path.   The command looks like this;

AT#XFOTA=1,"http://v2-firmware-bucket.s3.us-east-2.amazonaws.com/firmware/version%3D1.1.0/slm_app-1.1.bin?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIA3EZPSMKGNDANBL37%2F20230301%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20230301T211852Z&X-Amz-Expires=3600&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEO3%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMiJHMEUCIG92VWqQoRG0CgCPrWpycxFY%2FpU99ap6ZOGUG1LBbNoTAiEAwvKv%2FUf6j1JcdPRM7CX57pExlCJvwk%2Byxh3E8yg7wUkqsgMIlv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgw3NjYyMTQ1NjI0NDQiDE2sAUUCvXvz4aqNoiqGA%2FUQsEWje%2BH485x00YUkRNnw%2FqNn5ULItaJh6WG3QxYoCNUW2MEJXReYJfTfpkKtA9slJgkYzboob9dSR1KMJmnro0mUEc4Iqt7yl1pYeoiQ82a5wdL8XqEvO%2BEHqHRc8UvRMKOHiX7sIUvNKt6UP7KVRE2UQbQ5xzDUjttg0l4V5EdsZIFejIL1oHRm5KuEgbixlMGRunZZGPYhVYQXdVj5MgLCCL0G4KoeL8TPfDTgV5OSKqlhym%2F7AsO8u5NlR56ZZGwi5skGH5XN%2BQqGkXvOOgJYPhQ%2FSpvzdGNyrWDJcMJwziELoPnrIJRQeexuI5lwJACiQCpOjHxKbg5WkxZE8T0P0oba3510zWyQo52p8RvzLYhjzaWJfY9%2B9004RD1F5xMBoxZIvLz%2ButAWFUKxDg%2Bg7FrOMVfBrWvTRNo31Akg7vOzI9B0uW%2Bqpv7l8mEN14s7Z%2BWvmgRAohzUgX72uxtQbJaWy8O4YHJDEcmeme89fLVbFXetVWl%2FC0H9586Mhs7VvDC5gv%2BfBjqdATlRvtBZ4ZH2gYPOa4BDN8LMDoOxtXjT5p8zDmcvjxfj1mCRFsNeJinvvJ81JN6%2F2%2F7OLpjqeMpdFBa0SoTTXsCuhjAbILLgwEaHGp4om8be9VgHrXCg1mBpr7e9iT%2B%2FkPOwV%2Fk9mHaDcxGgJ4N025V1CDI80wwutHjaeHfn0MLzdmnQeJZ%2B%2B2wRNl3POmUXn7TWADB0uzazDSlzpz8%3D&X-Amz-Signature=40fab953a2dfa48427d393abe8b9e959a369ce2848caef92ef8a7e7fe2dcdcf7&X-Amz-SignedHeaders=host"

Is there anyway to get FOTA to use these tokens without having to rewrite a bunch of the module.

Randall

[Øyvind]

Hello Randall, 

My apologies for the late reply. I was assigned this ticket while I was out traveling. 

Are you able to share what kind of S3 buckets policy you are running? Will discuss the issue with our developers. 

Kind regards,
Øyvind

[Randall]

{
    "Action": [
      "s3:GetObject",
      "s3:ListBucket",
      "s3:GetBucketLocation",
      "s3:GetObjectVersion",
      "s3:GetLifecycleConfiguration"
    ],
    "Resource": [
      "arn:aws:s3:::v2-firmware-bucket",
      "arn:aws:s3:::v2-firmware-bucket/*"
    ],
    "Effect": "Allow"
}

[Øyvind]

Randall, what error do you get when running the command? 

First of all, the authentication token or presigned file, is possibly too long for the filename max. length. 

config DOWNLOAD_CLIENT_MAX_HOSTNAME_SIZE
        int "Maximum hostname length (stack)"
        range 8 256
        default 64
 
config DOWNLOAD_CLIENT_MAX_FILENAME_SIZE
        int "Maximum filename length (stack)"
        range 8 2048
        default 192

Currently, our download_client does not support this out of the box. There have been some customers asking about the same, and seeing the same issue e.g.  AWS FOTA using presigned URL from private S3 bucket

According to our developers, the path needs to follow the format as described in fota_download_start() function description

/**@brief Start downloading the given file from the given host.
 *
 * When the download is complete, the secondary slot of MCUboot is tagged as having
 * valid firmware inside it. The completion is reported through an event.
 *
 * @param host Name of host to start downloading from. Can include scheme
 *             and port number, e.g. https://google.com:443
 * @param file Path to the file you wish to download. May be either a single
 *              file path, relative to host (for example "path/to/binary.bin"),
 *              or, if bootloader FOTA is enabled, can be two paths (both relative
 *              to host), separated by a space (for example "path/to/s0.bin path/to/s1.bin").
 *              If two paths are provided, the download will be assumed to be a bootloader
 *              download, both paths will be treated as upgradable bootloader slot 0
 *              and slot 1 binaries respectively, and only the binary corresponding to
 *              the currently inactive bootloader slot will be selected and downloaded.
 *              See <a href="https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/ug_bootloader.html">
 *              Secure Bootloader Chain Docs</a> for details regarding the upgradable
 *              bootloader slots.
 * @param sec_tag Security tag you want to use with HTTPS set to -1 to Disable.
 * @param pdn_id Packet Data Network ID to use for the download, or 0 to use the default.
 * @param fragment_size Fragment size to be used for the download.
 *			If 0, @kconfig{CONFIG_DOWNLOAD_CLIENT_HTTP_FRAG_SIZE} is used.
 *
 * @retval 0	     If download has started successfully.
 * @retval -EALREADY If download is already ongoing.
 *                   Otherwise, a negative value is returned.
 */
int fota_download_start(const char *host, const char *file, int sec_tag,
			uint8_t pdn_id, size_t fragment_size);

'

Can you provide log output from your device with the following enabled in prj.conf:

CONFIG_DOWNLOAD_CLIENT_LOG_LEVEL_DBG=y
CONFIG_DOWNLOAD_CLIENT_LOG_HEADERS=y