Download Client Redirect Querry

Hi,

I am using download_client to get a bigger file (100KB) to my memory via LTE-M. The Files are located in a bucket and I use a shortURL to get the signedDownload URL. If I use this short URL in Download Client I get the Error "http/1.1 301 moved permanently". This seems to be correct, the URL is initialy ofcourse a redirect. But how can I handle this in my code? I couldnt find a way to read the header files with download client nor do I see any option for redirect hops or something like this. Is there even a solution? Or is it the limits of the low level integration?

Thanks for helping and best regards

daniel

Parents
  • Hi Daniel,

    The Files are located in a bucket and I use a shortURL to get the signedDownload URL.

    Can you show how (and where in code) you use this shortURL?

    If I use this short URL in Download Client I get the Error "http/1.1 301 moved permanently"

    Can you provide full log which contains the error?

    Best regards,
    Dejan

  • The ShortUrl is generated on a Serverside and send to the device. I can give you some snippet that shows how it works.

    prj.conf:

    # Download Client
    CONFIG_DOWNLOAD_CLIENT=y
    CONFIG_DOWNLOAD_CLIENT_STACK_SIZE=4096
    CONFIG_DOWNLOAD_CLIENT_LOG_LEVEL_DBG=y


    there is Downloader:
    #define URL "http://ul1.epaperframe.de/NDUmX2z.dl"
    static struct download_client downloader;
    static struct download_client_cfg config = {};      
          
            if (awsGetCloudConnected() && !doOnce) {
                doOnce = true;
                LOG_INF("[MAIN] DL INIT");
                int err = download_client_init(&downloader, callbackDl);
                if (err) {
                    printk("[MAIN] Failed to initialize the client, err %d", err);
                }
    
                err = download_client_get(&downloader, URL, &config, URL, 0);
                if (err) {
                    printk("[MAIN] Failed to start the downloader, err %d", err);
                    return 0;
                }
    
                printk("[MAIN] Downloading %s\n", URL);
            }
    
            if (dlClientDone) {
                dlClientDone = false;
                char recoverDataStr[2050] = {};
                strncpy(recoverDataStr, downloader.buf, sizeof(recoverDataStr));
                printk("[DL] Payload: \"%s\"\n", recoverDataStr);
            }

    And there is the callback:

    static int callbackDl(const struct download_client_evt *event) {
        static size_t downloaded;
        static size_t file_size;
    
        if (downloaded == 0) {
            download_client_file_size_get(&downloader, &file_size);
            downloaded += 0;
        }
    
        switch (event->id) {
            case DOWNLOAD_CLIENT_EVT_FRAGMENT:
                downloaded += event->fragment.len;
                if (file_size) {
                    progress_print(downloaded, file_size);
                    char recoverDataStr[2200] = {};
                    strncpy(recoverDataStr, event->fragment.buf, event->fragment.len);
                    printk("%s \n", recoverDataStr);
                } else {
                    printk("\r[ %d bytes ] ", downloaded);
                }
                return 0;
            case DOWNLOAD_CLIENT_EVT_CLOSED:
                printk("[DL] CLOSED\n");
                dlClientDone = true;
                return 0;
            case DOWNLOAD_CLIENT_EVT_DONE:
                printk("[DL] DONE\n");
                return 0;
            case DOWNLOAD_CLIENT_EVT_ERROR:
                printk("[DL] Error %d during download\n", event->error);
                return -1;
        }
        return 0;
    }

    I am not sure what you mean by more log, but here is a sample output with LOG DEBUG (DOWNLOAD_CLIENT_LOG_LEVEL_DBG):

    00> *** Booting nRF Connect SDK v2.7.0-5cb85570ca43 ***
    00> *** Using Zephyr OS v3.6.99-100befc70c74 ***
    00> I: [MAIN] Starting application 2.0.1
    00> I: 16 Sectors of 4096 bytes
    00> I: alloc wra: 2, fc0
    00> I: data wra: 2, a20
    00> I: [LED] led sample
    00> I: [SPI] init
    00> I: [DISPLAY] sleep start
    00> I: [DISPLAY] WAIT finished
    00> I: [DISPLAY] sleep done
    00> I: [MAIN] Chip is NEW - set LTE-M
    00> I: [MODEM] Modem Band Lock Set
    00> I: [MODEM] Version: 2.0.1
    00> I: [MODEM] Network registration status: Connected - roaming
    00> I: [MODEM] PSM parameter update: TAU: 3240, Active time: -1
    00> I: [MODEM] client_id: nrf-35940*
    00> I: [AWS] iot_connect - init done
    00> W: [AWS] iot_reconnect - wait 30 s connect try: 1/30
    00> W: [AWS] IOT_EVT_CONNECTED
    00> I: [MODEM] BOOT Image Confirmed
    00> I: [MAIN] DL INIT
    00> D: state = 1
    00> I: Downloading: http://ul1.epaperframe.de/NDUmX2z.dl [0]
    00> [MAIN] Downloading http://ul1.epaperframe.de/NDUmX2z.dl
    00> D: Port not specified, using default: 80
    00> D: Failed to resolve hostname ul1.epaperframe.de on IPv6
    00> D: family: 1, type: 1, proto: 6
    00> I: Connecting to 3.5.134.246
    00> D: fd 1, addrlen 8, fam IPv4, port 80
    00> D: state = 2
    00> D: Receiving up to 2048 bytes at 0x2000cbd8...
    00> D: Read 620 bytes from socket
    00> D: GET header size: 620
    00> E: Unexpected HTTP response: 301 moved permanently
    00> [DL] Error -77 during download
    00> D: state = 4
    00> D: state = 0
    00> [DL] CLOSED
    00> D: Connection closed
    00> [DL] Payload: "http/1.1 301 moved permanently"

    The links are save to stay public But they wont work anymore after some time

    The Download Client Sample worked fine btw. so thats not the issue.

  • Hi,

    Can you show your full project configuration?

    Do you use this Download Client sample?

    Which NCS version do you use?

    Best regards,
    Dejan

Reply Children
  • exactly, this was my starting point. also downloading the sample works in my code. the config itself can indeed work. here is the full config I use:

    CONFIG_REBOOT=y
    CONFIG_CRC=y
    
    # GPIO
    CONFIG_GPIO=y
    CONFIG_PWM=y
    
    #RTT Log
    CONFIG_LOG=y
    CONFIG_LOG_MODE_MINIMAL=y
    CONFIG_UART_CONSOLE=n
    CONFIG_RTT_CONSOLE=y
    CONFIG_LOG_BACKEND_UART=n
    CONFIG_USE_SEGGER_RTT=y
    CONFIG_LOG_DEFAULT_LEVEL=3
    
    # Heap and stacks
    CONFIG_HEAP_MEM_POOL_SIZE=8096
    CONFIG_MAIN_STACK_SIZE=4096
    CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048
    
    
    #WDT
    CONFIG_WDT_LOG_LEVEL_DBG=y
    CONFIG_WATCHDOG=y
    CONFIG_WDT_DISABLE_AT_BOOT=n
    
    # SPI
    CONFIG_SPI=y
    
    # LTE link control
    CONFIG_LTE_LINK_CONTROL=y
    CONFIG_LTE_NETWORK_MODE_NBIOT=y
    CONFIG_MODEM_INFO=y
    CONFIG_MODEM_KEY_MGMT=y
    
    #Networking
    CONFIG_NET_SOCKETS=y
    CONFIG_NETWORKING=y
    CONFIG_NET_NATIVE=n #TODO check this one
    CONFIG_NRF_MODEM_LIB=y
    
    # Date Time library
    CONFIG_DATE_TIME=y
    CONFIG_DATE_TIME_MODEM=y
    
    # cJSON
    CONFIG_CJSON_LIB=y
    
    # Image manager
    CONFIG_FLASH_MAP=y
    CONFIG_STREAM_FLASH=y
    CONFIG_MCUBOOT_IMG_MANAGER=y
    CONFIG_IMG_MANAGER=y
    
    #OTA
    CONFIG_FLASH=y
    CONFIG_FLASH_PAGE_LAYOUT=y
    CONFIG_IMG_ERASE_PROGRESSIVELY=y
    CONFIG_BOOTLOADER_MCUBOOT=y
    
    # AWS FOTA
    CONFIG_AWS_FOTA=y
    CONFIG_FOTA_DOWNLOAD=y
    CONFIG_FOTA_SOCKET_RETRIES=200
    CONFIG_FOTA_DOWNLOAD_PROGRESS_EVT=y
    CONFIG_DFU_TARGET=y
    
    # Download Client
    CONFIG_DOWNLOAD_CLIENT=y
    CONFIG_DOWNLOAD_CLIENT_STACK_SIZE=4096
    CONFIG_DOWNLOAD_CLIENT_LOG_LEVEL_DBG=y
    
    #AWS IOT Settings
    CONFIG_AWS_IOT=y
    CONFIG_AWS_IOT_BROKER_HOST_NAME="XXX-ats.iot.eu-central-1.amazonaws.com"
    CONFIG_MQTT_HELPER_SEC_TAG=6
    CONFIG_AWS_IOT_TOPIC_UPDATE_DELTA_SUBSCRIBE=y
    CONFIG_AWS_IOT_AUTO_DEVICE_SHADOW_REQUEST=n
    CONFIG_MQTT_CLEAN_SESSION=y
    CONFIG_MQTT_KEEPALIVE=200
    
    # Non-volatile Storage
    CONFIG_NVS=y
    CONFIG_MPU_ALLOW_FLASH_WRITE=y
    CONFIG_PM_PARTITION_SIZE_NVS_STORAGE=0xE000
    CONFIG_LZ4=y

  • Hi,

    dejans said:
    Which NCS version do you use?

    Which NCS version and modem firmware version do you use?

    Best regards,
    Dejan

  • Hi,

    Unfortunately, there is no support for URL shorteners in download client library. Resolving short URL before calling download client library with expanded URL is currently the solution. In your case, you could consider doing an HTTP HEAD request using the Zephyr HTTP library, then read out the Location header with the expanded URL and use that subsequently with the download client. It is also recommended that you use HTTPS instead of HTTP due to security concern.

    Best regards,
    Dejan

  • This sounds like what I want. Is there any sample to read headers from a request?

Related