Remove a dead/disconnected device from ZigBee controller.

I have this code that works like a charm when the device is connected and I want it to leave the network.
But if the device is dead or disconnected and I want to  remove it from the controller it fails and I am
looking for the best way to handle this any thoughts ? 
    zb_buf_t *buf = ZB_BUF_FROM_REF(param);
    zb_zdo_mgmt_leave_param_t *req = NULL;
    zb_uint8_t tsn;
    LOG("send_leave 0x%04hX\n", dst_addr);
    req = ZB_GET_BUF_PARAM(buf, zb_zdo_mgmt_leave_param_t);
    ZB_MEMSET(req->device_address, 0, sizeof(zb_ieee_addr_t));
    req->remove_children = ZB_FALSE;
    req->rejoin = ZB_FALSE;
    req->dst_addr = dst_addr;
    tsn = zdo_mgmt_leave_req(param, leave_callback);