Technical Thursday – Bug in azure-cli 2.0.30

In this week I would like to inform you about a bug in azure-cli 2.0.30 which can cause some inconveniences when you want to copy blobs in Azure storage accounts.

Some days ago I started to create a solution for copy files inside storage account (this is related a git pipeline solution) and I was facing an issue when I wanted to use ” az storage blob copy start” command with “–sas-token” parameter. The command was quite simple:

az storage blob copy start --account-name mystorageacc --sas-token "?sv=2017-07-29&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-04T15:43:14Z&st=2018-04-03T23:43:14Z&spr=https&sig=***************" --destination-container "files" --source-container "files" --source-blob "new/arm-template.json" --destination-blob "archive/arm-template.json"

and I received the following error:

The specified resource does not exist.ErrorCode: CannotVerifyCopySource
<?xml version="1.0" encoding="utf-8"?><Error><Code>CannotVerifyCopySource</Code><Message>The specified resource does not exist.
RequestId:fe725383-701e-0002-5aae-ccdd02000000
Time:2018-04-05T07:20:26.8836489Z</Message></Error>
Traceback (most recent call last):
  File "/usr/lib64/az/lib/python2.7/site-packages/knack/cli.py", line 197, in invoke
    cmd_result = self.invocation.execute(args)
  File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 347, in execute
    six.reraise(*sys.exc_info())
  File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 319, in execute
    result = cmd(params)
  File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 180, in __call__
    return super(AzCliCommand, self).__call__(*args, **kwargs)
  File "/usr/lib64/az/lib/python2.7/site-packages/knack/commands.py", line 109, in __call__
    return self.handler(*args, **kwargs)
  File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/core/__init__.py", line 420, in default_command_handler
    result = op(**command_args)
  File "/usr/lib64/az/lib/python2.7/site-packages/azure/multiapi/storage/v2017_07_29/blob/baseblobservice.py", line 3032, in copy_blob
    False)
  File "/usr/lib64/az/lib/python2.7/site-packages/azure/multiapi/storage/v2017_07_29/blob/baseblobservice.py", line 3102, in _copy_blob
    return self._perform_request(request, _parse_properties, [BlobProperties]).copy
  File "/usr/lib64/az/lib/python2.7/site-packages/azure/multiapi/storage/v2017_07_29/common/storageclient.py", line 354, in _perform_request
    raise ex
AzureMissingResourceHttpError: The specified resource does not exist.ErrorCode: CannotVerifyCopySource
<?xml version="1.0" encoding="utf-8"?><Error><Code>CannotVerifyCopySource</Code><Message>The specified resource does not exist.
RequestId:fe725383-701e-0002-5aae-ccdd02000000
Time:2018-04-05T07:20:26.8836489Z</Message></Error>

I started to check the possible reasons but finally I registered an issue on Azure/azure-cli on Git to MS: az storage blob copy start issue when I use “sas token”

Some days later I received this answer:

@the1bit Thanks for bringing this to our attention.
#6041 will apply the sas token specified by –sas-token for the source as well as the destination and will be available in our next release.

For now, please use –source-sas to apply the same sas towards your source, as –sas-token currently only applies towards the destination.

I tested again then I was sure there is a bug in the code. I used this command:

az storage blob copy start --account-name mystorageacc --sas-token "?sv=2017-07-29&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-04T15:43:14Z&st=2018-04-03T23:43:14Z&spr=https&sig=***************" --destination-container "files" --source-container "files" --source-blob "new/arm-template.json" --destination-blob "archive/arm-template.json" --source-account-name mystorageacc --source-sas "?sv=2017-07-29&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-04T15:43:14Z&st=2018-04-03T23:43:14Z&spr=https&sig=***************" --debug

And the error was a new one:

AzureHttpError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.ErrorCode: CannotVerifyCopySource
<?xml version="1.0" encoding="utf-8"?><Error><Code>CannotVerifyCopySource</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

Then I found a strange thing in debug output:

'x-ms-copy-source': 'https://mystorageacc.blob.core.cloudapi.de/files/new/arm-template.json??sv=2017-07-29&ss=bfqt&srt=sco&sp=rwdlacup&se=2118-04-10T16:23:59Z&st=2018-04-10T08:23:59Z&spr=https&sig=4dCQpoGDZnHZY%2FCk0TXKXtH6I%2BzZP%2BTW2ZkErE1LgjQ%3D',

As you can see there is a double ‘?’ around SAS token:

.json??sv=

Then I tested the –source-sas without ‘?’:

--source-sas "sv=2017-07-29&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-04T15:43:14Z&st=2018-04-03T23:43:14Z&spr=https&sig=***************"

and the copy works:

{
  "completionTime": null,
  "id": "077b06d7-d843-4fc3-ba8b-e48091753869",
  "progress": null,
  "source": null,
  "status": "success",
  "statusDescription": null
}

I sent this to MS and some another days later:

@the1bit I’ve raised a new issue for the bug you found: #6073
Thanks for finding this!

Workaround

There is a bug in “az storage blob copy start” with “–source-sas” parameter in azure-cli 2.0.30. I am sure they will fix this soon. MEanwhile you can apply the following workarounds:

1. use your sas token in “–source-sas” parameter without ‘?’
2. use storage account key instead of  sas token.

I hope this helps to avoid some struggling until the fix will be here.