Azure Python 下载存储 blob 返回“不满足使用 HTTP 条件标头指定的条件”

如何解决Azure Python 下载存储 blob 返回“不满足使用 HTTP 条件标头指定的条件”

使用以下代码：

import os
from azure.identity import (
    ClientSecretCredential
)

# Import the client object from the Azure library
from azure.storage.blob import BlobClient

t_id = "<tenant_id>"
c_id = "<client_id>"
s_acct = "<storage_account_name>"
s_acct_url = "%s.blob.core.windows.net" % s_acct
sek = "<client_sekret>"

print("+ Setup credentials.")
credential = ClientSecretCredential(t_id,c_id,sek)
print("+ Setup Blob Client")
bobc = BlobClient(s_acct_url,<container_name>,<blob_name>,credential=credential)
print("+ Setup streamer")
ssd = bobc.download_blob()
print("+ Get properties")
print(ssd.get_blob_properties())

但我收到以下错误：

$ python azdown.py
+ Setting up stream
+
+ Download stream:
+    Size: 136365212160
Traceback (most recent call last):
  File "C:\git\python38\lib\site-packages\azure\storage\blob\_download.py",line 192,in _download_chunk
    _,response = self.client.download(
  File "C:\git\python38\lib\site-packages\azure\storage\blob\_generated\operations\_blob_operations.py",line 179,in download
    raise models.StorageErrorException(response,self._deserialize)
azure.storage.blob._generated.models._models_py3.StorageErrorException: Operation returned an invalid status 'The condition specified using HTTP conditional header(s) is not met.'

During handling of the above exception,another exception occurred:

Traceback (most recent call last):
  File "azdown.py",line 29,in <module>
    download_stream.download_to_stream(my_blob)
  File "C:\git\python38\lib\site-packages\azure\storage\blob\_download.py",line 579,in download_to_stream
    self.readinto(stream)
  File "C:\git\python38\lib\site-packages\azure\storage\blob\_download.py",line 561,in readinto
    downloader.process_chunk(chunk)
  File "C:\git\python38\lib\site-packages\azure\storage\blob\_download.py",line 125,in process_chunk
    chunk_data = self._download_chunk(chunk_start,chunk_end - 1)
  File "C:\git\python38\lib\site-packages\azure\storage\blob\_download.py",line 201,in _download_chunk
    process_storage_error(error)
  File "C:\git\python38\lib\site-packages\azure\storage\blob\_shared\response_handlers.py",line 147,in process_storage_error
    raise error
azure.core.exceptions.ResourceModifiedError: The condition specified using HTTP conditional header(s) is not met.
RequestId:<request id>
Time:2021-01-10T01:23:24.8981731Z
ErrorCode:ConditionNotMet
Error:None

我查看了 [1]，但我使用的是 download_to_stream()。然后我尝试了 [2] 但我的存储是一个 gen1 容器。有一个链接说它与容器的网络权限有关，但容器设置为允许所有人下载。问题是，它确实下载到了某个点（atm，120G 中的 1.2G）然后它窒息了。所以这不是权限问题，而是某种竞争条件。

[1] - https://social.msdn.microsoft.com/Forums/azure/en-US/3b4df832-3340-4415-8d93-d71662e1c540/azure-blob-the-condition-specified-using-http-conditional-headers-is-not-met

[2] - https://forums.databricks.com/questions/20415/intermittent-http-error-when-loading-files-from-ad.html

[编辑]

我也尝试过使用 connection_string 方法：

credential = DefaultAzureCredential()

    blob_url = "DefaultEndpointsProtocol=https;AccountName=<storage_acct>;" + \
               "AccountKey=<account_key>;" + \
               "EndpointSuffix=core.windows.net"

    # Create the client object using the storage URL and the credential
    blob_client = BlobClient.from_connection_string(
        blob_url,container_name=<container_name>,blob_name=<blob_item>)

    if download_now:
        with open(<blob_item>,"wb") as my_blob:
            print("+ Setting up stream")
            download_stream = blob_client.download_blob()
            print("+")
            print("+ Download stream:")
            print("+    Size: %d" % download_stream.size)
            download_stream.download_to_stream(my_blob)

    else:
        print(blob_client.get_blob_properties())

即使在播放了大约一场演出后，我也遇到了同样的错误。

错误：

Traceback (most recent call last):
  File "C:\git\python38\lib\site-packages\azure\storage\blob\_download.py",another exception occurred:

Traceback (most recent call last):
  File "azuredl.py",line 52,in process_storage_error
    raise error
azure.core.exceptions.ResourceModifiedError: The condition specified using HTTP conditional header(s) is not met.
RequestId:<request id>
Time:2021-01-10T07:43:24.4927426Z
ErrorCode:ConditionNotMet
Error:None

有人有什么想法吗？

解决方法

当我对正在下载的文件进行一些更新（例如更改内容或添加一些元数据）时，我可以重现此问题：

此问题是由于 blob 更新后，其 Etag 也会同时更改。这导致了这个问题。详情见this doc。

要解决此问题，您可以要求此 blob 的 Lise 添加锁定，以便在下载时不会编辑或删除此 blob。

试试下面的代码：

blob_client = BlobClient.from_connection_string(conn_str='',container_name='',blob_name='')

#require lease that never expires
lease = blob_client.acquire_lease(lease_duration=-1)

with open("<some path>","wb") as my_blob:
            print("+ Setting up stream")
            download_stream = blob_client.download_blob()
            print("+")
            print("+ Download stream:")
            print("+    Size: %d" % download_stream.size)
            download_stream.download_to_stream(my_blob)

#break lease after download
lease.break_lease()