我希望从ASP.NET Core Web API在AWS S3存储桶中创建一个大型(多GB)文件.该文件足够大,我希望在将Stream上传到AWS S3之前不将其加载到内存中.
使用PutObjectAsync()我强制在将Stream传递给AWS SDK之前预先填充Stream,如下图所示:
var putObjectRequest = new PutObjectRequest { BucketName = "my-s3-bucket",Key = "my-file-name.txt",InputStream = stream }; var putObjectResponse = await amazonS3Client.PutObjectAsync(putObjectRequest);
我理想的模式将涉及AWS SDK返回StreamWriter(各种类型)我可以多次写入(),然后在完成后返回Finalize().
关于我的挑战的两个问题:
>我是否误以为在调用PutObjectAsync()之前必须预先填充Stream?
>我应该如何上传我的大(多GB)文件?
解决方法
对于这种情况,AWS docs提供了两种选择:
> Using the AWS .NET SDK for Multipart Upload (High-Level API)
> Using the AWS .NET SDK for Multipart Upload (Low-Level API)
高级API只是建议您创建一个指定了PartSize的TransferUtilityUploadRequest
,这样类本身就可以上传文件而无需自己维护上传.在这种情况下,您可以通过订阅StreamTransferProgress
事件获得分段上传的进度.您可以上载文件,流或目录.
显然,Low-level API更复杂,但更灵活 – 您可以启动上传,之后您会在循环中上传文件的下一部分.文档中的示例代码:
var s3Client = new AmazonS3Client(Amazon.RegionEndpoint.USEast1); // List to store upload part responses. var uploadResponses = new List<UploadPartResponse>(); // 1. Initialize. var initiateRequest = new InitiateMultipartUploadRequest { BucketName = existingBucketName,Key = keyName }; var initResponse = s3Client.InitiateMultipartUpload(initRequest); // 2. Upload Parts. var contentLength = new FileInfo(filePath).Length; var partSize = 5242880; // 5 MB try { long filePosition = 0; for (var i = 1; filePosition < contentLength; ++i) { // Create request to upload a part. var uploadRequest = new UploadPartRequest { BucketName = existingBucketName,Key = keyName,UploadId = initResponse.UploadId,PartNumber = i,PartSize = partSize,FilePosition = filePosition,FilePath = filePath }; // Upload part and add response to our list. uploadResponses.Add(s3Client.UploadPart(uploadRequest)); filePosition += partSize; } // Step 3: complete. var completeRequest = new CompleteMultipartUploadRequest { BucketName = existingBucketName,}; // add ETags for uploaded files completeRequest.AddPartETags(uploadResponses); var completeUploadResponse = s3Client.CompleteMultipartUpload(completeRequest); } catch (Exception exception) { Console.WriteLine("Exception occurred: {0}",exception.ToString()); var abortMPURequest = new AbortMultipartUploadRequest { BucketName = existingBucketName,UploadId = initResponse.UploadId }; s3Client.AbortMultipartUpload(abortMPURequest); }
Asynchronous version of UploadPart
也可用,因此如果您需要完全控制上传,则应调查该路径.
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。