如何将我的 AWS CLI 命令转换为 AWS Java API 代码？

如何解决如何将我的 AWS CLI 命令转换为 AWS Java API 代码？

我有一段在 HTTP 服务器中被频繁调用的代码。我得到管道上的响应并使用输出字符串。但显然将其用作 AWS CLI 管道作为 bash 脚本可能效率不高。

与这些命令等效的 AWS JAVA API 是什么？如何转换它们？它是否提高了效率、访问延迟等？

public void run() {
    String command = "aws --endpoint-url=" + S3server + " s3 cp s3://file-store/" + id + "/files/" + id + ".txt -";
    processBuilder.command("sh","-c",command + " ; true");
    Process process = processBuilder.start();
    //String command2 = "aws --endpoint-url=https://s3-ourserver.mystore.mycompany.net s3api head-object --bucket file-store --key FILES/3762/files/mytext.scc  --query 'ContentLength'";

    BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
    BufferedReader readerErrors = new BufferedReader(new InputStreamReader(process.getErrorStream()));

    String line;
    while ((line = reader.readLine()) != null) {
        response += line + "\n";
    }
    //...send the response back via HTTP 
    OutputStream os = he.getResponseBody();
    os.write(response.toString().getBytes());
    os.close();
}

解决方法

从 V2 Java SDK 中的 S3 读取的超级简单的内容如下所示：

Region region = Region.US_WEST_2;
String bucketName = "file-store";

S3Client s3Client = S3Client.builder().region(region).build();

...

GetObjectRequest getObjectRequest = GetObjectRequest.builder()
                                         .bucket(bucketName)
                                         .key(id + "/files/" + id + ".txt")
                                         .build();
OutputStream os = he.getResponseBody();
s3Client.getObject(getObjectRequest,ResponseTransformer.toOutputStream(outputStream));
os.close();

这会从 S3 下载文件并将字节直接写入 HTTP 响应中的输出流。如果您非常频繁地执行此操作，则应该在您进行读取的位置（即构造函数）之外创建 S3Client 并重复使用。

现在，这比您展示的速度快吗？我强烈希望它会是，但我没有办法在您正在运行的负载下对其进行测试。

另一种选择可能是创建一个客户端请求的 presigned URL，然后他们可以直接从 S3 下载文件，而不是通过您的服务器。我不确定这是否适用于您的用例。

编辑 我强烈建议您深入研究 AWS Java SDK docs，并针对您的用例深入研究 S3 Examples。但在那之前，下面是一个获取文件的简单程序以及获取内容长度的两种不同方法。这个使用本地凭据 - 您的用例可能会有所不同。

import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.core.exception.SdkClientException;
import software.amazon.awssdk.core.sync.ResponseTransformer;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.GetObjectResponse;
import software.amazon.awssdk.services.s3.model.HeadObjectRequest;
import software.amazon.awssdk.services.s3.model.HeadObjectResponse;
import software.amazon.awssdk.services.s3.model.NoSuchKeyException;
import software.amazon.awssdk.services.s3.model.S3Exception;

import java.io.FileOutputStream;
import java.io.OutputStream;


public class DownloadObjectV2 {

    public static void main(String[] args) {
        Region region = Region.US_EAST_1;
        String bucketName = "the-bucket-name";
        String fileObjKeyName = "file.txt";
        String fileName = "/tmp/file.txt";

        long startTime = System.currentTimeMillis();

        try {
            S3Client s3 = S3Client.builder()
                    .region(region)
                    .credentialsProvider(ProfileCredentialsProvider.create("profile-name"))
                    .build();

            // get meta data about the object without downloading it
            HeadObjectRequest headObjectRequest = HeadObjectRequest.builder()
                    .bucket(bucketName)
                    .key(fileObjKeyName)
                    .build();
            HeadObjectResponse headObjectResponse = s3.headObject(headObjectRequest);
            System.out.println("object is " + headObjectResponse.contentLength() + " bytes long from head call");

            // get the object,save it to a file
            GetObjectRequest getObjectRequest = GetObjectRequest.builder()
                    .bucket(bucketName)
                    .key(fileObjKeyName)
                    .build();

            OutputStream outputStream = new FileOutputStream(fileName);
            GetObjectResponse getObjectResponse = s3.getObject(getObjectRequest,ResponseTransformer.toOutputStream(outputStream));

            // meta data is also available from the GetObjectResponse
            System.out.println("object is " + getObjectResponse.contentLength() + " bytes long from get call");
        }
        catch( NoSuchKeyException nske ) { // bad key name
            System.out.println( "the key does not exist" );
        }
        catch( S3Exception s3e ) { // errors like bad credentials
            System.out.println( "a general s3 exception occurred: ");
            s3e.printStackTrace();
        }
        catch( SdkClientException sce ) { // other errors
            System.out.println( "a sdk exception occurred: ");
            sce.printStackTrace();
        }
        catch( Exception e ) {
            e.printStackTrace();
        }

        System.out.println( "run took " + (System.currentTimeMillis() - startTime) + "ms");
        System.exit(0);
    }
}