Spectrum S3 访问被拒绝

如何解决Spectrum S3 访问被拒绝

NI 正在尝试通过 Redshift Spectrum 将镶木地板数据加载到 Redshift。

我已经建立了信任关系等，可以很好地承担 Redshift 的角色。

但是我收到了一个 S3 访问被拒绝错误，我似乎无法解决。

S3 存储桶策略：

{
    "Version": "2012-10-17","Statement": [
        {
            "Effect": "Allow","Principal": "*","Action": "s3:*","Resource": [
                "arn:aws:s3:::<BUCKET>","arn:aws:s3:::<BUCKET>/*"
            ],"Condition": {
                "ArnEquals": {
                    "aws:PrincipalArn": [
                        "<ADMIN ROLE 1 ARN>","<ADMIN ROLE 2 ARN>"
                    ]
                }
            }
        },{
            "Effect": "Allow","Action": [
                "s3:GetBucketNotification","s3:GetBucketVersioning","s3:DeleteObject","s3:PutObject","s3:ListBucket","s3:Getobject","s3:ListBucketVersions"
            ],"Condition": {
                "ArnEquals": {
                    "aws:PrincipalArn": [
                        "arn:aws:iam::123456781234:role/glueRole","arn:aws:iam::123456781234:role/ExtractsqlRole","arn:aws:iam::123456781234:role/RedshiftRole"
                    ]
                }
            }
        },{
            "Effect": "Deny","Resource": [
                "arn:aws:s3:::<BUCKET>/*","arn:aws:s3:::<BUCKET>"
            ],"Condition": {
                "ArnNotEquals": {
                    "aws:PrincipalArn": [
                        "<ADMIN ROLE 1 ARN>","<ADMIN ROLE 2 ARN>","arn:aws:iam::123456781234:role/glueRole","arn:aws:iam::123456781234:role/RedshiftRole"
                    ]
                }
            }
        }
    ]
}

频谱架构创建于：

create external schema 'Schema1'
from data catalog
database 'spectrum_database'
iam_role 'arn:aws:iam::123456781234:role/RedshiftRole'
catalog_role 'arn:aws:iam::123456781234:role/glueRole'

胶水作用：

  glueRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: glue.amazonaws.com
            Action: sts:AssumeRole
          - Effect: Allow
            Principal:
              Service: redshift.amazonaws.com
            Action: sts:AssumeRole
            Condition:
              StringEquals:
                sts:ExternalId:
                  - arn:aws:iam::123456781234:role/glueRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSglueServiceRole

有了这个，我得到了一个表格列表，但总是出现这个错误：

我只需要将存储桶保护到某些角色，但还需要 Spectrum 来查询它...有什么提示吗？

解决方法

您对所有原则都明确拒绝：

{
            "Effect": "Deny","Principal": "*","Action": "s3:*","Resource": [
                "arn:aws:s3:::<BUCKET>/*","arn:aws:s3:::<BUCKET>"
            ],"Condition": {
                "ArnEquals": {
                    "aws:PrincipalArn": [
                        "<ADMIN ROLE 1 ARN>","<ADMIN ROLE 2 ARN>","arn:aws:iam::123456781234:role/GlueRole","arn:aws:iam::123456781234:role/ExtractSQLRole","arn:aws:iam::123456781234:role/RedshiftRole"
                    ]
                }
            }
        }

拒绝总是赢，所以你总是被拒绝，没有allow会改变它。我不确定你想用这个明确的拒绝来实现什么。也许您想使用 ArnNotEquals？

,

我正在尝试通过 Redshift Spectrum 将数据加载到 Redshift。

简单说一下 - 上次我看（可能是两年前）Spectrum 有很多问题，一些基本问题，CSV 文件，以至于我认为 Spectrum with CSV 不适合使用在生产中。如果您使用的是 CSV 苍蝇，我强烈建议您使用 COPY，而不是 Spectrum。