如何解决Terraform和AWS CLI不一致的本地执行人员
我试图用AWS cli修改DMS S3端点额外的连接属性,因为它在terraform资源中不起作用。
resource "aws_dms_endpoint" "s3_endpoint" {
count = length(var.zones)
endpoint_id = "${var.s3_endpoint_id_prefix}-${var.environment}-${var.shortApp}-${var.zones[count.index]}"
endpoint_type = "target"
engine_name = "s3"
ssl_mode = "none"
kms_key_arn = var.kms_key_arn
s3_settings {
bucket_folder = "${var.zones[count.index]}/dms-export/${var.shortApp}"
bucket_name = var.bucket_name
service_access_role_arn = var.service_access_role_arn
}
tags = merge(var.tags,{ "global.project" = "${local.global_project_tag}" })
provisioner "local-exec" {
command = "aws dms modify-endpoint --endpoint-arn ${self.endpoint_arn} --extra-connection-attributes ServiceAccessRoleArn=${var.service_access_role_arn};bucketFolder=${var.zones[count.index]}/dms-export/${var.shortApp};bucketName=${var.bucket_name};cannedAclForObjects=BUCKET_OWNER_FULL_CONTROL;cdcPath=undefined;compressionType=NONE;csvDelimiter=,;csvRowDelimiter=\n;dataFormat=csv;datePartitionEnabled=false;includeOpForFullLoad=true;timestampColumnName=CDCTIMESTAMP"
}
}
该代码不会出现任何错误,但是不会传递所有额外的连接属性。结果(摘自json响应)是这样的:
"ExtraConnectionAttributes": "ServiceAccessRoleArn=arn:aws:iam::<crossedout>:role/SDP_DMS_MIGRATION_TO_S3_ROLE;bucketFolder=test/dms-export/pc;bucketName=<crossedout>-nonprod-sdp-lz-stag-pnc;cannedAclForObjects=BUCKET_OWNER_FULL_CONTROL;cdcPath=undefined;compressionType=NONE;csvDelimiter=,;csvRowDelimiter=\\n;datePartitionEnabled=false;",
编辑:AWS CLI WORSK
aws dms modify-endpoint --endpoint-arn arn:aws:dms:eu-central-1:***********:endpoint:*********** --extra-connection-attributes 'ServiceAccessRoleArn=arn:aws:iam::***********:role/SDP_DMS_MIGRATION_TO_S3_ROLE;bucketFolder=komo/dms-export/pc;bucketName=***********-nonprod-sdp-lz-stag-pnc;cannedAclForObjects=BUCKET_OWNER_FULL_CONTROL;cdcPath=undefined;compressionType=NONE;csvDelimiter=,;csvRowDelimiter=\n;dataFormat=csv;datePartitionEnabled=false;includeOpForFullLoad=true;timestampColumnName=CDCTIMESTAMP'
解决方法
cli dms docs显示属性名称。它们的外壳与您使用的外壳不同。
文档显示
interface Type<T> extends Function {
new (...args: any[]): T;
}
type HookMap<T> = {
[func in keyof T]?: any[];
};
function Hooks<T>(hookMap: HookMap<T>) {
return (clazz: Type<T>): void => {
// ...
};
}
class Parent {
parentMethod() {
// ...
}
}
@Hooks({
childMethod: [/* ... */],parentMethod: [/* ... */]
})
class Child extends Parent {
childMethod() {
// ...
}
}
您有:
"ServiceAccessRoleArn": "string","ExternalTableDefinition": "string","CsvRowDelimiter": "string","CsvDelimiter": "string","BucketFolder": "string","BucketName": "string","CompressionType": "none"|"gzip","EncryptionMode": "sse-s3"|"sse-kms","ServerSideEncryptionKmsKeyId": "string","DataFormat": "csv"|"parquet","EncodingType": "plain"|"plain-dictionary"|"rle-dictionary","DictPageSizeLimit": integer,"RowGroupLength": integer,"DataPageSize": integer,"ParquetVersion": "parquet-1-0"|"parquet-2-0","EnableStatistics": true|false,"IncludeOpForFullLoad": true|false,"CdcInsertsOnly": true|false,"TimestampColumnName": "string","ParquetTimestampInMillisecond": true|false,"CdcInsertsAndUpdates": true|false,"DatePartitionEnabled": true|false,"DatePartitionSequence": "YYYYMMDD"|"YYYYMMDDHH"|"YYYYMM"|"MMYYYYDD"|"DDMMYYYY","DatePartitionDelimiter": "SLASH"|"UNDERSCORE"|"DASH"|"NONE"
更轻松的视图:
ServiceAccessRoleArn
bucketFolder
bucketName
cannedAclForObjects
cdcPath
compressionType
csvDelimiter
csvRowDelimiter
dataFormat
datePartitionEnabled
includeOpForFullLoad
timestampColumnName
我在文档中没有看到以下内容:
yours | docs
:-- | :--
bucketFolder | BucketFolder
bucketName | BucketName
compressionType | CompressionType
csvDelimiter | CsvDelimiter
csvRowDelimiter | CsvRowDelimiter
dataFormat | DataFormat
datePartitionEnabled | DatePartitionEnabled
includeOpForFullLoad | IncludeOpForFullLoad
serviceAccessRoleArn | ServiceAccessRoleArn
timestampColumnName | TimestampColumnName
将所有内容放在一起尝试:
CannedAclForObjects
CdcPath
provisioner "local-exec" {
command = format("aws dms modify-endpoint --endpoint-arn %s --extra-connection-attributes %s",self.endpoint_arn,join(";",[
"ServiceAccessRoleArn=${var.service_access_role_arn}","BucketFolder=${var.zones[count.index]}/dms-export/${var.shortApp}","BucketName=${var.bucket_name}","CannedAclForObjects=BUCKET_OWNER_FULL_CONTROL","CdcPath=undefined","CompressionType=NONE","CsvDelimiter=,","CsvRowDelimiter=\n","DataFormat=csv","DatePartitionEnabled=false","IncludeOpForFullLoad=true","TimestampColumnName=CDCTIMESTAMP",]))
}
上的join
也将使您的命令更易于阅读。它接受该列表,并将它们与;
分隔符组合。
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。