如何解决无法使AWS Athena escapeChar工作
我想尝试一下AWS Athena,但遇到了我要测试的csv文件问题。
使用以下命令scapeChar似乎不起作用。 我尝试使用搜寻器并在UI中指定escapeChar,使用和不使用双引号,但仍然没有运气。当一行中包含定界符时,即使将其转义也将被视为字段定界符。
DDL
CREATE EXTERNAL TABLE mytestcsvtable (
col_id string,col_description string,col_text string,col_decimal string,col_float string,col_date string,col_time string,col_timestamp string
)
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.OpenCSVSerde'
WITH SERDEPROPERTIES (
'separatorChar' = '|','escapeChar' = '\\'
)
STORED AS TEXTFILE
LOCATION 's3://mystaging2/tmp/';
数据
401|SingleBackslash|Goblin \\Gobby Gruesome|.00|0E0|2020-03-02|21.21.43|2020-03-02 12.35.52.145894
402|EvennumberOfSingleBackslash|Goblin \\Gobby\\ Gruesome|.00|0E0|2020-03-02|22.22.44|2020-03-02 12.35.52.156563
403|SingleSingleBackslash|Goblin \\\\Gobby\\\\ Gruesome|.00|0E0|2020-03-02|23.23.45|2020-03-02 12.35.52.158011
404|OddNumberOfSingleBackslash1|Goblin \\Gruesome\\ \\Gruesome|.00|0E0|2020-03-02|00.24.46|2020-03-02 12.35.52.159835
405|OddNumberOfSingleBackslash2|Goblin \\\\Gobby\\ Gruesome|.00|0E0|2020-03-02|01.25.47|2020-03-02 12.35.52.162538
406|OddNumberOfSingleSingleBackslash1|Goblin \\\\Gruesome\\\\ \\\\Gruesome|.00|0E0|2020-03-02|02.26.48|2020-03-02 12.35.52.163510
407|OddNumberOfSingleSingleBackslash2|Goblin \\\\Gobby\\\\\\ \\\\Gruesome|.00|0E0|2020-03-02|03.27.49|2020-03-02 12.35.53.167322
408|SingleSingleSingleBackslash1|Goblin \\\\\\Gobby|.00|0E0|2020-03-02|04.28.50|2020-03-02 12.35.53.179868
501|SinglePipe|Goblin \|Gobby Gruesome|.00|0E0|2020-03-02|05.29.51|2020-03-02 12.35.53.180025
502|EvennumberOfSinglePipe|Goblin \|Gobby\| Gruesome|.00|0E0|2020-03-02|06.30.52|2020-03-02 12.35.53.184042
503|SingleSinglePipe|Goblin \|\|Gobby\|\| Gruesome|.00|0E0|2020-03-02|07.31.53|2020-03-02 12.35.53.189979
504|OddNumberOfSinglePipe1|Goblin \|Gruesome\| \|Gruesome|.00|0E0|2020-03-02|08.32.54|2020-03-02 12.35.53.194734
505|OddNumberOfSinglePipe2|Goblin \|\|Gobby\| Gruesome|.00|0E0|2020-03-02|09.33.55|2020-03-02 12.35.53.196996
506|OddNumberOfSingleSinglePipe1|Goblin \|\|Gruesome\|\| \|\|Gruesome|.00|0E0|2020-03-02|10.34.56|2020-03-02 12.35.53.203568
507|OddNumberOfSingleSinglePipe2|Goblin \|\|Gobby\|\|\| \|\|Gruesome|.00|0E0|2020-03-02|11.35.57|2020-03-02 12.35.53.203999
508|SingleSingleSinglePipe1|Goblin \|\|\|Gobby|.00|0E0|2020-03-02|12.36.58|2020-03-02 12.35.54.208965
感谢您抽出宝贵的时间阅读我的帖子!
解决方法
在csv中,escapeChar
用于在带引号的字段中转义可能的quoteChar
,而不是转义定界符。
要“转义”字段中的分隔符,必须在字段中加上引号-例如:
501|SinglePipe|Goblin \|Gobby Gruesome|.00|0E0|2020-03-02
成为:
501|SinglePipe|"Goblin \|Gobby Gruesome"|.00|0E0|2020-03-02
or
501|SinglePipe|"Goblin |Gobby Gruesome"|.00|0E0|2020-03-02
如果字段中有引号,则必须将其转义:
501|SinglePipe|"Goblin|24\" monitor|Gobby Gruesome"|.00|0E0|2020-03-02
如果这不是默认设置,请确保告诉解析器使用quoteChar
。
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。