微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦!

将数据从 mySQL 导入 Clickhouse 失败

如何解决将数据从 mySQL 导入 Clickhouse 失败

我已经配置了 MysqL 和 clickhouse,以便我可以将 MysqL 数据导入到 clickhouse,当我运行以下命令时 clickhouse-MysqL-data reader 连接并在 clikhouse 服务器内创建一个表,但在插入查询时失败。以下是输出

2021-04-27 14:05:42,730/1619514342.730035:INFO:Starting
2021-04-27 14:05:42,730/1619514342.730357:DEBUG:{'app': {'binlog_position_file': None,'config_file': '/etc/clickhouse-MysqL/clickhouse-MysqL.conf','create_table_json_template': False,'create_table_sql': False,'create_table_sql_template': False,'csvpool': False,'daemon': False,'dry': False,'install': False,'log_file': None,'log_level': 0,'mempool': False,'mempool_max_events_num': 100000,'mempool_max_flush_interval': 60,'mempool_max_rows_num': 100000,'migrate_table': True,'pid_file': '/tmp/reader.pid','pump_data': False,'with_create_database': False},'converter': {'clickhouse': {'column_skip': [],'converter_class': None,'converter_file': None},'csv': {'column_default_value': None,'column_skip': []}},'reader': {'file': {'csv_file_path': None,'nice_pause': 1},'MysqL': {'blocking': True,'connection_settings': {'host': '222.133.111.62','password': 'pass','port': 3306,'user': 'chreader'},'log_pos': None,'nice_pause': 1,'resume_stream': False,'schemas': None,'server_id': 1,'tables': ['vtc.incoming_calls'],'tables_prefixes': None}},'table_builder': {'clickhouse': {'connection_settings': {'host': '127.0.0.1','password': '','port': 9000,'user': 'default'},'dst_cluster': None,'dst_create_table': True,'dst_distribute': False,'dst_schema': None,'dst_table': None,'dst_table_prefix': None},'MysqL': {'column_skip': [],'host': '222.133.111.62','tables_prefixes': None,'user': 'chreader'}},'table_migrator': {'clickhouse': {'connection_settings': {'host': '127.0.0.1','tables_where_clauses': None,'writer': {'clickhouse': {'connection_settings': {'host': '127.0.0.1','file': {'csv_file_path': None,'csv_file_path_prefix': '/tmp/csvpool_','csv_file_path_suffix_parts': [],'csv_keep_file': False,'dst_table_prefix': None}}}
2021-04-27 14:05:42,734/1619514342.734610:INFO:sys.path
2021-04-27 14:05:42,734/1619514342.734863:INFO:['/home/adminuser/clickhouse-MysqL-data-reader/clickhouse_MysqL/converter','/usr/local/bin','/usr/lib64/python36.zip','/usr/lib64/python3.6','/usr/lib64/python3.6/lib-dynload','/home/adminuser/.local/lib/python3.6/site-packages','/usr/local/lib64/python3.6/site-packages','/usr/local/lib/python3.6/site-packages','/home/haditeluser/clickhouse-MysqL-data-reader','/usr/lib64/python3.6/site-packages','/usr/lib/python3.6/site-packages']
2021-04-27 14:05:42,735/1619514342.735051:DEBUG:CHWriteConverter __init__()
2021-04-27 14:05:42,735/1619514342.735171:DEBUG:Converter __init__()
2021-04-27 14:05:42,735/1619514342.735296:DEBUG:[]
2021-04-27 14:05:42,735/1619514342.735435:INFO:CHWriter() connection_settings={'host': '127.0.0.1','user': 'default','password': ''} dst_schema=None dst_table=None dst_distribute=False
2021-04-27 14:05:42,735/1619514342.735559:INFO:CHClient() connection_settings={'host': '127.0.0.1','password': ''}
2021-04-27 14:05:42,735/1619514342.735764:INFO:CHClient() connection_settings={'host': '127.0.0.1',735/1619514342.735966:DEBUG:{'vtc': {'incoming_calls'}} group tables for explicitly specified db/tables
2021-04-27 14:05:42,736/1619514342.736097:DEBUG:{} group tables for prefix specified db/tables
2021-04-27 14:05:42,736/1619514342.736213:INFO:List for migration:
2021-04-27 14:05:42,736/1619514342.736338:INFO:  vtc.incoming_calls
2021-04-27 14:05:42,736/1619514342.736473:DEBUG:{'vtc': {'incoming_calls'}} group tables for explicitly specified db/tables
2021-04-27 14:05:42,736/1619514342.736598:DEBUG:{} group tables for prefix specified db/tables
2021-04-27 14:05:42,764/1619514342.764922:DEBUG:Connect to the database host=222.133.111.62 port=3306 user=chreader password=pass db=vtc
2021-04-27 14:05:42,768/1619514342.768492:INFO:Start migration vtc.incoming_calls
Running with chclient CREATE DATABASE IF NOT EXISTS `vtc`;
2021-04-27 14:05:42,768/1619514342.768675:DEBUG:Connecting. Database: default. User: default
2021-04-27 14:05:42,768/1619514342.768799:DEBUG:Connecting to 127.0.0.1:9000
2021-04-27 14:05:42,773/1619514342.773789:DEBUG:Connected to ClickHouse server version 21.3.4,revision: 54447
2021-04-27 14:05:42,896/1619514342.896687:DEBUG:Query: CREATE DATABASE IF NOT EXISTS `vtc`
2021-04-27 14:05:42,897/1619514342.897576:DEBUG:Block "" send time: 0.000080
Running with chclient CREATE TABLE IF NOT EXISTS `vtc`.`incoming_calls`  (
    `id` Int32,`client_id` String,`date` Date,`time` Nullable(String),`cids` Nullable(String),`datetime` DateTime,`did_num` Nullable(String),`line_status` Nullable(Int32),`duration` Nullable(Int32),`billing_minutes` Nullable(Float32),`Last_App` Nullable(String),`Last_App_ID` Nullable(Int32),`Last_App_Data` String,`call-status` Nullable(String),`dtmf` Int32,`Caller_ID` Nullable(Int32),`Recd_File_Path` String,`agent_name` Nullable(String),`agent_no` Nullable(String),`Lead_Check` Nullable(Int32)
)
ENGINE = ReplacingMergeTree() PARTITION BY toYYYYMM(date) ORDER BY (id,client_id,date,datetime,Last_App_Data,Recd_File_Path)
;
2021-04-27 14:05:42,899/1619514342.899724:DEBUG:Query: CREATE TABLE IF NOT EXISTS `vtc`.`incoming_calls`  (
    `id` Int32,Recd_File_Path)

2021-04-27 14:05:42,900/1619514342.900012:DEBUG:Block "" send time: 0.000071
2021-04-27 14:05:42,932/1619514342.932191:DEBUG:CHWriteConverter __init__()
2021-04-27 14:05:42,932/1619514342.932370:DEBUG:Converter __init__()
2021-04-27 14:05:42,932/1619514342.932487:DEBUG:[]
2021-04-27 14:05:42,932/1619514342.932637:INFO:CHWriter() connection_settings={'host': '127.0.0.1',932/1619514342.932763:INFO:CHClient() connection_settings={'host': '127.0.0.1',932/1619514342.932946:INFO:CHClient() connection_settings={'host': '127.0.0.1',934/1619514342.934955:DEBUG:{'vpbx': {'incoming_calls'}} group tables for explicitly specified db/tables
2021-04-27 14:05:42,935/1619514342.935480:DEBUG:{} group tables for prefix specified db/tables
2021-04-27 14:05:42,935/1619514342.935601:INFO:List for migration:
2021-04-27 14:05:42,935/1619514342.935717:INFO:  vtc.incoming_calls
2021-04-27 14:05:42,935/1619514342.935832:INFO:Start migration vtc.incoming_calls
2021-04-27 14:05:42,959/1619514342.959363:DEBUG:Connect to the database host=222.133.111.62 port=3306 user=chreader password=pass db=vtc
2021-04-27 14:05:42,962/1619514342.962478:INFO:migrate_table. sql=SELECT `id`,`client_id`,`date`,`time`,`cids`,`datetime`,`did_num`,`line_status`,`duration`,`billing_minutes`,`Last_App`,`Last_App_ID`,`Last_App_Data`,`call-status`,`dtmf`,`Caller_ID`,`Recd_File_Path`,`agent_name`,`agent_no`,`Lead_Check` FROM `vpbx`.`incoming_calls`
2021-04-27 14:05:42,986/1619514342.986669:DEBUG:Connect to the database host=222.133.111.62 port=3306 user=chreader password=pass db=vtc
2021-04-27 14:05:44,527/1619514344.527455:DEBUG:class:<class 'clickhouse_MysqL.writer.chwriter.CHWriter'> insert 1 event(s)
2021-04-27 14:05:44,891/1619514344.891846:DEBUG:class:<class 'clickhouse_MysqL.writer.chwriter.CHWriter'> insert 100000 row(s)
2021-04-27 14:05:44,892/1619514344.892111:DEBUG:schema=vtc table=incoming_calls self.dst_schema=None self.dst_table=None
2021-04-27 14:05:44,892/1619514344.892497:DEBUG:Connecting. Database: default. User: default
2021-04-27 14:05:44,892/1619514344.892591:DEBUG:Connecting to 127.0.0.1:9000
2021-04-27 14:05:44,894/1619514344.894158:DEBUG:Connected to ClickHouse server version 21.3.4,revision: 54447
2021-04-27 14:05:44,894/1619514344.894521:DEBUG:Query: INSERT INTO `vtc`.`incoming_calls` (`id`,`Lead_Check`) VALUES
2021-04-27 14:05:44,894/1619514344.894732:DEBUG:Block "" send time: 0.000056
2021-04-27 14:05:45,200/1619514345.200698:CRITICAL:QUERY Failed
2021-04-27 14:05:45,200/1619514345.200894:CRITICAL:ex='nonetype' object has no attribute 'encode'
2021-04-27 14:05:45,200/1619514345.200971:CRITICAL:sql=INSERT INTO `vtc`.`incoming_calls` (`id`,`Lead_Check`) VALUES

最后 4 行说明出了什么问题,但我不清楚如何解决错误。请帮忙。

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。

相关推荐


Selenium Web驱动程序和Java。元素在(x,y)点处不可单击。其他元素将获得点击?
Python-如何使用点“。” 访问字典成员?
Java 字符串是不可变的。到底是什么意思?
Java中的“ final”关键字如何工作?(我仍然可以修改对象。)
“loop:”在Java代码中。这是什么,为什么要编译?
java.lang.ClassNotFoundException:sun.jdbc.odbc.JdbcOdbcDriver发生异常。为什么?
这是用Java进行XML解析的最佳库。
Java的PriorityQueue的内置迭代器不会以任何特定顺序遍历数据结构。为什么?
如何在Java中聆听按键时移动图像。
Java“Program to an interface”。这是什么意思?