如何解决kylin - 扫描 10 439 900 行后协处理器超时
在 kylin Insight 中执行查询时扫描 10 439 900 行后获取协处理器超时。
我目前每年处理的数据约为 1.5 亿。
版本
- kylin-3.0.1
- hadoop-2.7.0
- hbase-1.1.2
在 kylin.properties
中的道具下方添加
kylin.query.scan.threshold=5000000000
kylin.storage.hbase.coprocessor-mem-gb=20
kylin.storage.partition.max-scan-bytes=0
异常跟踪
在执行 SQL 时扫描 10439900 行后协处理器超时:"select sum(gwp) gwp,sum(nwp) nwp,sum(gwp_od) gwp_od,SUM(gwp_tp) gwp_tp,sum(nwp_od) nwp_od,_sumtn(,sum(discount_gwp_od) discount_gwp_od,sum(discount_nwp_od) discount_nwp_od from ( select sum(INS_COVERAGE_PREMIUM) gwp,sum(ins_nwp) nwp,sum(case when INS_COVERAGE_CATEGORY PR = 'OD' then end_case INS_COVERAGE_CATEGORY PR = 'OD' 时,INS_OVER INS_OVER ('TP','PA') then INS_COVERAGE_PREMIUM else 0 end) gwp_tp,sum(case when INS_COVERAGE_CATEGORY = 'OD' then INS_NWP else 0 end) nwp_od,sum(case when INS_COVERAGE_CATEGORY in ('TP','PA') then INS_NWP else 0 end) nwp_tp,'PA') and INS_COVERAGE_TYPE = 'DC' then INS_COVERAGE_PREMIUM else 0 end) discount_gwp_od,'PA') and INS_COVERAGE_TYPE = 'DC' ) and INS_COVERAGE_TYPE = 'DC' then INS_NWP else 0 end) discount_nwp_od from RSA_KPI_FACT_INS_NEW ins LEFT JOIN KPI_FIN_YEAR_MASTER as KPI_FIN_ YEAR_MASTER ON ins.FINANCIAL_YEAR = KPI_FIN_YEAR_MASTER.FIN_YEAR LEFT JOIN KPI_FIN_MONTH_MASTER作为KPI_FIN_MONTH_MASTER ON ins.EFF_FIN_YEAR_MONTH = KPI_FIN_MONTH_MASTER.FIN_MONTH LEFT JOIN KPI_PRODUCT_MASTER作为KPI_PRODUCT_MASTER ON ins.PRODUCT_CODE = KPI_PRODUCT_MASTER.PRODUCT_CODE LEFT JOIN KPI_BRANCH_MASTER作为KPI_BRANCH_MASTER ON ins.BRANCH_CODE = KPI_BRANCH_MASTER.BRANCH_CODE LEFT JOIN KPI_CAMPAIGN_MASTER作为KPI_CAMPAIGN_MASTER ON ins.CAMPAIN_CODE = KPI_CAMPAIGN_MASTER.CAMPAIGN_CODE LEFT JOIN KPI_OA_MASTER_NW作为KPI_OA_MASTER_NW ON ins.OA_CODE = KPI_OA_MASTER_NW.OA_CODE LEFT JOIN KPI_MODEL_MASTER_NW作为KPI_MODEL_MASTER_NW ON ins.MAKE = KPI_MODEL_MASTER_NW.MAKE AND ins.MODELCODE = KPI_MODEL_MASTER_NW.MODEL_CODE LEFT JOIN KPI_SUB_CHANNEL_MASTER_NW作为KPI_SUB_CHANNEL_MASTER_NW ON ins.CHANNEL = KPI_SUB_CHANNEL_MASTER_NW.CHANNEL_NAME AND ins.SUB_CHANNEL = KPI_SUB_CHANNEL_MASTER_NW.SUB_CHANNEL LEFT JOIN KPI_BUSINESS_TYPE_MASTER 作为 KPI_BUSINESS_TYPE_MASTER ON ins = KPI_BUSINESS_TYPE_MASTER ON ins. NESS_TYPE_MASTER.BUSINESS_TYPE LEFT JOIN KPI_POLICY_CATEGORY_MASTER_NW作为KPI_POLICY_CATEGORY_MASTER_NW ON ins.POLICY_CATEGORY = KPI_POLICY_CATEGORY_MASTER_NW.POLICY_CATEGORY WHERE(ins.FIN_DATE> = '2018年4月1日' AND ins.FIN_DATE
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。