通过sqlalchemy创建ReplicatedMergeTree表

如何解决通过sqlalchemy创建ReplicatedMergeTree表

我正在尝试通过sqlalchemy在Clickhouse群集上创建一个新表。而且我不明白如何为ReplicatedMergeTree表的引擎指定所有必需的参数。

我使用了https://github.com/xzkostyan/clickhouse-sqlalchemy库

from sqlalchemy import create_engine,Column,MetaData,literal
from clickhouse_sqlalchemy import Table,make_session,get_declarative_base,types,engines

engine = create_engine(URL)
session = make_session(engine)
Metadata = MetaData(bind=engine)

# Задаются параметры таблицы ClickHouse
another_table = Table(
      'sqlalchemy_test',Metadata,Column('id',types.Int32,primary_key=True),Column('value',types.Int32),engines.ReplicatedMergeTree(
          table_path='/clickhouse/schema_name/tables/test_sqlalchemy',replica_name='{replica}-{shard}')
)

another_table.create()

如何将所有必需的参数放在Table类中？

DatabaseException: Orig exception: Code: 42,e.displayText() = DB::Exception: Storage ReplicatedMergeTree requires 5 to 6 parameters: 
path in ZooKeeper,replica name,name of column with date,[sampling element of primary key],primary key expression,index granularity

MergeTree is a family of storage engines.

MergeTrees are different in two ways:
- they may be replicated and non-replicated;
- they may do different actions on merge: nothing; sign collapse; sum; apply aggregete functions.

So we have 14 combinations:
    MergeTree,CollapsingMergeTree,SummingMergeTree,AggregatingMergeTree,ReplacingMergeTree,GraphiteMergeTree,VersionedCollapsingMergeTree
    ReplicatedMergeTree,ReplicatedCollapsingMergeTree,ReplicatedSummingMergeTree,ReplicatedAggregatingMergeTree,ReplicatedReplacingMergeTree,ReplicatedGraphiteMergeTree,ReplicatedVersionedCollapsingMergeTree


Examples:
   
ReplicatedMergeTree('/clickhouse/tables/{layer}-{shard}/hits','{replica}',EventDate,intHash32(UserID),(CounterID,EventTime),8192)

解决方法

要运行此代码，需要配置的ClickHouse集群（有关详细信息，请参见CH replication）。

CH根据群集配置替换所有占位符。

from sqlalchemy import create_engine,Column,MetaData,literal
from clickhouse_sqlalchemy import Table,make_session,get_declarative_base,types,engines

URL = 'clickhouse://default:@localhost/test'
CLUSTER_NAME='test_cluster'

engine = create_engine(URL)
session = make_session(engine)
metadata = MetaData(bind=engine)

another_table = Table(
    'sqlalchemy_test',metadata,Column('id',types.Int32),Column('value',engines.ReplicatedMergeTree(
        table_path='/clickhouse/clusters/{cluster}/tables/{database}/{shard}/{table}',replica_name='{replica}',order_by='id',partition_by=None,sample_by=None),clickhouse_cluster=CLUSTER_NAME
)

another_table.create()

---

出于测试目的，您可以使用docker-compose来建立测试CH集群（例如，查看neverlee/clickhouse-cluster-docker-compose）。