使用 Debezium 捕获数据更改但仅获取更改记录

如何解决使用 Debezium 捕获数据更改但仅获取更改记录

我有两个数据库（db1 和 db2）。这两个人每人有4张桌子。两个数据库中4张表的数据相同。现在我想以这样一种方式运行 CDC，它只会在 db2 中获取更改数据和更新。我不想把db1中的所有数据都拉到kafka主题中；因为它是第一次将所有数据拉入 kafka 主题。 Debezium 源连接器的配置是什么？

curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '
    {
        "name": "MysqL5-source","config": {
            "connector.class": "io.debezium.connector.MysqL.MysqLConnector","tasks.max": "10","database.hostname": "host","database.port": "3307","database.user": "root","database.password": "secret","database.server.id": "11","database.server.name": "dbserver","database.whitelist": "dbname","table.whitelist": "dbname.exm1,dbname.exm4,dbname.exm2,dbname.exm3","database.history.kafka.bootstrap.servers": "kafka:29092","database.history.kafka.topic": "MysqL5table","key.converter": "io.confluent.connect.avro.AvroConverter","value.converter": "io.confluent.connect.avro.AvroConverter","key.converter.schema.registry.url": "http://schema-registry:8081","value.converter.schema.registry.url": "http://schema-registry:8081","key.converter.schemas.enable":true,"value.converter.schemas.enable":true,"transforms": "unwrap,dropTopicPrefix,pushed_on,first_transfer_date,mem_dob,pushed_date,AL_Date,A_Last_login,live_time,A_Date,callMeDate","transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState","transforms.dropTopicPrefix.type":"org.apache.kafka.connect.transforms.RegexRouter","transforms.dropTopicPrefix.regex":"dbserver.indianmo_imc_new.(.*)","transforms.dropTopicPrefix.replacement":"$1"
  }
}'

我现在正在使用这个连接器。但它是第一次提取所有数据。我只需要新记录。提前致谢！

解决方法

我认为你正在寻找的是

snapshot.mode=schema_only

https://debezium.io/documentation/reference/1.4/connectors/mysql.html#mysql-connector-properties