使用外部 Zookeeper 对 NiFi 进行集群时的问题

如何解决使用外部 Zookeeper 对 NiFi 进行集群时的问题

我正在尝试将一些 Nifi 节点 (1.13.2) 连接到外部 Zookeeper 以便能够对服务进行集群化，但是当 NiFi 尝试使用 zookeeper 进行选举时，我遇到了一些错误。我在 nifi-app.log 上发现的错误是：

2021-07-12 12:32:24,673 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorleaderElectionManager org.apache.nifi.controller.leader.election.CuratorleaderElectionManager$ElectionListener@448ca30f Connection State changed to RECONNECTED
2021-07-12 12:32:24,673 WARN [main] o.a.nifi.controller.StandardFlowService There is currently no Cluster Coordinator. This often happens upon restart of NiFi when running an embedded ZooKeeper. Will register this node to become the active Cluster Coordinator and will attempt to connect to cluster again
2021-07-12 12:32:24,673 INFO [main] o.a.n.c.l.e.CuratorleaderElectionManager CuratorleaderElectionManager[stopped=false] Attempted to register leader Election for role 'Cluster Coordinator' but this role is already registered
2021-07-12 12:32:24,774 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: SUSPENDED
2021-07-12 12:32:24,774 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorleaderElectionManager org.apache.nifi.controller.leader.election.CuratorleaderElectionManager$ElectionListener@448ca30f Connection State changed to SUSPENDED
2021-07-12 12:32:24,893 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: RECONNECTED
2021-07-12 12:32:24,893 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorleaderElectionManager org.apache.nifi.controller.leader.election.CuratorleaderElectionManager$ElectionListener@448ca30f Connection State changed to RECONNECTED
2021-07-12 12:32:24,894 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: SUSPENDED
2021-07-12 12:32:24,894 INFO [Curator-ConnectionStateManager-0] o.a.n.c.l.e.CuratorleaderElectionManager org.apache.nifi.controller.leader.election.CuratorleaderElectionManager$ElectionListener@448ca30f Connection State changed to SUSPENDED
2021-07-12 12:32:24,894 ERROR [main-EventThread] o.a.c.f.imps.CuratorFrameworkImpl Background operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
        at org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:862)
        at org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:647)
        at org.apache.curator.framework.imps.WatcherRemovalFacade.processBackgroundOperation(WatcherRemovalFacade.java:152)
        at org.apache.curator.framework.imps.FindAndDeleteProtectednodeInBackground$2.processResult(FindAndDeleteProtectednodeInBackground.java:104)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:630)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510)

我的 nifi.properties 配置是：

nifi.state.management.configuration.file=./conf/state-management.xml
# The ID of the local state provider
nifi.state.management.provider.local=local-provider
# The ID of the cluster-wide state provider. This will be ignored if NiFi is not clustered but must be populated if running in a cluster.
nifi.state.management.provider.cluster=zk-provider
# Specifies whether or not this instance of NiFi should run an embedded ZooKeeper server
nifi.state.management.embedded.zookeeper.start=false
# Properties file that provides the ZooKeeper properties to use if <nifi.state.management.embedded.zookeeper.start> is set to true
nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties

nifi.cluster.is.node=true
nifi.cluster.node.address=nifi-node01
nifi.cluster.node.protocol.port=9999
nifi.cluster.node.protocol.threads=20
nifi.cluster.node.protocol.max.threads=50
nifi.cluster.node.event.history.size=25
nifi.cluster.node.connection.timeout=10 sec
nifi.cluster.node.read.timeout=10 sec
nifi.cluster.node.max.concurrent.requests=100
nifi.cluster.firewall.file=
nifi.cluster.flow.election.max.wait.time=1 mins
nifi.cluster.flow.election.max.candidates=

nifi.zookeeper.connect.string=zookeeper-node0.fqdn:2181,zookeeper-node1.fqdn:2181,zookeeper-node2.fqdn:2181
nifi.zookeeper.connect.timeout=10 secs
nifi.zookeeper.session.timeout=10 secs
nifi.zookeeper.root.node=/nifi-prod
nifi.zookeeper.client.secure=false
nifi.zookeeper.security.keystore=
nifi.zookeeper.security.keystoreType=
nifi.zookeeper.security.keystorePasswd=
nifi.zookeeper.security.truststore=
nifi.zookeeper.security.truststoreType=
nifi.zookeeper.security.truststorePasswd=

nifi.zookeeper.auth.type=default
nifi.zookeeper.kerberos.removeHostFromPrincipal=
nifi.zookeeper.kerberos.removeRealmFromPrincipal=

zk-provider 的状态管理配置是：

<cluster-provider>
            <id>zk-provider</id>
            <class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class>
            <property name="Connect String"> zookeeper-node0.fqdn:2181,zookeeper-node2.fqdn:2181</property>
            <property name="Root Node">/nifi-prod</property>
            <property name="Session Timeout">10 seconds</property>
            <property name="Access Control">Open</property>
        </cluster-provider>

CM (CDH-6.3.3) 上的 zookeeper 安全配置是：

nifi-prod 上的 znode 安全性是：

[zk: zookeeper-node0:2181(CONNECTED) 1] getAcl /nifi-prod
'world,'anyone
: cdrwa

** 我在 nifi 节点上使用以下命令将此 acl 连接到 zookeeper：/opt/cloudera/parcels/CDH-6.3.3-1.cdh6.3.3.p0.1796617/lib/zookeeper/bin/zkCli.sh -server zookeeper-node0:2181

这个错误会不会因为我没有放置安全配置而发生？即使没有从 nifi 节点工作的 zkCli 命令？ zookeeper 日志不打印任何错误