Kafka的单机安装部署
1.原生安装
下载kafka
https://archive.apache.org/dist/kafka/2.0.0/kafka_2.11-2.0.0.tgz
安装过程
安装过程非常简单,只需要解压就行,因为这个是编译好之后的可执行程序
tar -zxvf kafka_2.11-2.0.0.tgz
解压
配置zookeeper
因为kafka依赖于zookeeper来做master选举一起其他数据的维护,所以需要先启动zookeeper节点
kafka内置了zookeeper的服务,所以在bin目录下提供了这些脚本
zookeeper-server-start.sh
zookeeper-server-stop.sh
zookeeper.properties
server.properties
所以我们可以通过下面的脚本来启动zk服务,当然,也可以自己搭建zk的集群来实现
sh zookeeper-server-start.sh -daemon ../config/zookeeper.properties
启动和停止kafka
修改server.properties, 增加zookeeper的配置
zookeeper.connect=localhost:2181
## 集群或者夸宿主机配置注意点
broker.id=0
## 注册到zk是告诉zk kafka的应用地址
listeners=PLAINTEXT://<当前宿主机IP>:9092
启动kafka
sh kafka-server-start.sh -damoen config/server.properties
停止kafka
sh kafka-server-stop.sh -daemon config/server.properties
2.Docker安装
docker run --name zookeeper -p 2181:2181 -d zookeeper:3.5.8
docker run -d --name kafka -p 9092:9092 -v /data/kafka:/kafka \
-v /var/run/docker.sock:/var/run/docker.sock \
-e KAFKA_broKER_ID=0 \
-e KAFKA_ZOOKEEPER_CONNECT=<内网ip,不能是容器内ip>:2181 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://<公网ip>:9092 \
-e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 \
-e KAFKA_LOG_Dirs=/kafka/kafka-logs \
wurstmeister/kafka
3.window 启动单机kafka
docker run -d --name kafka -p 9092:9092 -e KAFKA_broKER_ID=0 -e KAFKA_ZOOKEEPER_CONNECT=172.18.30.181:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092 -e KAFKA_LISTENERS=PLAINTEXT://127.0.0.1:9092 -e KAFKA_LOG_Dirs=/kafka/kafka-logs wurstmeister/kafka
Kafka的集群部署
1.原生部署
配置server.properties文件
分别修改三台机器的server.properties配置,同一个集群中的每个机器的id必须唯一
broker.id=0
broker.id=1
broker.id=2
修改zookeeper的连接配置
zookeeper.connect=192.168.13.106:2181 # 内网ip,不能是容器内ip
修改listeners配置
如果配置了listeners,那么消息生产者和消费者会使用listeners的配置来进行消息的收发,否则,会使用localhost
PLAINTEXT表示协议,默认是明文,可以选择其他加密协议
listeners=PLAINTEXT://192.168.13.102:9092
分别启动三台服务器
sh kafka-server-start.sh -daemon ../config/server.properties
2.docker部署kafka集群
docker run --name zookeeper -p 2181:2181 -d wurstmeister/zookeeper:latest
docker run --name zookeeper -p 2181:2181 -d zookeeper:3.5.8
# -e KAFKA_ADVERTISED_LISTENERS 作用就是暴露给外部的listeners,如果没有设置,会用listeners
# 搭建集群
docker run -d --name kafka1 -p 9092:9092 -v /data/kafka/broker1:/kafka \
-v /var/run/docker.sock:/var/run/docker.sock \
-e KAFKA_broKER_ID=0 \
-e KAFKA_ZOOKEEPER_CONNECT=172.19.71.214:2181 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://101.132.117.5:9092 \
-e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 \
-e KAFKA_LOG_Dirs=/kafka/kafka-logs \
wurstmeister/kafka
docker run -d --name kafka2 -p 9095:9092 -v /data/kafka/broker2:/kafka \
-v /var/run/docker.sock:/var/run/docker.sock \
-e KAFKA_broKER_ID=1 \
-e KAFKA_ZOOKEEPER_CONNECT=172.19.71.214:2181 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://101.132.117.5:9095 \
-e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 \
-e KAFKA_LOG_Dirs=/kafka/kafka-logs \
wurstmeister/kafka
docker run -d --name kafka3 -p 9096:9092 -v /data/kafka/broker3:/kafka \
-v /var/run/docker.sock:/var/run/docker.sock \
-e KAFKA_broKER_ID=2 \
-e KAFKA_ZOOKEEPER_CONNECT=172.19.71.214:2181 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://101.132.117.5:9096 \
-e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 \
-e KAFKA_LOG_Dirs=/kafka/kafka-logs \
wurstmeister/kafka
3.docker-compose部署kafka集群
kafka-docker-compose.yml
version: '2'
services:
zookeeper:
container_name: "zookeeper"
image: wurstmeister/zookeeper
ports:
- "2181:2181"
restart: always
kafka1:
container_name: "kafka1"
image: wurstmeister/kafka:latest
ports:
- "9092:9092"
environment:
KAFKA_broKER_ID: 0
KAFKA_ZOOKEEPER_CONNECT: 172.19.71.214:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://101.132.117.5:9092
KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
KAFKA_HEAP_OPTS: -Xmx1G -xms1G
KAFKA_LOG_Dirs: /kafka/kafka-logs
volumes:
- /data/kafka/broker1:/kafka
- /var/run/docker.sock:/var/run/docker.sock
kafka2:
container_name: "kafka2"
image: wurstmeister/kafka:latest
ports:
- "9095:9092"
environment:
KAFKA_broKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 172.19.71.214:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://101.132.117.5:9095
KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
KAFKA_HEAP_OPTS: -Xmx1G -xms1G
KAFKA_LOG_Dirs: /kafka/kafka-logs
volumes:
- /data/kafka/broker2:/kafka
- /var/run/docker.sock:/var/run/docker.sock
kafka3:
container_name: "kafka3"
image: wurstmeister/kafka:latest
ports:
- "9096:9092"
environment:
KAFKA_broKER_ID: 2
KAFKA_ZOOKEEPER_CONNECT: 172.19.71.214:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://101.132.117.5:9096
KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
KAFKA_HEAP_OPTS: -Xmx1G -xms1G
KAFKA_LOG_Dirs: /kafka/kafka-logs
volumes:
- /data/kafka/broker3:/kafka
- /var/run/docker.sock:/var/run/docker.sock
在kafka-docker-compose.yml文件目录下运行:
docker-compose -f kafka-docker-compose.yml up -d
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。