kubernetess 多个部署使用一个代码库但不同的配置环境变量 使用 Kustomization 的示例部署清单为 app1 和 app2 添加配置

分类：编程问答

如何解决kubernetess 多个部署使用一个代码库但不同的配置环境变量 使用 Kustomization 的示例部署清单为 app1 和 app2 添加配置

我有一个项目，我们从 kafka 消费数据并发布到 mongo。实际上代码库只做一项任务，可能是 mongo 到 kafka 的迁移，kafka 到 mongo 的迁移或其他。

我们必须从不同的 kafka 主题中消费并发布到不同的 mongo 集合。现在这些是并行的工作流。

当前的设计是拥有一个代码库，可以从任何主题使用并发布到任何 mongo 集合，该集合可使用环境变量进行配置。所以我们创建了一个 kubernetes Pod，里面有多个容器。每个容器都有不同的环境变量。

我的问题：

  1. 在一个 Pod 中使用多个容器是否明智。易于区分，但由于它们紧密耦合，我猜测失败的可能性很高，而且实际上不是正确的微服务设计。
  2. 我是否应该为每个管道创建多个部署？将非常难以维护，因为每个都有不同的部署配置。
  3. 有没有更好的方法来解决这个问题？

步骤 1 的示例：

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations: {}
  name: test-raw-mongodb-sink-apps
  namespace: test-apps
spec:
  selector:
    matchLabels:
      app: test-raw-mongodb-sink-apps
  template:
    metadata:
      labels:
        app: test-raw-mongodb-sink-apps
    spec:
      containers:
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-alchemy
        - name: INPUT_TOPIC
          value: test.raw.ptv.alchemy
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8081"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/dpl/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-alchemy
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-bloomberg
        - name: INPUT_TOPIC
          value: test.raw.pretrade.bloomberg
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8082"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-bloomberg
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-calypso
        - name: INPUT_TOPIC
          value: test.raw.ptv.calypso
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8083"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-calypso
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-dtres
        - name: INPUT_TOPIC
          value: test.raw.ptv.dtres
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8084"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-dtres
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-feds
        - name: INPUT_TOPIC
          value: test.raw.ptv.feds
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8085"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-feds
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-hoops
        - name: INPUT_TOPIC
          value: test.raw.ptv.hoops
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8086"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-hoops
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxcore
        - name: INPUT_TOPIC
          value: test.raw.ptv.murex_core
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8087"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxcore
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxeqd
        - name: INPUT_TOPIC
          value: test.raw.ptv.murex_eqd_sa
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8088"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxeqd
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxgts
        - name: INPUT_TOPIC
          value: test.raw.ptv.murex_gts_sa
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8089"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxgts
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxmr
        - name: INPUT_TOPIC
          value: test.raw.ptv.murex_mr
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8090"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxmr
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxgtscf
        - name: INPUT_TOPIC
          value: test.raw.cashflow.murex_gts_sa
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8091"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxgtscf
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxcoll
        - name: INPUT_TOPIC
          value: test.raw.collateral.mxcoll
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8092"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxcoll
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxcoll-link
        - name: INPUT_TOPIC
          value: test.raw.collateral.mxcoll_link
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8093"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxcoll-link
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-ost
        - name: INPUT_TOPIC
          value: test.raw.ptv.ost
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8094"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-ost
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-posmon
        - name: INPUT_TOPIC
          value: test.raw.ptp.posmon
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8095"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-posmon
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true

谢谢

解决方法

像 Helm 这样的模板工具可以让您填写部署时设置中的环境变量值。在 Helm 中，这看起来像：

env:
  - name: EVENTS_TOPIC
    value: {{ .Values.eventsTopic }}
  - name: GROUP_ID
    value: {{ .Values.groupId }}
  - name: INPUT_TOPIC
    value: {{ .Values.inputTopic }}

然后您可以使用不同的主题集多次部署它：

helm install alchemy . \
  --set eventsTopic=test.ops.proc-events \
  --set groupId=test-mongodb-sink-alchemy \
  --set inputTopic=test.raw.ptv.alchemy
helm install bloomberg . \
  --set eventsTopic=test.ops.proc-events \
  --set groupId=test-mongodb-sink-bloomberg \
  --set inputTopic=test.raw.pretrade.bloomberg

您也可以编写 Helm 图表以配置主题集列表，并且只部署一次：

{{- $top := . -}}{{-/* because "range" overwrites "." */-}}
{{- $topic := range $topics -}}
---
apiVersion: v1
kind: Deployment
metadata:
  name: {{ $topic.name }}
spec:
  ...
    env:
      - name: EVENT_TOPIC
        value: {{ $top.Values.eventTopic }}{{/* common to all deployments */}}
      - name: GROUP_ID
        value: test-mongodb-sink-{{ $topic.name }}
      - name: INPUT_TOPIC
        value: {{ $topic.inputTopic }}

编写如下配置：

eventTopic: test.ops.proc-events
topics:
  - name: alchemy
    inputTopic: test.raw.ptv.alchemy
  - name: bloomberg
    inputTopic: test.raw.pretrade.bloomberg

并像这样部署：

helm install connector . -f topic-listing.yaml

无论如何，每个 Pod 只需要一个容器。这有几个原因。如果主题列表发生变化，这使您可以在不干扰其他主题的情况下创建或删除部署；如果所有东西都在一个 pod 中，你必须一起停止并重新启动所有东西，而 Kafka 可能需要一两分钟才能弄清楚会发生什么。在 Kafka 上下文中，您还可以运行与主题上的分区一样多的消费者，但不会更多；如果您有一个非常繁忙的主题，您可以轻松地将该部署的 replicas: 设置为为多个分区提供多个使用者，但如果所有内容都在一个 pod 中，您唯一的选择就是将所有内容一起扩展。

,

在一个 Pod 中使用多个容器是否明智。易于区分，但由于它们紧密耦合，我猜测失败的可能性很高，而且实际上不是正确的微服务设计。

您很可能希望将它们部署为单独的服务，以便您可以相互独立地更新或重新配置它们。

我应该为每个管道创建多个部署吗？将很难维护，因为每个都有不同的部署配置。

Kustomize 是 kubectl 中的内置工具，当您想在具有不同配置的多个环境中部署相同的清单时，它是一个不错的选择。除了 kubectl 之外，此解决方案不需要其他工具。

使用 Kustomize 部署到多个环境

目录结构：

base/
  - deployment.yaml      # fully deployable manifest - no templating
  - kustomization.yaml   # default values e.g. for dev environment
app1/
  - kustomization.yaml   # specific values for app1
app2/
  - kustomization.yaml   # specific values for app2

使用 Kustomization 的示例部署清单

这里，环境变量是从 ConfigMap 加载的，因此我们可以使用 configMapGenerator。这个文件是base/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongodb-sink
  namespace: test-apps
spec:
  template: // some fiels,e.g. labels are omitted in example
    spec:
      containers:
      - name: mongodb-sink
        image: mongodb-map:0.0.7.0-SNAPSHOT
        env:
          - name: MONGODB_HOST0
            value: test-mongodb-0.test-mongodb-headless.test-infra
          - name: MONGODB_HOST1
            value: test-mongodb-1.test-mongodb-headless.test-infra
          - name: GROUP_ID
            valueFrom:
              configMapKeyRef:
                name: my-values
                key: GROUP_ID
          - name: INPUT_TOPIC
            valueFrom:
              configMapKeyRef:
                name: my-values
                key: INPUT_TOPIC
      ...

还要添加一个 base/kustomization.yaml 文件来描述 configMapGenerator 和相关文件。

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- deployment.yaml

configMapGenerator:
- name: my-values
  behavior: replace
  literals:
  - GROUP_ID=test-mongodb-sink-calypso
  - INPUT_TOPIC=test.raw.ptv.calypso
  ... # also add your other values

预览清单

kubectl kustomize base/

应用清单

kubectl apply -k base/

为 app1 和 app2 添加配置

对于 app1，我们现在想要使用 base/ 中的清单，并覆盖 app1 的不同之处。此文件为 app1/kustomization.yaml，与 app2/kustomization.yaml 类似。

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

bases:
- ../base

namePrefix: bloomberg-sink-   # this gives your Deployment a prefixed name

configMapGenerator:
- name: my-values
  behavior: replace
  literals:
  - GROUP_ID=test-mongodb-sink-bloomberg
  - INPUT_TOPIC=test.raw.pretrade.bloomberg
  ... # also add your other values

预览清单

kubectl kustomize app1/

应用清单

kubectl apply -k app1/

文档

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。

相关推荐

matplotlib报错：AttributeError: module 'backend_interagg' has no attribute 'FigureCanvas'. Did you mean: 'FigureCanvasAgg'?
使用本地python环境可以成功执行 import pandas as pd import matplotlib.pyplot as plt # 设置字体 plt.rcParams['font.sans-serif'] = ['SimHei'] # 能正确显示负号 p
gitlab登录失败，报错：This challenge page was accidentally cached by an intermediary and is no longer available.
设置时间 控制面板
后端开发常见错误
错误1：Request method ‘DELETE‘ not supported 错误还原：controller层有一个接口，访问该接口时报错：Request method ‘DELETE‘ not supported 错误原因：没有接收到前端传入的参数，修改为如下 参考 错误2：cannot r
docker常见错误
错误1：启动docker镜像时报错：Error response from daemon: driver failed programming external connectivity on endpoint quirky_allen 解决方法：重启docker -> systemctl r
idea常见错误
错误1：private field ‘xxx‘ is never assigned 按Altʾnter快捷键，选择第2项 参考：https://blog.csdn.net/shi_hong_fei_hei/article/details/88814070 错误2：启动时报错，不能找到主启动类 #
pip安装依赖失败
报错如下，通过源不能下载，最后警告pip需升级版本 Requirement already satisfied: pip in c:\users\ychen\appdata\local\programs\python\python310\lib\site-packages (22.0.4) Coll
maven常见错误
错误1：maven打包报错 错误还原：使用maven打包项目时报错如下 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.2.0:resources (default-resources)
cloud项目中常见错误
错误1：服务调用时报错 服务消费者模块assess通过openFeign调用服务提供者模块hires 如下为服务提供者模块hires的控制层接口 @RestController @RequestMapping("/hires") public class FeignControl
springboot常见错误
错误1：运行项目后报如下错误 解决方案 报错2：Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project sb 解决方案：在pom.
springmvc拦截器中使用redisTemplate报空指针异常
参考 错误原因 过滤器或拦截器在生效时，redisTemplate还没有注入 解决方案：在注入容器时就生效 @Component //项目运行时就注入Spring容器 public class RedisBean { @Resource private RedisTemplate<String
vite报错：npm ERR! command C:WINDOWSsystem32cmd.exe /d /s /c C:UsersychenAppDataLocalTempnpx-dde1c848.cmd
使用vite构建项目报错 C:\Users\ychen\work>npm init @vitejs/app @vitejs/create-app is deprecated, use npm init vite instead C:\Users\ychen\AppData\Local\npm-
VM VirtualBox中安装centos时报错：设置基础软件仓库时出错
参考1 参考2 解决方案 # 点击安装源 协议选择 http:// 路径填写 mirrors.aliyun.com/centos/8.3.2011/BaseOS/x86_64/os URL类型 软件库URL 其他路径 # 版本 7 mirrors.aliyun.com/centos/7/os/x86
kafka常见问题
报错1 [root@slave1 data_mocker]# kafka-console-consumer.sh --bootstrap-server slave1:9092 --topic topic_db [2023-12-19 18:31:12,770] WARN [Consumer clie
hive常见问题
错误1 # 重写数据 hive (edu)> insert overwrite table dwd_trade_cart_add_inc > select data.id, > data.user_id, > data.course_id, > date_format(
装载数据时报错：Failed to execute spark task, with exception 'org.apache.hadoop.hive.ql.metadata.HiveException(Failed to create spark client.)'
错误1 hive (edu)> insert into huanhuan values(1,'haoge'); Query ID = root_20240110071417_fe1517ad-3607-41f4-bdcf-d00b98ac443e Total jobs = 1
flume常见问题
报错1：执行到如下就不执行了，没有显示Successfully registered new MBean. [root@slave1 bin]# /usr/local/software/flume-1.9.0/bin/flume-ng agent -n a1 -c /usr/local/softwa
hadoop集群常见错误
虚拟及没有启动任何服务器查看jps会显示jps，如果没有显示任何东西 [root@slave2 ~]# jps 9647 Jps 解决方案 # 进入/tmp查看 [root@slave1 dfs]# cd /tmp [root@slave1 tmp]# ll 总用量 48 drwxr-xr-x. 2
hive常见错误
报错1 hive> show databases; OK Failed with exception java.io.IOException:java.lang.RuntimeException: Error in configuring object Time taken: 0.474 se
Linux常见错误
报错1 [root@localhost ~]# vim -bash: vim: 未找到命令 安装vim yum -y install vim* # 查看是否安装成功 [root@hadoop01 hadoop]# rpm -qa |grep vim vim-X11-7.4.629-8.el7_9.x
使用sqoop一直卡在：mapreduce.Job: Running job: job_1703173956074_0001
修改hadoop配置 vi /usr/local/software/hadoop-2.9.2/etc/hadoop/yarn-site.xml # 添加如下 <configuration> <property> <name>yarn.nodemanager.res
pythonJavaScriptjavaHTMLPHPreactjsC#AndroidCSSNode.jssqlrpython-3.xMysqLjQueryc++pandasFlutterangularIOSdjangolinuxswifttypescript路由器JSON路由器设置无线路由器h3c华三华三路由器设置华三路由器电脑软件教程arraysdocker软件图文教程Cvue.jslaravelspring-boot