在Kubernetes中部署的私有存储库Airflow的image_pull_secrets错误

如何解决在Kubernetes中部署的私有存储库Airflow的image_pull_secrets错误

我已经在天蓝色的kubernetes中部署了Apache气流。

Apache气流头盔存储库：https://github.com/apache/airflow/tree/master/chart

AKS版本：1.16.13

一旦部署了气流，我就会用这个dag测试它：

from airflow import DAG
from datetime import datetime,timedelta
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.operators.dummy_operator import DummyOperator


default_args = {
    'owner': 'airflow','depends_on_past': False,'start_date': datetime.utcnow(),'email': ['[e-mail]'],'email_on_failure': False,'email_on_retry': False,'retries': 1,'retry_delay': timedelta(minutes=5)
}

dag = DAG(
    'kubernetes_sample',default_args=default_args,schedule_interval=timedelta(minutes=10))


start = DummyOperator(task_id='run_this_first',dag=dag)

passing = KubernetesPodOperator(namespace='default',image="python:3.8-slim-buster",cmds=["python3","-c"],arguments=["print('hello world')"],labels={"foo": "bar"},name="passing-test",task_id="passing-task",get_logs=True,dag=dag
                          )


passing.set_upstream(start)

工作正常。现在，我想使用自己的图像。为此，我正在使用azure容器，并遵循以下指南：https://airflow.readthedocs.io/en/latest/howto/operator/kubernetes.html，我正在使用以下代码创建秘密来访问我的azure注册表：

kubectl create secret docker-registry testquay \
    --docker-server=quay.io \
    --docker-username=<Profile name> \
    --docker-password=<password>

我建立了映像，并在本地进行了测试，并且可以正常工作。我将图像上传到azure容器注册表，并编写了以下dag：

from airflow import DAG
from datetime import datetime,timedelta
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.operators.dummy_operator import DummyOperator
from airflow.contrib.kubernetes import secret
import logging
import os
import sys
import traceback 

try:
    env_var_secret = secret.Secret(
        deploy_type='env',deploy_target='VERSION_NUMBER',secret='myregistrykey',key='VERSION_NUMBER',)
    default_args = {
        'owner': 'airflow','retry_delay': timedelta(minutes=5)
    }

    dag = DAG(
        'jordi_test_get_secert2',schedule_interval=timedelta(minutes=10))


    start = DummyOperator(task_id='run_this_first',dag=dag)

    quay_k8s = KubernetesPodOperator(
            namespace='default',name="passing-test7",image='docker.io/test-pai-1',image_pull_secrets=env_var_secret,task_id="passing-task6",dag=dag
        )


    start >> quay_k8s

except Exception as e:

    error_message = {
            "message": "An internal error ocurred","error": str(e),"error information" : str(sys.exc_info()),"traceback": str(traceback.format_exc())
        }
    logging.info(error_message)

给我这个错误：

 File "/home/airflow/.local/lib/python3.6/site-packages/airflow/kubernetes/pod_generator.py",line 272,in __init__
    for image_pull_secret in image_pull_secrets.split(','):
AttributeError: 'Secret' object has no attribute 'split'

遵循本指南：https://airflow.readthedocs.io/en/latest/howto/operator/kubernetes.html

from airflow import DAG
from datetime import datetime,timedelta
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.operators.dummy_operator import DummyOperator
from kubernetes.client import models as k8s
import logging
import os
import sys
import traceback 

try:

    default_args = {
        'owner': 'airflow','retry_delay': timedelta(minutes=5)
    }

    dag = DAG(
        'jordi_test2',dag=dag)

    
    quay_k8s = KubernetesPodOperator(
    namespace='default',image_pull_secrets=[k8s.V1LocalObjectReference('myregistrykey')],dag=dag
)



    start >> quay_k8s

except Exception as e:

    error_message = {
            "message": "An internal error ocurred","traceback": str(traceback.format_exc())
        }
    logging.info(error_message)

但是给我这个错误：

for image_pull_secret in image_pull_secrets.split(','):
AttributeError: 'list' object has no attribute 'split'

如果我转到KubernetesPodOperator的Airflow文档：https://airflow.apache.org/docs/stable/_api/airflow/contrib/operators/kubernetes_pod_operator/index.html

说：

 image_pull_secrets (str) – Any image pull secrets to be given to the pod. If more than one secret is required,provide a comma separated list: secret_a,secret_b

正确的书写方式是什么？

解决方法

请您澄清一下，但我没有声誉。因此，提供一些假设的解决方案。

1. 由于您提供了指向最新版Airflow的文档链接，因此我假设您正在使用它。
2. 您发布的代码未更改。

在这里，您的秘密被命名为 testquay

kubectl create secret docker-registry testquay \
    --docker-server=quay.io \
    --docker-username=<Profile name> \
    --docker-password=<password>

但是您在发布的代码中将其引用为 myregistrykey ，意在遵循您引用的示例。

image_pull_secrets=[k8s.V1LocalObjectReference('myregistrykey')],

应根据示例引用秘密名称。

image_pull_secrets=[k8s.V1LocalObjectReference('testquay')],

对于第一个DAG代码，我也不认为您可以以这种方式传递Secrets对象。它们打算在运行时作为卷或env变量注入到k8s pod中。 https://github.com/apache/airflow/blob/v1-10-stable/airflow/kubernetes/secret.py#L35-L40