如何解决kubeflow 卡在管道启动中:无法附加或挂载卷:未挂载的卷=[mlpipeline-minio-artifact]
kubernetes 和 kubeflow 都在我的 GPU 服务上设置,而当我尝试启动教程管道时出现错误,看起来卡在了关于 pvc 和 pv 的某些东西中。
详情如下:
...
Environment Variables from:
metadata-grpc-configmap ConfigMap Optional: true
Environment:
WORKFLOW_ID: (v1:metadata.labels['workflows.argoproj.io/workflow'])
KFP_POD_NAME: parameterized-tfx-oss-s7krr-2814409879 (v1:metadata.name)
KFP_NAMESPACE: kubeflow-user-example-com (v1:metadata.namespace)
WORKFLOW_ID: (v1:metadata.labels['workflows.argoproj.io/workflow'])
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-editor-token-qn279 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
podmetadata:
Type: DownwardAPI (a volume populated by information about the pod)
Items:
metadata.annotations -> annotations
docker-sock:
Type: HostPath (bare host directory volume)
Path: /var/run/docker.sock
HostPathType: Socket
mlpipeline-minio-artifact:
Type: Secret (a volume populated by a Secret)
SecretName: mlpipeline-minio-artifact
Optional: false
default-editor-token-qn279:
Type: Secret (a volume populated by a Secret)
SecretName: default-editor-token-qn279
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedMount 43m (x65 over 13h) kubelet,admin Unable to attach or mount volumes: unmounted volumes=[mlpipeline-minio-artifact],unattached volumes=[mlpipeline-minio-artifact default-editor-token-qn279 podmetadata docker-sock]: timed out waiting for the condition
Warning FailedMount 34m (x73 over 14h) kubelet,unattached volumes=[default-editor-token-qn279 podmetadata docker-sock mlpipeline-minio-artifact]: timed out waiting for the condition
Warning FailedMount 30m (x58 over 14h) kubelet,unattached volumes=[docker-sock mlpipeline-minio-artifact default-editor-token-qn279 podmetadata]: timed out waiting for the condition
Warning FailedMount 15m (x421 over 14h) kubelet,admin MountVolume.SetUp failed for volume "mlpipeline-minio-artifact" : secret "mlpipeline-minio-artifact" not found
Warning FailedMount 5m19s (x175 over 14h) kubelet,unattached volumes=[podmetadata docker-sock mlpipeline-minio-artifact default-editor-token-qn279]: timed out waiting for the condition
解决方法
在这里我找到了这个问题的方法。 k8s1.19 用于部署 kubeflow1.3,而当我回退到 k8s1.17 时问题就消失了。就是这样。
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。