带有负载均衡器错误上游的 Azure-AKS 入口过早关闭了 websocket 连接上的连接

如何解决带有负载均衡器错误上游的 Azure-AKS 入口过早关闭了 websocket 连接上的连接

我有一个 nodejs express 服务器，它带有在 Azure Kubernetes 服务上运行的 socket.io 应用程序。简单的 Http/s 请求工作正常，Websocket 连接在入口日志中出现以下错误

2021/03/30 06:59:51 [error] 1773#1773: *316873 upstream prematurely closed connection while reading response header from upstream,client: 123.212.143.111,server: socketio.medflix.app,request: "GET / HTTP/1.1",upstream: "http://10.244.0.10:3000/",host: "mydomain.com"
2021/03/30 06:59:52 [error] 1773#1773: *316873 upstream prematurely closed connection while reading response header from upstream,host: "mydomain.com"
2021/03/30 06:59:53 [error] 1773#1773: *316873 upstream prematurely closed connection while reading response header from upstream,host: "mydomain.com"
123.212.143.111 - - [30/Mar/2021:06:59:53 +0000] "GET / HTTP/1.1" 502 559 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/89.0.4389.90 Safari/537.36" 536 3.005 [default-websocketsamplesvc-8080] [] 10.244.0.10:3000,10.244.0.10:3000,10.244.0.10:3000 0,0 1.004,1.000,1.004 502,502,502 f167278bb44053d41f3b7d0ee10093d5

这是我使用的集群设置

---
apiVersion: v1
kind: Service
Metadata:
  name: websocketsamplesvc
  annotations: 
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
  ports:
  - port: 8080
    targetPort: 3000
    protocol: TCP
  selector:
    app: websocketsampledeploy
  type: LoadBalancer

---
apiVersion: apps/v1
kind: Deployment
Metadata:
  name: websocketsampledeploy
spec:
  selector:
    matchLabels:
      app: websocketsampledeploy
  replicas: 1
  progressDeadlineseconds: 3600
  template:
    Metadata:
      labels:
        app: websocketsampledeploy
    spec:
      volumes:
        - name: secret-volume
          secret:
            secretName: env-secret-websocketsample     
      imagePullSecrets:
        - name: mycreds
      containers:
        - name: websocketsampledeploy
          image: myuser/myimage:v1
          imagePullPolicy: Always
          ports:
            - containerPort: 3000
          resources:
            limits:
              cpu: "1"
              memory: 1200Mi
            requests:
              cpu: 500m
              memory: 800Mi    
          envFrom:
            - secretRef:
                name: env-secret-websocketsample
          volumeMounts:
            - name: secret-volume
              readOnly: true
              mountPath: "/etc/secret-volume"



--- 
  apiVersion: extensions/v1beta1
  kind: Ingress
  Metadata: 
    annotations: 
      kubernetes.io/ingress.class: Nginx-api
      Nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
      Nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600"
      Nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
      Nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
      Nginx.ingress.kubernetes.io/send-timeout: "3600"
      Nginx.ingress.kubernetes.io/proxy-body-size: 10m    
      cert-manager.io/cluster-issuer: letsencrypt  
      Nginx.ingress.kubernetes.io/websocket-services: "websocketsamplesvc"
      Nginx.org/websocket-services: "websocketsamplesvc"
    name: websocketsampleingress
  spec: 
    tls:
      - hosts:
        - mydomain.com
        secretName: tls-secret
    rules: 
      - 
        host: mydomain.com
        http:
          paths: 
            - 
              backend: 
                serviceName: websocketsamplesvc
                servicePort: 8080
              path: /



---
apiVersion: apps/v1
kind: Deployment
Metadata:
  name: nginx-ingress-controller
  namespace: ingress-Nginx
  labels:
    app.kubernetes.io/name: ingress-Nginx
    app.kubernetes.io/part-of: ingress-Nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-Nginx
      app.kubernetes.io/part-of: ingress-Nginx
  template:
    Metadata:
      labels:
        app.kubernetes.io/name: ingress-Nginx
        app.kubernetes.io/part-of: ingress-Nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeselector:
        kubernetes.io/os: linux
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:master
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/Nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-Nginx
            - --annotations-prefix=Nginx.ingress.kubernetes.io
            - --ingress-class=Nginx-api
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 101
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldpath: Metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldpath: Metadata.namespace
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successthreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successthreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

要测试：我正在使用浏览器 API，--> new WebSocket('wss://mydomain.com')

似乎负载均衡器服务正在拒绝 wss 连接？ 似乎无法弄清楚我错过了什么？请帮忙。