如何解决无法启动 Spark 集群：ERROR worker.Worker：无法启动外部 shuffle 服务 java.net.BindException
我尝试根据此网站在 NodeManager 中配置 External shuffle Service：http://spark.apache.org/docs/2.4.5/running-on-yarn.html。 在配置此服务之前，集群工作正常。 在NodeManager中配置External shuffle Service后，spark集群启动失败。报错如下：
21/01/01 15:50:35 ERROR worker.Worker: Failed to start external shuffle service
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220)
at io.netty.channel.socket.nio.NioServerSocketChannel.dobind(NioServerSocketChannel.java:132)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:551)
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1346)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:503)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:488)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:985)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:247)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:344)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:510)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:518)
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
在每个节点运行jps命令，有Master进程但没有Worker进程。我该如何解决问题？
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。
