如何解决在 bitbucket 管道中通过 xdist 并行运行 pytest 的错误

如何解决如何解决在 bitbucket 管道中通过 xdist 并行运行 pytest 的错误

我们在 bitbucket 管道中为我们的 python 存储库执行单元测试。最近，我们开始使用 pytest-xdist (https://pypi.org/project/pytest-xdist/) 包，并通过 coverage run -m pytest -n auto 命令在本地运行时成功地并行运行我们的单元测试。这会在我们本地机器上的所有可用 cpu 上执行所有单元测试。但是，当我们通过 bitbucket 管道使用相同的命令运行测试时，我们会遇到以下异常。有没有关于如何解决这个问题的建议？

+ coverage run -m pytest -n auto
============================= test session starts ==============================
platform linux -- Python 3.7.5,pytest-5.4.3,py-1.10.0,pluggy-0.13.1
rootdir: /opt/atlassian/pipelines/agent/build
plugins: celery-4.4.7,forked-1.3.0,xdist-2.2.1,mock-3.5.1
gw0 I / gw1 I / gw2 I / gw3 I
INTERNALERROR> def worker_internal_error(self,node,formatted_error):
INTERNALERROR>         """
INTERNALERROR>         pytest_internalerror() was called on the worker.
INTERNALERROR>     
INTERNALERROR>         pytest_internalerror() arguments are an excinfo and an excrepr,which can't
INTERNALERROR>         be serialized,so we go with a poor man's solution of raising an exception
INTERNALERROR>         here ourselves using the formatted message.
INTERNALERROR>         """
INTERNALERROR>         self._active_nodes.remove(node)
INTERNALERROR>         try:
INTERNALERROR> >           assert False,formatted_error
INTERNALERROR> E           AssertionError: Traceback (most recent call last):
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/_pytest/main.py",line 191,in wrap_session
INTERNALERROR> E                 session.exitstatus = doit(config,session) or 0
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/_pytest/main.py",line 246,in _main
INTERNALERROR> E                 config.hook.pytest_collection(session=session)
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py",line 286,in __call__
INTERNALERROR> E                 return self._hookexec(self,self.get_hookimpls(),kwargs)
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py",line 93,in _hookexec
INTERNALERROR> E                 return self._inner_hookexec(hook,methods,line 87,in <lambda>
INTERNALERROR> E                 firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py",line 208,in _multicall
INTERNALERROR> E                 return outcome.get_result()
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py",line 80,in get_result
INTERNALERROR> E                 raise ex[1].with_traceback(ex[2])
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py",line 187,in _multicall
INTERNALERROR> E                 res = hook_impl.function(*args)
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/_pytest/main.py",line 257,in pytest_collection
INTERNALERROR> E                 return session.perform_collect()
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/_pytest/main.py",line 453,in perform_collect
INTERNALERROR> E                 self.config.pluginmanager.check_pending()
INTERNALERROR> E               File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py",line 277,in check_pending
INTERNALERROR> E                 % (name,hookimpl.plugin),INTERNALERROR> E             pluggy.manager.PluginValidationError: unkNown hook 'pytest_warning_recorded' in plugin <__channelexec__.WorkerInteractor object at 0x7f11772d84d0>
INTERNALERROR> E           assert False
INTERNALERROR> 
INTERNALERROR> /usr/local/lib/python3.7/site-packages/xdist/dsession.py:187: AssertionError
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/_pytest/main.py",in wrap_session
INTERNALERROR>     session.exitstatus = doit(config,session) or 0
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/_pytest/main.py",line 247,in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/pluggy/hooks.py",in __call__
INTERNALERROR>     return self._hookexec(self,kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/pluggy/manager.py",in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook,in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py",in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py",in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/pluggy/callers.py",in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/xdist/dsession.py",line 112,in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/xdist/dsession.py",line 135,in loop_once
INTERNALERROR>     call(**kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.7/site-packages/xdist/dsession.py",line 175,in worker_workerfinished
INTERNALERROR>     self._active_nodes.remove(node)
INTERNALERROR> KeyError: <WorkerController gw0>

在此先感谢您的帮助！

解决方法

我遇到了同样的问题，但在将 pytest 升级到 6.2.2

后我无法解决此问题 ,

发现问题！ 我正在运行旧版本的 pytest。固定最新版本的 pytest（在撰写本文时是 pytest==6.2.2 到我的 requirements.txt 解决了这个问题。随后的问题也是 more-itertools 包没有内置到 python 环境中我配置的 bitbucket 管道容器也将其固定在要求中