如何解决NameError: name '_mysql' is not defined -- 在 MacOSX 中启动气流
有很多关于标题问题的文章，但没有一篇对我有用。详细错误如下：
Traceback (most recent call last):
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/MysqLdb/__init__.py",line 18,in <module>
from . import _MysqL
ImportError: dlopen(/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/MysqLdb/_MysqL.cpython-37m-darwin.so,2): Symbol not found: _MysqL_affected_rows
Referenced from: /Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/MysqLdb/_MysqL.cpython-37m-darwin.so
Expected in: flat namespace
in /Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/MysqLdb/_MysqL.cpython-37m-darwin.so
During handling of the above exception,another exception occurred:
Traceback (most recent call last):
File "/Users/hiteshagarwal/Documents/venv/bin/airflow",line 25,in <module>
from airflow.configuration import conf
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/airflow/__init__.py",line 47,in <module>
settings.initialize()
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/airflow/settings.py",line 403,in initialize
configure_adapters()
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/airflow/settings.py",line 326,in configure_adapters
import MysqLdb.converters
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/MysqLdb/__init__.py",line 24,in <module>
version_info,_MysqL.version_info,_MysqL.__file__
NameError: name '_MysqL' is not defined
如果我使用 pip 从 python 环境中卸载 MysqLclient，那么我会收到这样的错误，然后我的气流网络服务器和调度程序运行但不是气流工作器，而在上述情况下，任何气流命令都不起作用。
-------------- celery@Hiteshs-MBP v4.4.7 (cliffs)
--- ***** -----
-- ******* ---- Darwin-20.3.0-x86_64-i386-64bit 2021-05-09 16:11:23
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app: airflow.executors.celery_executor:0x7fe6603a9668
- ** ---------- .> transport: sqla+MysqL://airflow:**@localhost:3306/airflow
- ** ---------- .> results: MysqL://airflow:**@localhost:3306/airflow
- *** --- * --- .> concurrency: 16 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> default exchange=default(direct) key=default
[tasks]
. airflow.executors.celery_executor.execute_command
[2021-05-09 16:11:23,308: INFO/MainProcess] Connected to sqla+MysqL://airflow:**@localhost:3306/airflow
[2021-05-09 16:11:23,318: CRITICAL/MainProcess] Unrecoverable error: ModuleNotFoundError("No module named 'MysqLdb'")
Traceback (most recent call last):
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/celery/worker/worker.py",line 208,in start
self.blueprint.start(self)
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/celery/bootsteps.py",line 119,in start
step.start(parent)
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/celery/bootsteps.py",line 369,in start
return self.obj.start()
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/celery/worker/consumer/consumer.py",line 318,in start
blueprint.start(self)
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/celery/bootsteps.py",in start
step.start(parent)
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/celery/worker/consumer/tasks.py",line 41,in start
c.connection,on_decode_error=c.on_decode_error,File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/celery/app/amqp.py",line 304,in TaskConsumer
**kw
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/kombu/messaging.py",line 386,in __init__
self.revive(self.channel)
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/kombu/messaging.py",line 408,in revive
self.declare()
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/kombu/messaging.py",line 421,in declare
queue.declare()
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/kombu/entity.py",line 611,in declare
self._create_queue(Nowait=Nowait,channel=channel)
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/kombu/entity.py",line 620,in _create_queue
self.queue_declare(Nowait=Nowait,passive=False,line 655,in queue_declare
Nowait=Nowait,File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/kombu/transport/virtual/base.py",line 531,in queue_declare
self._new_queue(queue,**kwargs)
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/kombu/transport/sqlalchemy/__init__.py",line 101,in _new_queue
self._get_or_create(queue)
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/kombu/transport/sqlalchemy/__init__.py",line 80,in _get_or_create
obj = self.session.query(self.queue_cls) \
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/kombu/transport/sqlalchemy/__init__.py",line 75,in session
_,Session = self._open()
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/kombu/transport/sqlalchemy/__init__.py",line 65,in _open
engine = self._engine_from_config()
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/kombu/transport/sqlalchemy/__init__.py",line 54,in _engine_from_config
return create_engine(conninfo.hostname,**transport_options)
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/sqlalchemy/engine/__init__.py",line 520,in create_engine
return strategy.create(*args,**kwargs)
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py",line 87,in create
dbapi = dialect_cls.dbapi(**dbapi_args)
File "/Users/hiteshagarwal/Documents/venv/lib/python3.7/site-packages/sqlalchemy/dialects/MysqL/MysqLdb.py",line 118,in dbapi
return __import__("MysqLdb")
ModuleNotFoundError: No module named 'MysqLdb'
Starting flask
* Serving Flask app "airflow.bin.cli" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production Wsgi server instead.
* Debug mode: off
[2021-05-09 16:11:23,815] {_internal.py:122} INFO - * Running on http://0.0.0.0:8793/ (Press CTRL+C to quit)
(venv) hiteshagarwal@Hiteshs-MBP Documents %
我在airflow.cfg 中使用的数据库配置 --
executor = CeleryExecutor
sql_alchemy_conn = postgresql+psycopg2://hiteshagarwal:<password>@127.0.0.1:5432/airflow --postgres
broker_url = sqla+MysqL+pyMysqL://airflow:<password>@localhost:3306/airflow --MysqL
celery_result_backend = db+MysqL+pyMysqL://hiteshagarwal:<password>@localhost/airflow --MysqL
更新
安装pip install apache-airflow-backport-providers-MysqL后，得到以下错误：
(airflow-env) hiteshagarwal@Hiteshs-MBP Documents % airflow worker
-------------- celery@Hiteshs-MBP v4.4.7 (cliffs)
--- ***** -----
-- ******* ---- Darwin-20.3.0-x86_64-i386-64bit 2021-05-09 17:02:30
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app: airflow.executors.celery_executor:0x7fd1fafa79b0
- ** ---------- .> transport: sqla+MysqL://airflow:**@localhost:3306/airflow
- ** ---------- .> results: MysqL://airflow:**@localhost:3306/airflow
- *** --- * --- .> concurrency: 16 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> default exchange=default(direct) key=default
[tasks]
. airflow.executors.celery_executor.execute_command
[2021-05-09 17:02:31,159: INFO/MainProcess] Connected to sqla+MysqL://airflow:**@localhost:3306/airflow
[2021-05-09 17:02:31,178: CRITICAL/MainProcess] Unrecoverable error: ImportError('dlopen(/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/_MysqL.cpython-37m-darwin.so,2): Symbol not found: _MysqL_affected_rows\n Referenced from: /Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/_MysqL.cpython-37m-darwin.so\n Expected in: flat namespace\n in /Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/_MysqL.cpython-37m-darwin.so')
Traceback (most recent call last):
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/celery/worker/worker.py",in start
self.blueprint.start(self)
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/celery/bootsteps.py",in start
step.start(parent)
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/celery/bootsteps.py",in start
return self.obj.start()
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/celery/worker/consumer/consumer.py",in start
blueprint.start(self)
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/celery/bootsteps.py",in start
step.start(parent)
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/celery/worker/consumer/tasks.py",File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/celery/app/amqp.py",in TaskConsumer
**kw
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/kombu/messaging.py",in __init__
self.revive(self.channel)
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/kombu/messaging.py",in revive
self.declare()
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/kombu/messaging.py",in declare
queue.declare()
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/kombu/entity.py",channel=channel)
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/kombu/entity.py",File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/kombu/transport/virtual/base.py",**kwargs)
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/kombu/transport/sqlalchemy/__init__.py",in _new_queue
self._get_or_create(queue)
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/kombu/transport/sqlalchemy/__init__.py",in _get_or_create
obj = self.session.query(self.queue_cls) \
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/kombu/transport/sqlalchemy/__init__.py",Session = self._open()
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/kombu/transport/sqlalchemy/__init__.py",in _open
engine = self._engine_from_config()
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/kombu/transport/sqlalchemy/__init__.py",**transport_options)
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/sqlalchemy/engine/__init__.py",line 500,**kwargs)
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py",in create
dbapi = dialect_cls.dbapi(**dbapi_args)
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/sqlalchemy/dialects/MysqL/MysqLdb.py",in dbapi
return __import__("MysqLdb")
File "/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/MysqLdb/__init__.py",in <module>
import _MysqL
ImportError: dlopen(/Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/_MysqL.cpython-37m-darwin.so,2): Symbol not found: _MysqL_affected_rows
Referenced from: /Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/_MysqL.cpython-37m-darwin.so
Expected in: flat namespace
in /Users/hiteshagarwal/Documents/WMall/airflow-env/lib/python3.7/site-packages/_MysqL.cpython-37m-darwin.so
Starting flask
* Serving Flask app "airflow.bin.cli" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production Wsgi server instead.
* Debug mode: off
[2021-05-09 17:02:32,079] {_internal.py:122} INFO - * Running on http://0.0.0.0:8793/ (Press CTRL+C to quit)
(airflow-env) hiteshagarwal@Hiteshs-MBP Documents %
我尝试了网络上提供的许多解决方案，但都没有奏效。以下是我在 venv 中安装的库列表：
alabaster==0.7.12
alembic==1.6.2
amqp==2.6.1
apache-airflow==1.10.12
apispec==1.3.3
appdirs==1.4.4
argcomplete==1.12.3
astor==0.8.1
attrs==19.3.0
autoflake==1.4
Babel==2.9.1
bandit==1.7.0
billiard==3.6.4.0
black==20.8b1
cached-property==1.5.2
cattrs==1.0.0
celery==4.4.7
certifi==2020.12.5
chardet==4.0.0
click==7.1.2
colorama==0.4.4
colorlog==4.0.2
configparser==3.5.3
coverage==5.5
croniter==0.3.37
darglint==1.8.0
defusedxml==0.7.1
dill==0.3.3
distlib==0.3.1
dnspython==2.1.0
doc8==0.8.1
docutils==0.17.1
email-validator==1.1.2
eradicate==1.0
filelock==3.0.12
Flake8==3.9.2
Flake8-2020==1.6.0
Flake8-bandit==2.1.2
Flake8-broken-line==0.2.1
Flake8-bugbear==19.8.0
Flake8-builtins==1.5.3
Flake8-commas==2.0.0
Flake8-comprehensions==3.4.0
Flake8-debugger==3.2.1
Flake8-docstrings==1.6.0
Flake8-eradicate==0.4.0
Flake8-fixme==1.1.1
Flake8-isort==3.0.1
Flake8-logging-format==0.6.0
Flake8-mutable==1.2.0
Flake8-polyfill==1.0.2
Flake8-quotes==2.1.2
Flake8-rst-docstrings==0.0.12
Flake8-string-format==0.2.3
Flake8-variables-names==0.0.3
Flask==1.1.2
Flask-Admin==1.5.4
Flask-AppBuilder==2.3.4
flask-babel==1.0.0
Flask-Caching==1.3.3
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-OpenID==1.2.5
Flask-sqlAlchemy==2.4.4
flask-swagger==0.2.14
Flask-WTF==0.14.3
funcsigs==1.0.2
future==0.18.2
gitdb==4.0.7
gitpython==3.1.15
graphviz==0.16
greenlet==1.1.0
gunicorn==20.1.0
idna==2.10
imagesize==1.2.0
importlib-Metadata==4.0.1
iniconfig==1.1.1
iso8601==0.1.14
isort==4.3.21
itsdangerous==1.1.0
Jinja2==2.11.3
json-merge-patch==0.2
jsonschema==3.2.0
kombu==4.6.11
lazy-object-proxy==1.6.0
lockfile==0.12.2
Mako==1.1.4
Markdown==2.6.11
MarkupSafe==1.1.1
marshmallow==2.21.0
marshmallow-enum==1.5.1
marshmallow-sqlalchemy==0.25.0
mccabe==0.6.1
mypy-extensions==0.4.3
natsort==7.1.1
numpy==1.20.2
packaging==20.9
pandas==1.1.5
pathspec==0.8.1
pbr==5.6.0
pendulum==1.4.4
pep8-naming==0.9.1
pluggy==0.13.1
prison==0.1.3
psutil==5.8.0
psycopg2-binary==2.8.6
py==1.10.0
pycodestyle==2.7.0
pydocstyle==5.1.1
pyflakes==2.3.1
Pygments==2.9.0
PyJWT==2.1.0
PyMysqL==1.0.2
pyparsing==2.4.7
pyrsistent==0.17.3
pytest==6.2.4
pytest-cov==2.11.1
python-daemon==2.3.0
python-dateutil==2.8.1
python-editor==1.0.4
python-nvd3==0.15.0
python-slugify==4.0.1
python3-openid==3.2.0
pytz==2021.1
pytzdata==2020.1
pyupgrade==2.15.0
PyYAML==5.4.1
regex==2021.4.4
requests==2.25.1
restructuredtext-lint==1.3.2
setproctitle==1.2.2
six==1.16.0
smmap==4.0.0
snowballstemmer==2.1.0
Sphinx==2.4.4
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
sqlAlchemy==1.3.23
sqlAlchemy-JSONField==0.9.0
sqlAlchemy-Utils==0.37.2
stevedore==3.3.0
tabulate==0.8.9
tenacity==4.12.0
testfixtures==6.17.1
text-unidecode==1.3
thrift==0.13.0
tokenize-rt==4.1.0
toml==0.10.2
tox==3.23.1
tox-travis==0.12
typed-ast==1.4.3
typing-extensions==3.10.0.0
tzlocal==1.5.1
unicodecsv==0.14.1
urllib3==1.26.4
vine==1.3.0
virtualenv==20.4.6
wemake-python-styleguide==0.14.1
Werkzeug==0.16.1
WTForms==2.3.3
zipp==3.4.1
zope.deprecation==4.4.0
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。
