如何解决如何对 docker 寄予厚望
我尝试了几个步骤，但遇到了问题。
下面是我的项目结构
GE-ToolProject
---great_expectations
--checkpoints
--expectations
-.ge_store_backend_id
-demo-suite.json
-Expectations.py
--notebooks
--plugins
--uncommitted
--great_expectations.yml
---dockerfile
---requirement.txt
在上面的结构中，GE-ToolProject 是父文件夹，它包含 great_expectations 文件夹结构，当你运行 great_expectations init 命令时，它会自动创建 我添加的附加内容是 Expectations.py 文件，我在其中编写了 Python 代码来运行期望并生成文档。
这里我的数据源是 redshift 数据库，我正在运行自定义查询来获取数据集。
dockerfile 包含：
FROM python:alpine3.7
copY . /app
workdir /app
RUN pip install -r requirements.txt
ENTRYPOINT [ "python" ]
CMD [ "Expectations.py" ]
requirements.txt 包含
Flask-sqlAlchemy==0.16
sqlalchemy==1.3.24
sqlalchemy-redshift
great_expectations
我想从 docker 运行 Expectations.py 文件。
当我尝试构建 docker 图像时出现以下错误
[+] Building 75.2s (9/9) FINISHED
=> [internal] load build deFinition from Dockerfile 0.0s
=> => transferring dockerfile: 179B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load Metadata for docker.io/library/python:alpine3.7 5.7s
=> [auth] library/python:pull token for registry-1.docker.io 0.0s
=> [internal] load build context 0.4s
=> => transferring context: 18.95kB 0.4s
=> CACHED [1/4] FROM docker.io/library/python:alpine3.7@sha256:35f6f83ab08f98c727dbefd53738e3b3174a48b4571ccb191 0.0s
=> [2/4] copY . /app 0.1s
=> [3/4] workdir /app 0.1s
=> ERROR [4/4] RUN pip install -r requirements.txt 68.8s
------
> [4/4] RUN pip install -r requirements.txt:
#9 1.462 Collecting Flask-sqlAlchemy==0.16 (from -r requirements.txt (line 1))
#9 4.117 Downloading https://files.pythonhosted.org/packages/92/31/0a7347379af42d211c6c2dc8a4e33f1855264220aed024902baa6f4ddaa2/Flask-sqlAlchemy-0.16.tar.gz (97kB)
#9 5.562 Collecting sqlalchemy==1.3.24 (from -r requirements.txt (line 2))
#9 6.920 Downloading https://files.pythonhosted.org/packages/c5/ab/81bef2f960abf3cdaf32fbf1994f0c6f5e6a5f1667b5713ed6ebf162b6a2/sqlAlchemy-1.3.24.tar.gz (6.4MB)
#9 18.41 Collecting sqlalchemy-redshift (from -r requirements.txt (line 3))
#9 18.82 Downloading https://files.pythonhosted.org/packages/9f/60/47bd9ab38ad5f0b1b20e8ccb1605dc173f75e962ef40deb5d794e0354352/sqlalchemy_redshift-0.8.2-py2.py3-none-any.whl
#9 18.85 Collecting great_expectations (from -r requirements.txt (line 4))
#9 19.39 Downloading https://files.pythonhosted.org/packages/36/9e/49c98831f32d4bf0fae61ffc83c2af39bb3522454b35b21963c193f4c27c/great_expectations-0.13.17-py3-none-any.whl (4.7MB)
#9 25.17 Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from Flask-sqlAlchemy==0.16->-r requirements.txt (line 1)) (40.7.1)
#9 25.18 Collecting Flask (from Flask-sqlAlchemy==0.16->-r requirements.txt (line 1))
#9 25.60 Downloading https://files.pythonhosted.org/packages/f2/28/2a03252dfb9ebf377f40fba6a7841b47083260bf8bd8e737b0c6952df83f/Flask-1.1.2-py2.py3-none-any.whl (94kB)
#9 25.73 Collecting packaging (from sqlalchemy-redshift->-r requirements.txt (line 3))
#9 26.15 Downloading https://files.pythonhosted.org/packages/3e/89/7ea760b4daa42653ece2380531c90f64788d979110a2ab51049d92f408af/packaging-20.9-py2.py3-none-any.whl (40kB)
#9 26.19 Collecting numpy<1.20.0,>=1.14.1 (from great_expectations->-r requirements.txt (line 4))
#9 27.78 Downloading https://files.pythonhosted.org/packages/51/60/3f0fe5b7675a461d96b9d6729beecd3532565743278a9c3fe6dd09697fa7/numpy-1.19.5.zip (7.3MB)
#9 41.58 Installing build dependencies: started
#9 48.57 Installing build dependencies: finished with status 'done'
#9 48.57 Getting requirements to build wheel: started
#9 48.86 Getting requirements to build wheel: finished with status 'done'
#9 48.86 Preparing wheel Metadata: started
#9 68.57 Preparing wheel Metadata: finished with status 'error'
#9 68.57 Complete output from command /usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_Metadata_for_build_wheel /tmp/tmpzb00tid0:
#9 68.57 Processing numpy/random/_bounded_integers.pxd.in
#9 68.57 Processing numpy/random/_pcg64.pyx
#9 68.57 Processing numpy/random/_philox.pyx
#9 68.57 Processing numpy/random/mtrand.pyx
#9 68.57 Processing numpy/random/_generator.pyx
#9 68.57 Processing numpy/random/_common.pyx
#9 68.57 Processing numpy/random/_mt19937.pyx
#9 68.57 Processing numpy/random/_sfc64.pyx
#9 68.57 Processing numpy/random/_bounded_integers.pyx.in
#9 68.57 Processing numpy/random/bit_generator.pyx
#9 68.57 Cythonizing sources
#9 68.57 blas_opt_info:
#9 68.57 blas_mkl_info:
#9 68.57 customize UnixCCompiler
#9 68.57 libraries mkl_rt not found in ['/usr/local/lib','/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 blis_info:
#9 68.57 libraries blis not found in ['/usr/local/lib','/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 openblas_info:
#9 68.57 libraries openblas not found in ['/usr/local/lib','/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 atlas_3_10_blas_threads_info:
#9 68.57 Setting PTATLAS=ATLAS
#9 68.57 libraries tatlas not found in ['/usr/local/lib','/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 atlas_3_10_blas_info:
#9 68.57 libraries satlas not found in ['/usr/local/lib','/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 atlas_blas_threads_info:
#9 68.57 Setting PTATLAS=ATLAS
#9 68.57 libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib','/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 atlas_blas_info:
#9 68.57 libraries f77blas,cblas,'/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 accelerate_info:
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 blas_info:
#9 68.57 libraries blas not found in ['/usr/local/lib','/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 blas_src_info:
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 non-existing path in 'numpy/distutils': 'site.cfg'
#9 68.57 lapack_opt_info:
#9 68.57 lapack_mkl_info:
#9 68.57 libraries mkl_rt not found in ['/usr/local/lib','/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 openblas_lapack_info:
#9 68.57 libraries openblas not found in ['/usr/local/lib','/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 openblas_clapack_info:
#9 68.57 libraries openblas,lapack not found in ['/usr/local/lib','/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 flame_info:
#9 68.57 libraries flame not found in ['/usr/local/lib','/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 atlas_3_10_threads_info:
#9 68.57 Setting PTATLAS=ATLAS
#9 68.57 libraries lapack_atlas not found in /usr/local/lib
#9 68.57 libraries tatlas,tatlas not found in /usr/local/lib
#9 68.57 libraries lapack_atlas not found in /usr/lib
#9 68.57 libraries tatlas,tatlas not found in /usr/lib
#9 68.57 <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 atlas_3_10_info:
#9 68.57 libraries lapack_atlas not found in /usr/local/lib
#9 68.57 libraries satlas,satlas not found in /usr/local/lib
#9 68.57 libraries lapack_atlas not found in /usr/lib
#9 68.57 libraries satlas,satlas not found in /usr/lib
#9 68.57 <class 'numpy.distutils.system_info.atlas_3_10_info'>
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 atlas_threads_info:
#9 68.57 Setting PTATLAS=ATLAS
#9 68.57 libraries lapack_atlas not found in /usr/local/lib
#9 68.57 libraries ptf77blas,atlas not found in /usr/local/lib
#9 68.57 libraries lapack_atlas not found in /usr/lib
#9 68.57 libraries ptf77blas,atlas not found in /usr/lib
#9 68.57 <class 'numpy.distutils.system_info.atlas_threads_info'>
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 atlas_info:
#9 68.57 libraries lapack_atlas not found in /usr/local/lib
#9 68.57 libraries f77blas,atlas not found in /usr/local/lib
#9 68.57 libraries lapack_atlas not found in /usr/lib
#9 68.57 libraries f77blas,atlas not found in /usr/lib
#9 68.57 <class 'numpy.distutils.system_info.atlas_info'>
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 lapack_info:
#9 68.57 libraries lapack not found in ['/usr/local/lib','/usr/lib']
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 lapack_src_info:
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 NOT AVAILABLE
#9 68.57
#9 68.57 numpy_linalg_lapack_lite:
#9 68.57 FOUND:
#9 68.57 language = c
#9 68.57 define_macros = [('HAVE_BLAS_ILP64',None),('BLAS_SYMBOL_SUFFIX','64_')]
#9 68.57
#9 68.57 running dist_info
#9 68.57 running build_src
#9 68.57 build_src
#9 68.57 building py_modules sources
#9 68.57 creating build
#9 68.57 creating build/src.linux-x86_64-3.7
#9 68.57 creating build/src.linux-x86_64-3.7/numpy
#9 68.57 creating build/src.linux-x86_64-3.7/numpy/distutils
#9 68.57 building library "npymath" sources
#9 68.57 Could not locate executable gfortran
#9 68.57 Could not locate executable f95
#9 68.57 Could not locate executable ifort
#9 68.57 Could not locate executable ifc
#9 68.57 Could not locate executable lf95
#9 68.57 Could not locate executable pgfortran
#9 68.57 Could not locate executable nvfortran
#9 68.57 Could not locate executable f90
#9 68.57 Could not locate executable f77
#9 68.57 Could not locate executable fort
#9 68.57 Could not locate executable efort
#9 68.57 Could not locate executable efc
#9 68.57 Could not locate executable g77
#9 68.57 Could not locate executable g95
#9 68.57 Could not locate executable pathf95
#9 68.57 Could not locate executable nagfor
#9 68.57 don't kNow how to compile Fortran code on platform 'posix'
#9 68.57 Running from numpy source directory.
#9 68.57 setup.py:480: UserWarning: Unrecognized setuptools command,proceeding with generating Cython sources and expanding templates
#9 68.57 run_build = parse_setuppy_commands()
#9 68.57 /tmp/pip-install-c4ba9ifm/numpy/numpy/distutils/system_info.py:1914: UserWarning:
#9 68.57 Optimized (vendor) Blas libraries are not found.
#9 68.57 Falls back to netlib Blas library which has worse performance.
#9 68.57 A better performance should be easily gained by switching
#9 68.57 Blas library.
#9 68.57 if self._calc_info(blas):
#9 68.57 /tmp/pip-install-c4ba9ifm/numpy/numpy/distutils/system_info.py:1914: UserWarning:
#9 68.57 Blas (http://www.netlib.org/blas/) libraries not found.
#9 68.57 Directories to search for the libraries can be specified in the
#9 68.57 numpy/distutils/site.cfg file (section [blas]) or by setting
#9 68.57 the BLAS environment variable.
#9 68.57 if self._calc_info(blas):
#9 68.57 /tmp/pip-install-c4ba9ifm/numpy/numpy/distutils/system_info.py:1914: UserWarning:
#9 68.57 Blas (http://www.netlib.org/blas/) sources not found.
#9 68.57 Directories to search for the sources can be specified in the
#9 68.57 numpy/distutils/site.cfg file (section [blas_src]) or by setting
#9 68.57 the BLAS_SRC environment variable.
#9 68.57 if self._calc_info(blas):
#9 68.57 /tmp/pip-install-c4ba9ifm/numpy/numpy/distutils/system_info.py:1748: UserWarning:
#9 68.57 Lapack (http://www.netlib.org/lapack/) libraries not found.
#9 68.57 Directories to search for the libraries can be specified in the
#9 68.57 numpy/distutils/site.cfg file (section [lapack]) or by setting
#9 68.57 the LAPACK environment variable.
#9 68.57 return getattr(self,'_calc_info_{}'.format(name))()
#9 68.57 /tmp/pip-install-c4ba9ifm/numpy/numpy/distutils/system_info.py:1748: UserWarning:
#9 68.57 Lapack (http://www.netlib.org/lapack/) sources not found.
#9 68.57 Directories to search for the sources can be specified in the
#9 68.57 numpy/distutils/site.cfg file (section [lapack_src]) or by setting
#9 68.57 the LAPACK_SRC environment variable.
#9 68.57 return getattr(self,'_calc_info_{}'.format(name))()
#9 68.57 /usr/local/lib/python3.7/distutils/dist.py:274: UserWarning: UnkNown distribution option: 'define_macros'
#9 68.57 warnings.warn(msg)
#9 68.57 Traceback (most recent call last):
#9 68.57 File "/usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py",line 207,in <module>
#9 68.57 main()
#9 68.57 File "/usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py",line 197,in main
#9 68.57 json_out['return_val'] = hook(**hook_input['kwargs'])
#9 68.57 File "/usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py",line 69,in prepare_Metadata_for_build_wheel
#9 68.57 return hook(Metadata_directory,config_settings)
#9 68.57 File "/tmp/pip-build-env-sn8bmpub/overlay/lib/python3.7/site-packages/setuptools/build_Meta.py",line 157,in prepare_Metadata_for_build_wheel
#9 68.57 self.run_setup()
#9 68.57 File "/tmp/pip-build-env-sn8bmpub/overlay/lib/python3.7/site-packages/setuptools/build_Meta.py",line 142,in run_setup
#9 68.57 exec(compile(code,__file__,'exec'),locals())
#9 68.57 File "setup.py",line 508,in <module>
#9 68.57 setup_package()
#9 68.57 File "setup.py",line 500,in setup_package
#9 68.57 setup(**Metadata)
#9 68.57 File "/tmp/pip-install-c4ba9ifm/numpy/numpy/distutils/core.py",line 169,in setup
#9 68.57 return old_setup(**new_attr)
#9 68.57 File "/tmp/pip-build-env-sn8bmpub/overlay/lib/python3.7/site-packages/setuptools/__init__.py",line 165,in setup
#9 68.57 return distutils.core.setup(**attrs)
#9 68.57 File "/usr/local/lib/python3.7/distutils/core.py",line 148,in setup
#9 68.57 dist.run_commands()
#9 68.57 File "/usr/local/lib/python3.7/distutils/dist.py",line 966,in run_commands
#9 68.57 self.run_command(cmd)
#9 68.57 File "/usr/local/lib/python3.7/distutils/dist.py",line 985,in run_command
#9 68.57 cmd_obj.run()
#9 68.57 File "/tmp/pip-build-env-sn8bmpub/overlay/lib/python3.7/site-packages/setuptools/command/dist_info.py",line 31,in run
#9 68.57 egg_info.run()
#9 68.57 File "/tmp/pip-install-c4ba9ifm/numpy/numpy/distutils/command/egg_info.py",line 24,in run
#9 68.57 self.run_command("build_src")
#9 68.57 File "/usr/local/lib/python3.7/distutils/cmd.py",line 313,in run_command
#9 68.57 self.distribution.run_command(command)
#9 68.57 File "/usr/local/lib/python3.7/distutils/dist.py",in run_command
#9 68.57 cmd_obj.run()
#9 68.57 File "/tmp/pip-install-c4ba9ifm/numpy/numpy/distutils/command/build_src.py",line 144,in run
#9 68.57 self.build_sources()
#9 68.57 File "/tmp/pip-install-c4ba9ifm/numpy/numpy/distutils/command/build_src.py",line 155,in build_sources
#9 68.57 self.build_library_sources(*libname_info)
#9 68.57 File "/tmp/pip-install-c4ba9ifm/numpy/numpy/distutils/command/build_src.py",line 288,in build_library_sources
#9 68.57 sources = self.generate_sources(sources,(lib_name,build_info))
#9 68.57 File "/tmp/pip-install-c4ba9ifm/numpy/numpy/distutils/command/build_src.py",line 378,in generate_sources
#9 68.57 source = func(extension,build_dir)
#9 68.57 File "numpy/core/setup.py",line 663,in get_mathlib_info
#9 68.57 raise RuntimeError("broken toolchain: cannot link a simple C program")
#9 68.57 RuntimeError: broken toolchain: cannot link a simple C program
#9 68.57
#9 68.57 ----------------------------------------
#9 68.68 Command "/usr/local/bin/python /usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_Metadata_for_build_wheel /tmp/tmpzb00tid0" Failed with error code 1 in /tmp/pip-install-c4ba9ifm/numpy
#9 68.71 You are using pip version 19.0.1,however version 21.0.1 is available.
#9 68.71 You should consider upgrading via the 'pip install --upgrade pip' command.
------
executor Failed running [/bin/sh -c pip install -r requirements.txt]: exit code: 1```
what I am missing here.
can someone please help me to resolve this.
解决方法
问题是docker容器无法编译。 使用
RUN apk add make automake gcc g++
或 alpine-sdk 等价于 build-essential
RUN apk add alpine-sdk
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。
