从 Amazon Sagemaker 工作室的源代码安装软件包

如何解决从 Amazon Sagemaker 工作室的源代码安装软件包

我正在 Amazon Sagemaker Studio 笔记本上从源代码安装一些包克隆 Git 存储库。

在 Sagemaker 之外，我设法安装了例如 neuralcoref，没有任何问题：

git clone https://github.com/huggingface/neuralcoref.git
cd neuralcoref
pip install -r requirements.txt
pip install -e .

成功运行 README 中的示例。

# Load your usual SpaCy model (one of SpaCy English models)
import spacy
nlp = spacy.load('en')

# Add neural coref to SpaCy's pipe
import neuralcoref
neuralcoref.add_to_pipe(nlp)

# You're done. You can Now use NeuralCoref as you usually manipulate a SpaCy document annotations.
doc = nlp(u'My sister has a dog. She loves him.')

doc._.has_coref
doc._.coref_clusters

但是，如果我在 Amazon Sagemaker Studio 笔记本上执行相同操作，则会在安装过程中出现错误：

/opt/conda/lib/python3.7/site-packages/secretstorage/dhcrypto.py:16: CryptographyDeprecationWarning: int_from_bytes is deprecated,use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/opt/conda/lib/python3.7/site-packages/secretstorage/util.py:25: CryptographyDeprecationWarning: int_from_bytes is deprecated,use int.from_bytes instead
  from cryptography.utils import int_from_bytes
Requirement already satisfied: spacy<3.0.0,>=2.1.0 in /opt/conda/lib/python3.7/site-packages (from -r ../neuralcoref/requirements.txt (line 1)) (2.3.7)
Requirement already satisfied: cython>=0.25 in /opt/conda/lib/python3.7/site-packages (from -r ../neuralcoref/requirements.txt (line 2)) (0.29.15)
Requirement already satisfied: pytest in /opt/conda/lib/python3.7/site-packages (from -r ../neuralcoref/requirements.txt (line 3)) (5.3.5)
Requirement already satisfied: plac<1.2.0,>=0.9.6 in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (1.1.3)
Requirement already satisfied: thinc<7.5.0,>=7.4.1 in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (7.4.5)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (56.2.0)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (3.0.5)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (2.0.5)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (2.25.1)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (4.42.1)
Requirement already satisfied: catalogue<1.1.0,>=0.0.7 in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (1.0.0)
Requirement already satisfied: blis<0.8.0,>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (0.7.4)
Requirement already satisfied: srsly<1.1.0,>=1.0.2 in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (1.0.5)
Requirement already satisfied: numpy>=1.15.0 in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (1.20.0)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (1.0.5)
Requirement already satisfied: wasabi<1.1.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (0.8.2)
Requirement already satisfied: importlib-Metadata>=0.20 in /opt/conda/lib/python3.7/site-packages (from catalogue<1.1.0,>=0.0.7->spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (1.5.0)
Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-Metadata>=0.20->catalogue<1.1.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (3.5.0)
Requirement already satisfied: chardet<5,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (2021.5.30)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (1.26.4)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0,>=2.1.0->-r ../neuralcoref/requirements.txt (line 1)) (2.8)
Requirement already satisfied: py>=1.5.0 in /opt/conda/lib/python3.7/site-packages (from pytest->-r ../neuralcoref/requirements.txt (line 3)) (1.10.0)
Requirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from pytest->-r ../neuralcoref/requirements.txt (line 3)) (21.0)
Requirement already satisfied: attrs>=17.4.0 in /opt/conda/lib/python3.7/site-packages (from pytest->-r ../neuralcoref/requirements.txt (line 3)) (19.3.0)
Requirement already satisfied: more-itertools>=4.0.0 in /opt/conda/lib/python3.7/site-packages (from pytest->-r ../neuralcoref/requirements.txt (line 3)) (8.2.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in /opt/conda/lib/python3.7/site-packages (from pytest->-r ../neuralcoref/requirements.txt (line 3)) (0.13.1)
Requirement already satisfied: wcwidth in /opt/conda/lib/python3.7/site-packages (from pytest->-r ../neuralcoref/requirements.txt (line 3)) (0.1.8)
Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->pytest->-r ../neuralcoref/requirements.txt (line 3)) (2.4.6)
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 21.1.1; however,version 21.1.3 is available.
You should consider upgrading via the '/opt/conda/bin/python -m pip install --upgrade pip' command.
/opt/conda/lib/python3.7/site-packages/secretstorage/dhcrypto.py:16: CryptographyDeprecationWarning: int_from_bytes is deprecated,use int.from_bytes instead
  from cryptography.utils import int_from_bytes
Obtaining file:///root/experiments/neuralcoref
Requirement already satisfied: numpy>=1.15.0 in /opt/conda/lib/python3.7/site-packages (from neuralcoref==4.0) (1.20.0)
Requirement already satisfied: boto3 in /opt/conda/lib/python3.7/site-packages (from neuralcoref==4.0) (1.17.74)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /opt/conda/lib/python3.7/site-packages (from neuralcoref==4.0) (2.25.1)
Requirement already satisfied: spacy<3.0.0,>=2.1.0 in /opt/conda/lib/python3.7/site-packages (from neuralcoref==4.0) (2.3.7)
Requirement already satisfied: chardet<5,>=2.13.0->neuralcoref==4.0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests<3.0.0,>=2.13.0->neuralcoref==4.0) (2021.5.30)
Requirement already satisfied: urllib3<1.27,>=2.13.0->neuralcoref==4.0) (1.26.4)
Requirement already satisfied: idna<3,>=2.13.0->neuralcoref==4.0) (2.8)
Requirement already satisfied: plac<1.2.0,>=2.1.0->neuralcoref==4.0) (1.1.3)
Requirement already satisfied: preshed<3.1.0,>=2.1.0->neuralcoref==4.0) (3.0.5)
Requirement already satisfied: cymem<2.1.0,>=2.1.0->neuralcoref==4.0) (2.0.5)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from spacy<3.0.0,>=2.1.0->neuralcoref==4.0) (56.2.0)
Requirement already satisfied: tqdm<5.0.0,>=2.1.0->neuralcoref==4.0) (4.42.1)
Requirement already satisfied: blis<0.8.0,>=2.1.0->neuralcoref==4.0) (0.7.4)
Requirement already satisfied: thinc<7.5.0,>=2.1.0->neuralcoref==4.0) (7.4.5)
Requirement already satisfied: srsly<1.1.0,>=2.1.0->neuralcoref==4.0) (1.0.5)
Requirement already satisfied: catalogue<1.1.0,>=2.1.0->neuralcoref==4.0) (1.0.0)
Requirement already satisfied: wasabi<1.1.0,>=2.1.0->neuralcoref==4.0) (0.8.2)
Requirement already satisfied: murmurhash<1.1.0,>=2.1.0->neuralcoref==4.0) (1.0.5)
Requirement already satisfied: importlib-Metadata>=0.20 in /opt/conda/lib/python3.7/site-packages (from catalogue<1.1.0,>=2.1.0->neuralcoref==4.0) (1.5.0)
Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-Metadata>=0.20->catalogue<1.1.0,>=2.1.0->neuralcoref==4.0) (3.5.0)
Requirement already satisfied: s3transfer<0.5.0,>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from boto3->neuralcoref==4.0) (0.4.2)
Requirement already satisfied: botocore<1.21.0,>=1.20.74 in /opt/conda/lib/python3.7/site-packages (from boto3->neuralcoref==4.0) (1.20.74)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3->neuralcoref==4.0) (0.10.0)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /opt/conda/lib/python3.7/site-packages (from botocore<1.21.0,>=1.20.74->boto3->neuralcoref==4.0) (2.8.1)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.7/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.21.0,>=1.20.74->boto3->neuralcoref==4.0) (1.14.0)
Installing collected packages: neuralcoref
  Attempting uninstall: neuralcoref
    Found existing installation: neuralcoref 4.0
    Can't uninstall 'neuralcoref'. No files were found to uninstall.
  Running setup.py develop for neuralcoref
    ERROR: Command errored out with exit status 1:
     command: /opt/conda/bin/python -c 'import io,os,sys,setuptools,tokenize; sys.argv[0] = '"'"'/root/xperiments/neuralcoref/setup.py'"'"'; __file__='"'"'/root/experiments/neuralcoref/setup.py'"'"';f = getattr(tokenize,'"'"'open'"'"',open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"','"'"'\n'"'"');f.close();exec(compile(code,__file__,'"'"'exec'"'"'))' develop --no-deps
         cwd: /root/experiments/neuralcoref/
    Complete output (15 lines):
    Cythonizing sources
    running develop
    running egg_info
    writing neuralcoref.egg-info/PKG-INFO
    writing dependency_links to neuralcoref.egg-info/dependency_links.txt
    writing requirements to neuralcoref.egg-info/requires.txt
    writing top-level names to neuralcoref.egg-info/top_level.txt
    adding license file 'LICENSE.txt' (matched pattern 'LICEN[CS]E*')
    reading manifest file 'neuralcoref.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'neuralcoref.egg-info/SOURCES.txt'
    running build_ext
    building 'neuralcoref.neuralcoref' extension
    gcc -pthread -B /opt/conda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/Opt/conda/include/python3.7m -I/root/experiments/neuralcoref/include -I/Opt/conda/include/python3.7m -c neuralcoref/neuralcoref.cpp -o build/temp.linux-x86_64-3.7/neuralcoref/neuralcoref.o -O2 -Wno-strict-prototypes -Wno-unused-function
    error: command 'gcc' Failed with exit status 1
    ----------------------------------------
  ERROR: Can't roll back neuralcoref; was not uninstalled
ERROR: Command errored out with exit status 1: /opt/conda/bin/python -c 'import io,tokenize; sys.argv[0] = '"'"'/root/experiments/neuralcoref/setup.py'"'"'; __file__='"'"'/root/experiments/neuralcoref/setup.py'"'"';f = getattr(tokenize,'"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

我尝试这样做：conda develop . 而不是 pip install -e . 并且它没有给出错误，但是在脚本的执行中它引发了错误。 neuralcoref 只是一个例子：我用相同的操作安装了 fairseq，它给了我这样的错误：

ImportError: Please build Cython components with: `pip install --editable .` or `python setup.py build_ext --inplace`

就像在 Sagemaker 中，我无法使用 pip install --editable . 安装 Cython 依赖项。我该怎么做？