由于 wandb 的使用，PyTorch Lightning 希望在导入时创建一个文件夹，这会在 AWS Lambda 上引发错误

如何解决由于 wandb 的使用，PyTorch Lightning 希望在导入时创建一个文件夹，这会在 AWS Lambda 上引发错误

所以我想用 PyTorch Lightning 构建一个可以与 AWS lambda 一起使用的 Docker 镜像。然而，当该函数被调用时，它会引发一个操作系统错误，声称它使用的是只读文件系统，并且 wandb.py 想要写一些东西。

我尝试了这些：

1. 覆盖 pytroch Lightning 的 wandb.py 文件，它不会初始化 wandb --> 引发错误
2. 在 Dockerfile 中执行一个 python 脚本，文件是在 docker build 上创建并存在的，当调用 lambda 函数时 --> 相同的操作系统错误

有人知道跳过 wandb.py 的方法吗？

这是错误信息：

START RequestId: ddae284d-4f32-4dc6-8160-d1fa62ba9772 Version: $LATEST
OpenBLAS WARNING - could not determine the L2 cache size on this system,assuming 256k
[ERROR] OSError: [Errno 30] Read-only file system: '/home/sbx_user1051'
Traceback (most recent call last):
  File "/var/lang/lib/python3.8/imp.py",line 234,in load_module
    return load_source(name,filename,file)
  File "/var/lang/lib/python3.8/imp.py",line 171,in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>",line 702,in _load
  File "<frozen importlib._bootstrap>",line 671,in _load_unlocked
  File "<frozen importlib._bootstrap_external>",line 783,in exec_module
  File "<frozen importlib._bootstrap>",line 219,in _call_with_frames_removed
  File "/var/task/inference.py",line 5,in <module>
    import pytorch_lightning as pl
  File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/__init__.py",line 63,in <module>
    from pytorch_lightning.callbacks import Callback
  File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/callbacks/__init__.py",line 25,in <module>
    from pytorch_lightning.callbacks.swa import StochasticWeightAveraging
  File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/callbacks/swa.py",line 26,in <module>
    from pytorch_lightning.trainer.optimizers import _get_default_scheduler_config
  File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/trainer/__init__.py",line 18,in <module>
    from pytorch_lightning.trainer.trainer import Trainer
  File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py",line 30,in <module>
    from pytorch_lightning.loggers import LightningLoggerBase
  File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/loggers/__init__.py",line 31,in <module>
    from pytorch_lightning.loggers.wandb import _WANDB_AVAILABLE,WandbLogger  # noqa: F401
  File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/loggers/wandb.py",line 34,in <module>
    import wandb
  File "/var/lang/lib/python3.8/site-packages/wandb/__init__.py",line 131,in <module>
    api = InternalApi()
  File "/var/lang/lib/python3.8/site-packages/wandb/apis/internal.py",line 17,in __init__
    self.api = InternalApi(*args,**kwargs)
  File "/var/lang/lib/python3.8/site-packages/wandb/sdk/internal/internal_api.py",line 73,in __init__
    self._settings = Settings(
  File "/var/lang/lib/python3.8/site-packages/wandb/old/settings.py",in __init__
    self._global_settings.read([Settings._global_path()])
  File "/var/lang/lib/python3.8/site-packages/wandb/old/settings.py",line 105,in _global_path
    util.mkdir_exists_ok(config_dir)
  File "/var/lang/lib/python3.8/site-packages/wandb/util.py",line 687,in mkdir_exists_ok
    os.makedirs(path)
  File "/var/lang/lib/python3.8/os.py",line 213,in makedirs
    makedirs(head,exist_ok=exist_ok)
  File "/var/lang/lib/python3.8/os.py",line 223,in makedirs
    mkdir(name,mode)
END RequestId: ddae284d-4f32-4dc6-8160-d1fa62ba9772
REPORT RequestId: ddae284d-4f32-4dc6-8160-d1fa62ba9772  Duration: 27000.33 ms   Billed Duration: 27001 ms   Memory Size: 10240 MB   Max Memory Used: 241 MB 
Unknown application error occurred

解决方法

你需要确保你在某处有写权限。

然后可以使用wandb environment variables来修改文件本地保存的默认位置，特别是查看WANDB_DIR、WANDB_CONFIG_DIR和WANDB_CACHE_DIR。