OpenAI GYM/stable_baselines3 和多核

如何解决OpenAI GYM/stable_baselines3 和多核

似乎 GYM/stable_baselines3 随机器内核数量的增加而严重扩展。是“正常”吗？还是我机器上的硬件配置问题？ FPS 严重下降（随着时间的增加）。 这是一些mwe

128 核：~100 FPS
$ time python3 mwe.py
真正的 2m35,323s
用户 139m27,725s
系统 18m44,585s

2核
$ taskset --cpu-list 1,2 python3 mwe.py
1300 帧/秒
真正的 0m9,331s
用户 0m12,990s
系统 0m0,244s

32 核
$ time taskset --cpu-list 0-31 python3 mwe.py
800 帧/秒
真正的 0m16,181s
用户 2m54,283s
系统 0m25,847s

from stable_baselines3 import PPO
import gym
from gym import error,spaces,utils
from gym.utils import seeding

#https://github.com/MatePocs/gym-basic/blob/main/gym_basic/envs/basic_env.py
class BasicEnv(gym.Env):
    Metadata = {'render.modes': ['human']}

    def __init__(self):
        # There are two actions,first will get reward of 1,second reward of -1. 
        self.action_space = spaces.discrete(5)
        self.observation_space = spaces.discrete(2)

    def step(self,action):

        # if we took an action,we were in state 1
        state = 1
    
        if action == 2:
            reward = 1
        else:
            reward = -1
            
        # regardless of the action,game is done after a single step
        done = True

        info = {}

        return state,reward,done,info

    def reset(self):
        state = 0
        return state
  
    def render(self,mode='human'):
        pass

    def close(self):
        pass

env = BasicEnv()

ppo = PPO ("MlpPolicy",env,verbose=1,learning_rate=0.0003
        )
ppo.learn(total_timesteps=10000)