python – “http.client.CannotSendRequest：Request-sent”错误

奇怪的问题在这里.我有一个24/7运行的Python 3脚本,使用Selenium和Firefox进入网页,每5分钟从下载链接下载一个文件(我不能用urllib下载,或者其他什么,因为即使下载文件的链接地址保持不变,文件中的数据不断变化,每次重新加载页面时都不同,并且还取决于指定的标准.该脚本几乎一直运行良好,但我无法摆脱每隔一段时间弹出一次终止脚本的错误.这是错误：

Traceback (most recent call last):
  File "/Users/Shared/ROTH_1/Folio/get_F_notes.py", line 248, in <module>
    driver.get(search_url)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 187, in get
    self.execute(Command.GET, {'url': url})
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/selenium/webdriver/remote/remote_connection.py", line 349, in execute
    return self._request(command_info[0], url, body=data)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/selenium/webdriver/remote/remote_connection.py", line 379, in _request
    self._conn.request(method, parsed_url.path, body, headers)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1090, in request
    self._send_request(method, url, body, headers)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1118, in _send_request
    self.putrequest(method, url, **skips)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 966, in putrequest
    raise CannotSendRequest(self.__state)
http.client.CannotSendRequest: Request-sent

以下是我的脚本中出现问题的部分,具体来说,脚本命中“除了ConnectionRefusedError：”部分,并按照预期打印出“警告1：ConnectionRefusedError：搜索页面未加载;现在再次尝试”.但是,我认为,当循环再次开始并再次尝试“driver.get(search_url)”时,我得到上述错误.脚本在那时窒息并给我上述错误.

我对此进行了相当多的研究,似乎脚本试图从第一次尝试开始重用相同的连接.修复似乎是创建一个新的连接.但这就是我能够收集的全部内容,我不知道如何与Selenium建立新的连接.你呢？或者,这里还有其他一些问题吗？

search_url = 'https://www.example.com/download_page'
loop_get_search_page = 1
while loop_get_search_page < 7:
    if loop_get_search_page == 6:
        print('WARNING: tried to load search page 5 times; exiting script to try again later')
        ##### log out
        try:
            driver.find_element_by_link_text('Sign Out')
        except NoSuchElementException:
            print('WARNING: NoSuchElementException: Unable to find the link text for the "Sign Out" button')
        driver.quit()
        raise SystemExit
    try:
        driver.get(search_url)
    except TimeoutException:
        print('WARNING ', loop_get_search_page, ': TimeoutException: search page did not load; Now trying again', sep='')
        loop_get_search_page += 1
        continue
    except ConnectionRefusedError:
        print('WARNING ', loop_get_search_page, ': ConnectionRefusedError: search page did not load; Now trying again')
        loop_get_search_page += 1
        continue
    else:
        break

解决方法:

我自己遇到了这个问题.在我的情况下,我有另一个线程正在运行,也通过WebDriver发出请求.结果证明WebDriver不是线程安全的.

查看Can Selenium use multi threading in one browser?的讨论以及那里的链接以获取更多背景信息.

当我删除其他线程时,一切都按预期开始工作.

您是否有可能在新线程中每隔5米跑一次？

我知道“创建新连接”的唯一方法是启动WebDriver的新实例.如果你做了很多请求,那可能会变慢,但由于你每隔5米只做一次,所以它不应该真正影响你的吞吐量.只要您在完成dl时始终清理WebDriver实例,这对您来说可能是一个不错的选择.

原文地址：https://codeday.me/bug/20190623/1271394.html

python – “http.client.CannotSendRequest：Request-sent”错误

相关推荐