Webscraping 'urllib.error.HTTPError: HTTP Error 404: Not Found' 错误

如何解决Webscraping 'urllib.error.HTTPError: HTTP Error 404: Not Found' 错误

我正在学习从 yahoo Finance 网站抓取数据的教程，但我的代码出现此错误：

'urllib.error.HTTPError：HTTP 错误 404：未找到'错误。

我的代码：

import pandas as pd

amd_profile = pd.read_html("https://finance.yahoo.com/quote/AMD/profile?p=AMD")

print(amd_profile)

错误：

Traceback (most recent call last):
  File "c:\Users\x\Desktop\My Python Sandbox Projects\vscode folder 1\code_sandbox.py",line 3,in <module>
    profile = pd.read_html("https://finance.yahoo.com/quote/AMD/profile?p=AMD")
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\util\_decorators.py",line 299,in wrapper
    return func(*args,**kwargs)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\io\html.py",line 1085,in read_html      
    return _parse(
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\io\html.py",line 893,in _parse
    tables = p.parse_tables()
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\io\html.py",line 213,in parse_tables    
    tables = self._parse_tables(self._build_doc(),self.match,self.attrs)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\io\html.py",line 732,in _build_doc      
    raise e
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\io\html.py",line 713,in _build_doc      
    with urlopen(self.io) as f:
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\io\common.py",line 195,in urlopen       
    return urllib.request.urlopen(*args,**kwargs)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\urllib\request.py",line 214,in urlopen
    return opener.open(url,data,timeout)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\urllib\request.py",line 523,in open
    response = meth(req,response)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\urllib\request.py",line 632,in http_response
    response = self.parent.error(
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\urllib\request.py",line 561,in error
    return self._call_chain(*args)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\urllib\request.py",line 494,in _call_chain
    result = func(*args)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\urllib\request.py",line 641,in http_error_default
    raise HTTPError(req.full_url,code,msg,hdrs,fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

解决方法

快速阅读 Pandas 文档：

https://pandas.pydata.org/docs/reference/api/pandas.read_html.html?highlight=read_html#pandas.read_html

"一个 URL、一个类似文件的对象或一个包含 HTML 的原始字符串。请注意，lxml 只接受 http、ftp 和文件 url 协议。如果您有一个以 'https' 开头的 URL，您可以尝试删除's'。”