UnicodeDecodeError：'utf-8'编解码器在读取熊猫中的csv文件时无法解码位置1的字节0x8b：无效的起始字节

如何解决UnicodeDecodeError：'utf-8'编解码器在读取熊猫中的csv文件时无法解码位置1的字节0x8b：无效的起始字节

仍然最有可能压缩数据。gzip的魔数是0x1f 0x8b，与UnicodeDecodeError您得到的一致。

您可以尝试动态解压缩数据：

with open('destinations.csv', 'rb') as fd:
    gzip_fd = gzip.GzipFile(fileobj=fd)
    destinations = pd.read_csv(gzip_fd)

解决方法

我知道已经问过类似的问题，我已经看过所有问题并且尝试过，但几乎没有帮助。我正在使用OSX 10.11 El
Capitan，python3.6。，虚拟环境，也尝试不使用它。我正在使用jupyter笔记本和spyder3。

我是python的新手，但是了解基本的ML，并在一篇博文中学习了如何解决Kaggle的挑战：链接到Blog，链接到数据集

我被困在代码的前几行

import pandas as pd

destinations = pd.read_csv("destinations.csv")
test = pd.read_csv("test.csv")
train = pd.read_csv("train.csv")

这给了我错误

UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-19-a928a98eb1ff> in <module>()
      1 import pandas as pd
----> 2 df = pd.read_csv('destinations.csv',compression='infer',date_parser=True,usecols=([0,1,3]))
      3 df.head()

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer,sep,delimiter,header,names,index_col,usecols,squeeze,prefix,mangle_dupe_cols,dtype,engine,converters,true_values,false_values,skipinitialspace,skiprows,nrows,na_values,keep_default_na,na_filter,verbose,skip_blank_lines,parse_dates,infer_datetime_format,keep_date_col,date_parser,dayfirst,iterator,chunksize,compression,thousands,decimal,lineterminator,quotechar,quoting,escapechar,comment,encoding,dialect,tupleize_cols,error_bad_lines,warn_bad_lines,skipfooter,skip_footer,doublequote,delim_whitespace,as_recarray,compact_ints,use_unsigned,low_memory,buffer_lines,memory_map,float_precision)
    653                     skip_blank_lines=skip_blank_lines)
    654 
--> 655         return _read(filepath_or_buffer,kwds)
    656 
    657     parser_f.__name__ = name

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in _read(filepath_or_buffer,kwds)
    403 
    404     # Create the parser.
--> 405     parser = TextFileReader(filepath_or_buffer,**kwds)
    406 
    407     if chunksize or iterator:

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in __init__(self,f,**kwds)
    762             self.options['has_index_names'] = kwds['has_index_names']
    763 
--> 764         self._make_engine(self.engine)
    765 
    766     def close(self):

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in _make_engine(self,engine)
    983     def _make_engine(self,engine='c'):
    984         if engine == 'c':
--> 985             self._engine = CParserWrapper(self.f,**self.options)
    986         else:
    987             if engine == 'python':

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in __init__(self,src,**kwds)
   1603         kwds['allow_leading_cols'] = self.index_col is not False
   1604 
-> 1605         self._reader = parsers.TextReader(src,**kwds)
   1606 
   1607         # XXX

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.__cinit__ (pandas/_libs/parsers.c:6175)()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._get_header (pandas/_libs/parsers.c:9691)()

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

关于stakoverflow的一些答案表明这是因为它已压缩，但Chrome下载了.csv文件，而.csv.gz却无处可见，并且找不到返回的文件错误。

然后encoding='latin1'，我在某个地方阅读了要使用的内容，但是这样做之后，我得到了解析器错误：

---------------------------------------------------------------------------
ParserError                               Traceback (most recent call last)
<ipython-input-21-f9c451f864a2> in <module>()
      1 import pandas as pd
      2 
----> 3 destinations = pd.read_csv("destinations.csv",encoding='latin1')
      4 test = pd.read_csv("test.csv")
      5 train = pd.read_csv("train.csv")

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer,kwds)
    409 
    410     try:
--> 411         data = parser.read(nrows)
    412     finally:
    413         parser.close()

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in read(self,nrows)
   1003                 raise ValueError('skipfooter not supported for iteration')
   1004 
-> 1005         ret = self._engine.read(nrows)
   1006 
   1007         if self.options.get('as_recarray'):

/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py in read(self,nrows)
   1746     def read(self,nrows=None):
   1747         try:
-> 1748             data = self._reader.read(nrows)
   1749         except StopIteration:
   1750             if self._first_chunk:

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.read (pandas/_libs/parsers.c:10862)()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_low_memory (pandas/_libs/parsers.c:11138)()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._read_rows (pandas/_libs/parsers.c:11884)()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._tokenize_rows (pandas/_libs/parsers.c:11755)()

pandas/_libs/parsers.pyx in pandas._libs.parsers.raise_parser_error (pandas/_libs/parsers.c:28765)()

ParserError: Error tokenizing data. C error: Expected 2 fields in line 11,saw 3

我花了几个小时来调试它，试图在Atom上打开CSV文件（没有其他应用程序可以打开它），在线Web应用程序（有些崩溃了），但是没有帮助。我尝试使用已经解决了其他问题的人的内核问题，但没有帮助。