使用 **persist()** 后无法从 rdd 打印行

如何解决使用 **persist()** 后无法从 rdd 打印行

我正在使用以下代码

def process_row(row):
items = row.replace('"','')
items2 = items.split(' ')
for i in range(len(items2)):
#if we find ‘-’ we will replace it with ‘0’
  if(row[-1]=='-'):
    row[i]='0'
return [items2[0],items2[1],items2[2],items2[3],items2[4],int(items2[5])]

nasa = (
nasa_raw.map(process_row)
 )                     

nasa = nasa.persist()
first10 = nasa.collect()[:10]
for i,line in enumerate(first10):
print(f'Line {i+1:02}:  {line}')

在文本文件上：

in24.inetnebr.com [01/Aug/1995:00:00:01] "GET /shuttle/missions/sts-68/news/sts-68-mcc-05.txt" 200 1839 uplherc.upl.com [01/Aug/1995:00:00:07] "GET /" 304 0 uplherc.upl.com [01/Aug/1995:00:00:08] "GET /images/ksclogo-medium.gif" 304 0 uplherc.upl.com [01/Aug/1995:00:00:08] "GET /images/MOSAIC-logosmall.gif" 304 0 uplherc.upl.com [01/Aug/1995:00:00:08] "GET /images/USA-logosmall.gif" 304 0 ix-esc-ca2-07.ix.netcom.com [01/Aug/1995:00:00:09] "GET /images/launch-logo.gif" 200 1713 uplherc.upl.com [01/Aug/1995:00:00:10] "GET /images/WORLD-logosmall.gif" 304 0 slppp6.intermind.net [01/Aug/1995:00:00:10] "GET /history/skylab/skylab.html" 200 1687 piweba4y.prodigy.com [01/Aug/1995:00:00:10] "GET /images/launchmedium.gif" 200 11853 slppp6.intermind.net [01/Aug/1995:00:00:11] "GET /history/skylab/skylab-small.gif" 200 9202

我正在尝试打印 RDD 的前 10 个元素的行，每个元素出现在不同的输出行上。

我收到类似的错误

对象不支持项目分配'

有什么想法吗？

解决方法

使用您提供的示例数据，您的代码看起来不错（我将其重新格式化如下）。我想问题可能来自您的数据本身。尝试分解或限制您的数据集？

in24.inetnebr.com [01/Aug/1995:00:00:01] "GET /shuttle/missions/sts-68/news/sts-68-mcc-05.txt" 200 1839
uplherc.upl.com [01/Aug/1995:00:00:07] "GET /" 304 0
uplherc.upl.com [01/Aug/1995:00:00:08] "GET /images/ksclogo-medium.gif" 304 0
uplherc.upl.com [01/Aug/1995:00:00:08] "GET /images/MOSAIC-logosmall.gif" 304 0
uplherc.upl.com [01/Aug/1995:00:00:08] "GET /images/USA-logosmall.gif" 304 0
ix-esc-ca2-07.ix.netcom.com [01/Aug/1995:00:00:09] "GET /images/launch-logo.gif" 200 1713
uplherc.upl.com [01/Aug/1995:00:00:10] "GET /images/WORLD-logosmall.gif" 304 0
slppp6.intermind.net [01/Aug/1995:00:00:10] "GET /history/skylab/skylab.html" 200 1687
piweba4y.prodigy.com [01/Aug/1995:00:00:10] "GET /images/launchmedium.gif" 200 11853
slppp6.intermind.net [01/Aug/1995:00:00:11] "GET /history/skylab/skylab-small.gif" 200 9202

def process_row(row):
    items = row.replace('"','')
    items2 = items.split(' ')
    for i in range(len(items2)):
        #if we find ‘-’ we will replace it with ‘0’
        if(row[-1]=='-'):
            row[i]='0'
    return [items2[0],items2[1],items2[2],items2[3],items2[4],int(items2[5])]

nasa_raw = sc.textFile('a.txt')
nasa = (
    nasa_raw.map(process_row)
)                     

nasa = nasa.persist()
first10 = nasa.collect()[:10]
for i,line in enumerate(first10):
    print(f'Line {i+1:02}:  {line}')

# Output
Line 01:  ['in24.inetnebr.com','[01/Aug/1995:00:00:01]','GET','/shuttle/missions/sts-68/news/sts-68-mcc-05.txt','200',1839]
Line 02:  ['uplherc.upl.com','[01/Aug/1995:00:00:07]','/','304',0]
Line 03:  ['uplherc.upl.com','[01/Aug/1995:00:00:08]','/images/ksclogo-medium.gif',0]
Line 04:  ['uplherc.upl.com','/images/MOSAIC-logosmall.gif',0]
Line 05:  ['uplherc.upl.com','/images/USA-logosmall.gif',0]
Line 06:  ['ix-esc-ca2-07.ix.netcom.com','[01/Aug/1995:00:00:09]','/images/launch-logo.gif',1713]
Line 07:  ['uplherc.upl.com','[01/Aug/1995:00:00:10]','/images/WORLD-logosmall.gif',0]
Line 08:  ['slppp6.intermind.net','/history/skylab/skylab.html',1687]
Line 09:  ['piweba4y.prodigy.com','/images/launchmedium.gif',11853]
Line 10:  ['slppp6.intermind.net','[01/Aug/1995:00:00:11]','/history/skylab/skylab-small.gif',9202]