微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦!

python-熊猫从系列列表中写入可变数量的新行

我正在使用Pandas作为从Selenium写入数据的方法.

网页上的搜索框ac_results的两个示例结果:

#Search for product_id = "01"
ac_results = "Orange (10)"

#Search for product_id = "02"
ac_result = ["Banana (10)", "Banana (20)", "Banana (30)"]

Orange仅返回一个价格($10),而Banana从不同供应商返回可变数量的价格,在此示例中,三个价格分别为($10),($20)和($30).

代码通过re.findall使用正则表达式来获取每个价格并将它们放入列表中.只要re.findall仅找到一个列表项(对于Oranges),该代码就可以正常工作.
问题是价格波动不定,例如搜索香蕉时.我想为每个所述价格创建一个新行,并且该行还应包含product_id和item_name.

电流输出

product_id      prices                  item_name
01              10                      Orange
02              [u'10', u'20', u'30']   Banana

所需的输出

product_id      prices                  item_name
01              10                      Orange
02              10                      Banana
02              20                      Banana
02              30                      Banana

当前代码

df = pd.read_csv("product_id.csv")
def crawl(product_id):
    #Enter search input here, omitted
    #Getting results:
    search_result = driver.find_element_by_class_name("ac_results")
    item_name = re.match("^.*(?=(\())", search_result.text).group().encode("utf-8")
    prices = re.findall("((?<=\()[0-9]*)", search_reply.text)
    return pd.Series([prices, item_name])

df[["prices", "item_name"]] = df["product_id"].apply(crawl)
df.to_csv("write.csv", index=False)

仅供参考:带有csv模块的可行解决方案,但我想使用Pandas.

with open("write.csv", "a") as data_write:
    wr_data = csv.writer(data_write, delimiter = ",")
    for price in prices: #<-- This is the important part!
        wr_insref.writerow([product_id, price, item_name])

解决方法:

# initializing here for reproducibility
pids = ['01','02']
prices = [10, [u'10', u'20', u'30']]
names = ['Orange','Banana']
df = pd.DataFrame({"product_id": pids, "prices": prices, "item_name": names})

在您应用(抓取)之后,以下代码段应起作用.

# convert all of the prices to lists (even if they only have one element)
df.prices = df.prices.apply(lambda x: x if isinstance(x, list) else [x])

# Create a new dataframe which splits the lists into separate columns.
# Then flatten using stack. The explicit MultiIndex allows us to keep
# the item_name and product_id associated with each price.
idx = pd.MultiIndex.from_tuples(zip(*[df['item_name'],df['product_id']]), 
                                names = ['item_name', 'product_id'])
df2 = pd.DataFrame(df.prices.tolist(), index=idx).stack()

# drop the hierarchical index and select columns of interest
df2 = df2.reset_index()[['product_id', 0, 'item_name']]
# rename back to prices
df2.columns = ['product_id', 'prices', 'item_name']

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。

相关推荐