pandas get_close_matches - 返回空值

如何解决pandas get_close_matches - 返回空值

我正在处理一个要求，有 2 个 CSV 如下 -

CSV1.csv

   Short Description                                                    Category
    Device is DOWN!                                                      Server Down
    cpu Warning Monitoron  XSSXSXSXSXSX.com                              cpu utilization
    cpu Warning Monitoron  XSSXSXSXSXSX.com                              cpu utilization
    cpu Warning Monitoron  XSSXSXSXSXSX.com                              cpu utilization
    cpu Warning Monitoron  XSSXSXSXSXSX.com                              cpu utilization
    Device Performance Alerts was triggered on Physical memory           Memory utilization
    Device Performance Alerts was triggered on Physical memory           Memory utilization
    Device Performance Alerts was triggered on Physical memory           Memory utilization
    disk Space Is Lowon ;E:                                              disk Space utilization
    disk Space Is Lowon;C:                                               disk Space utilization
    Network Interface Down                                               Interface Down
    Active Directory                                                     

和reference.csv

Category                         Complexity
Server Down                      Simple
Network Interface down           Complex
Drive Cleanup Windows            Medium
cpu utilization                  Medium
Memory utilization               Medium
disk Space utilization Unix      Simple
Windows Service Restart          Medium
UNIX Service Restart             Medium
Web Tomcat Instance Restart      Simple

Expected Output

Short Description                                                    Category                    Complexity
Device is DOWN!                                                      Server Down                 Simple
cpu Warning Monitoron  XSSXSXSXSXSX.com                              cpu utilization             Medium
cpu Warning Monitoron  XSSXSXSXSXSX.com                              cpu utilization             Medium
cpu Warning Monitoron  XSSXSXSXSXSX.com                              cpu utilization             Medium
cpu Warning Monitoron  XSSXSXSXSXSX.com                              cpu utilization             Medium
Device Performance Alerts was triggered on Physical memory           Memory utilization          Medium
Device Performance Alerts was triggered on Physical memory           Memory utilization          Medium
Device Performance Alerts was triggered on Physical memory           Memory utilization          Medium
disk Space Is Lowon ;E:                                              disk Space utilization      Medium
disk Space Is Lowon;C:                                               disk Space utilization      Medium
Network Interface Down                                               Interface Down              Complex

我尝试了以下代码 - 但在输出数据框中我可以看到空白 [] 不知道我错过了什么。在输出复杂度列中，每行我只能看到 []。我试图获得精确匹配，但我需要获得所有可能的组合，所以我使用了 get_close_matches。如何在下面的代码中传递数据帧中的可能性参数，我不知道传递可能性的方法。

我尝试了一些其他的可能性，例如精确但没有给出预期的结果，因为我在比较列与字符串时寻找所有可能的组合

import pandas as pd
import difflib
df1 = pd.read_csv('csv1.csv')
df1 = df1.fillna('')
df2 = pd.read_csv('reference.csv')
my_dict = dict(zip(df2['Category'].values,df2['Complexity'].values))
def match_key(key,default_value):
    if not key:
        return default_value
    for d_key in my_dict.keys():
        if key in d_key or d_key in key:
            return my_dict[d_key]

    return default_value
df1['Complexity'] = df1['Category'].apply(lambda x: difflib.get_close_matches(x,list(my_dict.keys(),n=1)))
df1 = df1.explode('Complexity')
df1['Complexity'] = df1['Complexity'].map(my_dict)
print(df1)

解决方法

difflib.get_close_matches 期望第一个参数是“单词”，在您的情况下为 x，第二个参数是“可能性”。您提供的空字符串。这就是为什么你的函数不起作用，它试图匹配一个没有任何东西的单词。

my_dict 包含有效选项作为键，因此我们可以将它们用作“可能性”列表

# Use n=1,so only tries to get 1 match
df1['Complexity'] = df1['Category'].apply(lambda x: difflib.get_close_matches(x,list(my_dict.keys()),n=1))
# The output of get_close_matches is a list,we use explode to convert it to a string
df1 = df1.explode('Complexity')
# We can now apply our map,to the *Complexity* column,# which is technically the best match *Category*,via get_close_matches
df1['Complexity'] = df1['Complexity'].map(my_dict)

原始错误答案

然而，与其继续使用 difflib，我认为您可以改变您的方法。您想将 my_dict 应用到 Category 的 df1 列。这在传统上称为应用 map。 pandas 通过 pandas.Series.map 准备好这个实现。

df1['Complexity'] = df1['Category'].map(my_dict)