如何解决不平衡的多类分类管道
在3类分类问题中,我有一个矩阵X(特征,昏暗的320x7)和一个数组y(目标,昏暗的320x1)。我为5种不同的模型实现了gridsearchcv,以检查哪种模型最好,并调整其超参数。为了尽可能严格,我创建了一个管道来对每个折叠执行数据预处理。管道包括缩放,重采样和分类器。
由于目标向量不平衡(请参见图片),因此我使用SMOTE来平衡这三个类。现在的问题是,尽管我认为整个过程的顺序是正确的,但我的训练准确度非常高(0.9-0.99),但测试准确度却很低(0.5-0.55),我无法确定这是由于过拟合造成的错误的超参数空间搜索,或者是由于我的数据质量。
如果有人能照亮它,我将不胜感激。谢谢。
这是代码
# Import modules
import pickle
import numpy as np
from imblearn.over_sampling import SMOTE
from sklearn.model_selection import gridsearchcv
from sklearn.preprocessing import MinMaxScaler
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier,BaggingClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors import KNeighborsClassifier
from imblearn.pipeline import Pipeline
from sklearn.decomposition import PCA
from sklearn.svm import SVC
# Load features and target
with open('data.pkl','rb') as f:
dataset = pickle.load(f)
X = dataset['X']
y = dataset['y']
#%% Grid-search cross validation for different models
# Model search space
models = {
'LogisticRegression' : LogisticRegression(),'SVM' : SVC(),'RandomForestClassifier' : RandomForestClassifier(),'BaggingClassifier' : BaggingClassifier(),'KNN' : KNeighborsClassifier()}
# Hyperparameter search space
hyper = {
'LogisticRegression':{
'clf__penalty' : ['l2'],'clf__C' : np.logspace(-4,4,10),'clf__solver' : ['lbfgs','newton-cg','saga'],'clf__max_iter' : [1000,1500,2000],'clf__random_state': [0]},'SVM':{
'clf__C' : [0.001,0.01,0.1,1,10,100,1000,10000],'clf__gamma' : [0.01,0.001,0.0001],'clf__kernel' : ['rbf','linear'],'RandomForestClassifier':{
'clf__max_depth': [2,3,5,6],'clf__max_features': [2,'auto','sqrt'],'clf__n_estimators': [10,500,1000],'KNN':{
'clf__n_neighbors': [3,8,16]},'BaggingClassifier':{
'clf__base_estimator': [DecisionTreeClassifier()],'clf__max_samples' : [0.05,0.2,0.5],'clf__n_estimators' : [10,50,500],'clf__max_features' : [0.1,0.3,0.5]}
}
# Create variable to store CV results
grid_cv_results = dict()
# Scaler
escaler = MinMaxScaler()
# resampling
resampler = SMOTE(sampling_strategy = 'all',random_state = 0)
# Perform cross-validation for each model
for model_name in models.keys():
# Get model and hyperparameters
clf = models[model_name]
params = hyper[model_name]
# Pipeline (standarization + classifier)
pipe = Pipeline([('scale',escaler),('sampling',resampler),('clf',clf)])
# Gridsearch cross-validation
grid = gridsearchcv(estimator = pipe,param_grid = params,cv = 5,return_train_score = True)
grid.fit(X,y)
# Gridsearch cross-validation results
best_param = grid.best_params_
best_param_test_score_mean = grid.cv_results_['mean_test_score'][grid.best_index_]
best_param_test_score_std = grid.cv_results_['std_test_score'][grid.best_index_]
best_param_train_score_mean = grid.cv_results_['mean_train_score'][grid.best_index_]
best_param_train_score_std = grid.cv_results_['std_train_score'][grid.best_index_]
# Store gridsearch cross-validation results
grid_cv_results[model_name] = {
'best_param' : best_param,'best_param_test_score_mean' : best_param_test_score_mean,'best_param_test_score_std' : best_param_test_score_std,'best_param_train_score_mean' : best_param_train_score_mean,'best_param_train_score_std' : best_param_train_score_std,
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。