python机器学习-sklearn挖掘乳腺癌细胞( 博主亲自录制)
网易云观看地址
https://study.163.com/course/introduction.htm?courseId&＃61;1005269003&utm_campaign&＃61;commission&utm_source&＃61;cp-400000000398149&utm_medium&＃61;share
模型验证
分类器好坏验证&＃xff0c;模型建立好后&＃xff0c;不是万事大吉&＃xff0c;需要进行crossvalidation, AUC,GINi&＃xff0c;KS&＃xff0c;GainTable检验
KS可以检测模型区分好坏客户能力&＃xff0c;如果有一个分数段区分能力强&＃xff0c;KS会大于0.2
AUC检测模型分类器效果&＃xff0c;分类器敏感度越高&＃xff0c;AUC越大&＃xff0c;一般AUC大于0.7&＃xff0c;分类器准确性就不错。
Gain Table可以检测模型收益情况和排序能力
模型验证中数据要拆分为train(训练),test&＃xff08;测试&＃xff09;,oot&＃xff08;跨时间&＃xff09;
train和test是同一个时间段&＃xff0c;一般三七开&＃xff0c;train占百分之70&＃xff0c;test占百分之30
oot的时间段在train&＃xff0c;test后面&＃xff0c;用于测试未来数据
下图是模型验证的可视化&＃xff1a;
包括ROC&＃xff0c;提升图&＃xff0c;KS,PSI四个指标
由于时间关系&＃xff0c;我们只详细说明一下ROC/AUC检验
auc分数有两种计算方式&＃xff0c;第一种是根据目标变量y_true,预测分数/预测概率y_socres,通过roc_auc_score(y_true, y_scores)计算AUC
第二种方法是通过fpr,tpr&＃xff0c;通过auc(fpr,tpr)来计算AUC
excel 绘图ROC
ROC的前置条件是分数越高&＃xff0c;阳性率越高&＃xff0c;但风控模型中&＃xff0c;有的分数越低&＃xff0c;坏客户概率越高&＃xff0c;例如蜜罐分数&＃xff0c;因此ROC绘制出来是反的&＃xff0c;需要对阳性标签反转pos_label&＃61;0
由于分数越低&＃xff0c;坏客户概率越高&＃xff0c;画出来的ROC曲线是反转的&＃xff0c;需要纠正
AUC/ROC检验代码
# -*- coding: utf-8 -*-
"""
Created on Thu Apr 12 22:31:31 2018&＃64;author: 231469242&＃64;qq.com
"""
import numpy as np
from sklearn import metrics
from sklearn.metrics import roc_curve, auc,roc_auc_score ###计算roc和aucimport pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
import mglearn
import matplotlib.pyplot as pltcancer&＃61;load_breast_cancer()#mglearn.plots.plot_knn_classification(n_neighbors&＃61;3)
X_train,x_test,y_train,y_test&＃61;train_test_split(cancer.data,cancer.target,stratify&＃61;cancer.target,random_state&＃61;42)knn&＃61;KNeighborsClassifier()
knn.fit(X_train,y_train)
print("accuracy on the training subset:{:.3f}".format(knn.score(X_train,y_train)))
print("accuracy on the test subset:{:.3f}".format(knn.score(x_test,y_test)))#Auc验证&＃xff0c;数据采用测试集数据
#癌症的概率
proba_cancer&＃61;knn.predict_proba(x_test)
y_scores&＃61;pd.DataFrame(proba_cancer)[1]
y_scores&＃61;np.array(y_scores)
y_true&＃61;y_test
#auc分数
#auc分数有两种计算方式&＃xff0c;第一种是根据目标变量y_true,预测分数/预测概率y_socres,通过roc_auc_score(y_true, y_scores)计算AUC
AUC&＃61;roc_auc_score(y_true, y_scores)
print("AUC:",AUC)
#auc第二种方法是通过fpr,tpr&＃xff0c;通过auc(fpr,tpr)来计算AUC
fpr, tpr, thresholds &＃61; metrics.roc_curve(y_true, y_scores, pos_label&＃61;1)
AUC1 &＃61; auc(fpr,tpr) ###计算auc的值 #print("fpr:",fpr)
#print("tpr:",tpr)
#print("thresholds:",thresholds)
print("AUC1:",AUC1)if AUC >&＃61;0.7:print("good classifier")
if 0.7>AUC>0.6:print("not very good classifier")
if 0.6>&＃61;AUC>0.5:print("useless classifier")
if 0.5>&＃61;AUC:print("bad classifier,with sorting problems")#绘制ROC曲线
#画对角线
plt.plot([0, 1], [0, 1], &＃39;--&＃39;, color&＃61;(0.6, 0.6, 0.6), label&＃61;&＃39;Diagonal line&＃39;)
plt.plot(fpr,tpr,label&＃61;&＃39;ROC curve (area &＃61; %0.2f)&＃39; % AUC)
plt.title(&＃39;ROC curve&＃39;)
plt.legend(loc&＃61;"lower right")
扫二维码&＃xff0c;关注博主主页&＃xff0c;学习更多Python知识
https://m.study.163.com/user/1135726305.htm?utm_campaign&＃61;share&utm_medium&＃61;iphoneShare&utm_source&＃61;weixin&utm_u&＃61;1015941113
京公网安备 11010802041100号