利用scikitlearn画ROC曲线实例
一个完整的数据挖掘模型,最后都要进行模型评估,对于二分类来说,AUC,ROC这两个指标用到最多,所以利用sklearn里面相应的函数进行模块搭建。
具体实现的代码可以参照下面博友的代码,评估svm的分类指标。注意里面的一些细节需要注意,一个是调用roc_curve方法时,指明目标标签,否则会报错。
具体是这个参数的设置pos_label,以前在unionbigdata实习时学到的。
重点是以下的代码需要根据实际改写:
mean_tpr=0.0 mean_fpr=np.linspace(0,1,100) all_tpr=[] y_target=np.r_[train_y,test_y] cv=StratifiedKFold(y_target,n_folds=6) #画ROC曲线和计算AUC fpr,tpr,thresholds=roc_curve(test_y,predict,pos_label=2)##指定正例标签,pos_label=###########在数之联的时候学到的,要制定正例 mean_tpr+=interp(mean_fpr,fpr,tpr)#对mean_tpr在mean_fpr处进行插值,通过scipy包调用interp()函数 mean_tpr[0]=0.0#初始处为0 roc_auc=auc(fpr,tpr) #画图,只需要plt.plot(fpr,tpr),变量roc_auc只是记录auc的值,通过auc()函数能计算出来 plt.plot(fpr,tpr,lw=1,label='ROC%s(area=%0.3f)'%(classifier,roc_auc))
然后是博友的参考代码:
#-*-coding:utf-8-*- """ CreatedonSunApr1908:57:132015 @author:shifeng """ print(__doc__) importnumpyasnp fromscipyimportinterp importmatplotlib.pyplotasplt fromsklearnimportsvm,datasets fromsklearn.metricsimportroc_curve,auc fromsklearn.cross_validationimportStratifiedKFold ############################################################################### #DataIOandgeneration,导入iris数据,做数据准备 #importsomedatatoplaywith iris=datasets.load_iris() X=iris.data y=iris.target X,y=X[y!=2],y[y!=2]#去掉了label为2,label只能二分,才可以。 n_samples,n_features=X.shape #Addnoisyfeatures random_state=np.random.RandomState(0) X=np.c_[X,random_state.randn(n_samples,200*n_features)] ############################################################################### #ClassificationandROCanalysis #分类,做ROC分析 #Runclassifierwithcross-validationandplotROCcurves #使用6折交叉验证,并且画ROC曲线 cv=StratifiedKFold(y,n_folds=6) classifier=svm.SVC(kernel='linear',probability=True, random_state=random_state)#注意这里,probability=True,需要,不然预测的时候会出现异常。另外rbf核效果更好些。 mean_tpr=0.0 mean_fpr=np.linspace(0,1,100) all_tpr=[] fori,(train,test)inenumerate(cv): #通过训练数据,使用svm线性核建立模型,并对测试集进行测试,求出预测得分 probas_=classifier.fit(X[train],y[train]).predict_proba(X[test]) #printset(y[train])#set([0,1])即label有两个类别 #printlen(X[train]),len(X[test])#训练集有84个,测试集有16个 #print"++",probas_#predict_proba()函数输出的是测试集在lael各类别上的置信度, ##在哪个类别上的置信度高,则分为哪类 #ComputeROCcurveandareathecurve #通过roc_curve()函数,求出fpr和tpr,以及阈值 fpr,tpr,thresholds=roc_curve(y[test],probas_[:,1]) mean_tpr+=interp(mean_fpr,fpr,tpr)#对mean_tpr在mean_fpr处进行插值,通过scipy包调用interp()函数 mean_tpr[0]=0.0#初始处为0 roc_auc=auc(fpr,tpr) #画图,只需要plt.plot(fpr,tpr),变量roc_auc只是记录auc的值,通过auc()函数能计算出来 plt.plot(fpr,tpr,lw=1,label='ROCfold%d(area=%0.2f)'%(i,roc_auc)) #画对角线 plt.plot([0,1],[0,1],'--',color=(0.6,0.6,0.6),label='Luck') mean_tpr/=len(cv)#在mean_fpr100个点,每个点处插值插值多次取平均 mean_tpr[-1]=1.0#坐标最后一个点为(1,1) mean_auc=auc(mean_fpr,mean_tpr)#计算平均AUC值 #画平均ROC曲线 #printmean_fpr,len(mean_fpr) #printmean_tpr plt.plot(mean_fpr,mean_tpr,'k--', label='MeanROC(area=%0.2f)'%mean_auc,lw=2) plt.xlim([-0.05,1.05]) plt.ylim([-0.05,1.05]) plt.xlabel('FalsePositiveRate') plt.ylabel('TruePositiveRate') plt.title('Receiveroperatingcharacteristicexample') plt.legend(loc="lowerright") plt.show()
补充知识:批量进行One-hot-encoder且进行特征字段拼接,并完成模型训练demo
importorg.apache.spark.ml.Pipeline importorg.apache.spark.ml.feature.{StringIndexer,OneHotEncoder} importorg.apache.spark.ml.feature.VectorAssembler importml.dmlc.xgboost4j.scala.spark.{XGBoostEstimator,XGBoostClassificationModel} importorg.apache.spark.ml.evaluation.BinaryClassificationEvaluator importorg.apache.spark.ml.tuning.{ParamGridBuilder,CrossValidator} importorg.apache.spark.ml.PipelineModel valdata=(spark.read.format("csv") .option("sep",",") .option("inferSchema","true") .option("header","true") .load("/Affairs.csv")) data.createOrReplaceTempView("res1") valaffairs="casewhenaffairs>0then1else0endasaffairs," valdf=(spark.sql("select"+affairs+ "gender,age,yearsmarried,children,religiousness,education,occupation,rating"+ "fromres1")) valcategoricals=df.dtypes.filter(_._2=="StringType")map(_._1) valindexers=categoricals.map( c=>newStringIndexer().setInputCol(c).setOutputCol(s"${c}_idx") ) valencoders=categoricals.map( c=>newOneHotEncoder().setInputCol(s"${c}_idx").setOutputCol(s"${c}_enc").setDropLast(false) ) valcolArray_enc=categoricals.map(x=>x+"_enc") valcolArray_numeric=df.dtypes.filter(_._2!="StringType")map(_._1) valfinal_colArray=(colArray_numeric++colArray_enc).filter(!_.contains("affairs")) valvectorAssembler=newVectorAssembler().setInputCols(final_colArray).setOutputCol("features") /* valpipeline=newPipeline().setStages(indexers++encoders++Array(vectorAssembler)) pipeline.fit(df).transform(df) */ /// //CreateanXGBoostClassifier valxgb=newXGBoostEstimator(Map("num_class"->2,"num_rounds"->5,"objective"->"binary:logistic","booster"->"gbtree")).setLabelCol("affairs").setFeaturesCol("features") //XGBoostparamatergrid valxgbParamGrid=(newParamGridBuilder() .addGrid(xgb.round,Array(10)) .addGrid(xgb.maxDepth,Array(10,20)) .addGrid(xgb.minChildWeight,Array(0.1)) .addGrid(xgb.gamma,Array(0.1)) .addGrid(xgb.subSample,Array(0.8)) .addGrid(xgb.colSampleByTree,Array(0.90)) .addGrid(xgb.alpha,Array(0.0)) .addGrid(xgb.lambda,Array(0.6)) .addGrid(xgb.scalePosWeight,Array(0.1)) .addGrid(xgb.eta,Array(0.4)) .addGrid(xgb.boosterType,Array("gbtree")) .addGrid(xgb.objective,Array("binary:logistic")) .build()) //CreatetheXGBoostpipeline valpipeline=newPipeline().setStages(indexers++encoders++Array(vectorAssembler,xgb)) //Setupthebinaryclassifierevaluator valevaluator=(newBinaryClassificationEvaluator() .setLabelCol("affairs") .setRawPredictionCol("prediction") .setMetricName("areaUnderROC")) //CreatetheCrossValidationpipeline,usingXGBoostastheestimator,the //BinaryClassificationevaluator,andxgbParamGridforhyperparameters valcv=(newCrossValidator() .setEstimator(pipeline) .setEvaluator(evaluator) .setEstimatorParamMaps(xgbParamGrid) .setNumFolds(3) .setSeed(0)) //Createthemodelbyfittingthetrainingdata valxgbModel=cv.fit(df) //Testthedatabyscoringthemodel valresults=xgbModel.transform(df) //PrintoutacopyoftheparametersusedbyXGBoost,attentionpipeline (xgbModel.bestModel.asInstanceOf[PipelineModel] .stages(5).asInstanceOf[XGBoostClassificationModel] .extractParamMap().toSeq.foreach(println)) results.select("affairs","prediction").show println("---ConfusionMatrix------") results.stat.crosstab("affairs","prediction").show() //Whatwastheoverallaccuracyofthemodel,usingAUC valauc=evaluator.evaluate(results) println("----AUC--------") println("auc="+auc)
以上这篇利用scikitlearn画ROC曲线实例就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持毛票票。