Because this renders the results useless.
该提问来源于开源项目:automl/auto-sklearn
Here's a minimal example code to reproduce the behavior
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | import sklearn.model_selection import sklearn.datasets import sklearn.metrics import numpy as np import pandas as pd import autosklearn.classification import autosklearn.metrics from sklearn.metrics import confusion_matrix, recall_score def main(): X, y = sklearn.datasets.load_breast_cancer(return_X_y=True) X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, random_state=1) automl = autosklearn.classification.AutoSklearnClassifier( time_left_for_this_task=90, per_run_time_limit=30, delete_tmp_folder_after_terminate=False, resampling_strategy='cv', resampling_strategy_arguments={'folds': 5} ) scorer = autosklearn.metrics.make_scorer('recall', recall_score, pos_label=0) automl.fit(X_train.copy(), y_train.copy(), dataset_name='breast_cancer', metric=scorer) automl.refit(X_train.copy(), y_train.copy()) print(automl.show_models()) predictions = automl.predict(X_test) print("Accuracy score", sklearn.metrics.accuracy_score(y_test, predictions)) print(confusion_matrix(y_test, predictions)) if __name__ == '__main__': main() |
And here's the standard output (Note that I added a print line from the \autosklearn\ensemble_builder.py
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 | PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:15:58,481:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:15:58,494:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:00,498:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:02,502:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:04,507:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:06,510:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:08,514:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:10,518:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:12,522:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:14,527:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:16,531:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:18,535:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:20,541:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:22,545:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:24,549:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:26,552:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:28,556:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:30,561:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:32,564:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:34,568:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:36,571:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:38,583:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:40,587:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:42,593:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:44,598:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:46,604:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:48,607:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:50,611:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:52,617:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:54,620:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:56,624:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:16:58,630:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:17:00,633:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:17:02,637:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:17:04,642:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:17:06,646:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:17:08,649:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:17:10,653:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:17:12,657:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:17:14,661:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:17:16,665:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:17:18,669:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! PRINTING DUMMY_SCORE FROM ENSEMBLE_BUILDER: 1.0 [WARNING] [2018-12-01 23:17:20,672:EnsembleBuilder(1):breast_cancer] No models better than random - using Dummy Score! [(1.000000, MyDummyClassifier(configuration=1, init_params=None, random_state=None)), ] Accuracy score 0.38461538461538464 [[55 0] [88 0]] |