Easy and comprehensive assessment of predictive power, with support for neuroimaging features

Related tags

Deep Learningmachine-learningscikit-learncross-validationreporteasy-to-useneuroimagingpattern-recognitionnilearntractographystructural-imaginganatomical-mrifunctional-connectivitytract-based-statistics
Overview

docs/logo_neuropredict.png

https://landscape.io/github/raamana/neuropredict/master/landscape.svg?style=flat https://api.codacy.com/project/badge/Grade/501e560b8a424562a1b8f7cd2f3cadfe https://travis-ci.org/raamana/neuropredict.svg?branch=master

Documentation: https://raamana.github.io/neuropredict/

News

  • As of v0.6, neuropredict now supports regression applications i.e. predicting continuous targets (in addition to categorical classes), as well as allow you to regress out covariates / confounds within the nested-CV (following all the best practices). Utilizing this feature requires the input datasets be specified in the pyradigm data structures: code @ https://github.com/raamana/pyradigm, docs @ https://raamana.github.io/pyradigm/. Check the changelog below for more details.

Older news

  • neuropredict can handle missing data now (that are encoded with numpy.NaN). This is done respecting the cross-validation splits without any data leakage.

Overview

On a high level,

roleofneuropredict

On a more detailed level,

roleofneuropredict

Long term goals

neuropredict, the tool, is part of a broader initiative described below to develop easy, comprehensive and standardized predictive analysis:

longtermgoals

Citation

If neuropredict helped you in your research in one way or another, please consider citing one or more of the following, which were essential building blocks of neuropredict: - Pradeep Reddy Raamana. (2017, November 18). neuropredict: easy machine learning and standardized predictive analysis of biomarkers (Version 0.4.5). Zenodo. http://doi.org/10.5281/zenodo.1058993 - Raamana et al, (2017), Python class defining a machine learning dataset ensuring key-based correspondence and maintaining integrity, Journal of Open Source Software, 2(17), 382, doi:10.21105/joss.00382

Change Log - version 0.6

  • Major feature: Ability to predict continuous variables (regression)
  • Major feature: Ability to handle confounds (regress them out, augmenting etc)
  • Redesigned the internal structure for easier extensibility
  • New CVResults class for easier management of a wealth of outputs generated in the Classification and Regression workflows
  • API access is refreshed and easier

Change Log - version 0.5.2

  • Imputation of missing values
  • Additional classifiers such as XGBoost, Decision Trees
  • Better internal code structure
  • Lot more tests
  • More precise tests, as we vary number of classes wildly in test suites
  • several bug fixes and enhancements
  • More cmd line options such as --print_options from a previous run
Comments
  • ImportError: cannot import name 'MultiDatasetClassify'

    ImportError: cannot import name 'MultiDatasetClassify'

    Hi,

    When I try to run neuropredict on the command line using neuropredict -m ~/NeuroPredict/meta_data.csv -o ~/NeuroPredict/results/ --user_feature_paths ~/NeuroPredict/face/ ~/NeuroPredict/cnlg/ ~/NeuroPredict/glmn/ ~/NeuroPredict/cnsr/ ~/NeuroPredict/bimndir/ ~/NeuroPredict/bimnemo/ ~/NeuroPredict/msep/ -t 0.8 -n 20 -g 'light' I get the error that ImportError: cannot import name 'MultiDatasetClassify' from 'pyradigm.multiple' It appears that the multiple.py file that I've downloaded does not match the current version that is in your pyradigm repository. I installed it just now though, using pip3 install neuropredict. Do you know why I am getting this problem?

    Thanks

    opened by cpappas18 13
  • Add new classifier: need of probability output?

    Add new classifier: need of probability output?

    Hi @raamana ,

    I would like to add LinearSVC classifier based on liblinear implementation. Does the current implementation of neuropredict need that predictions are based on probability values? Because, LinearSVC doesn't allow prediction of probabilities.

    opened by mattvan83 12
  • Implement option to let user select number of features to select

    Implement option to let user select number of features to select

    To do:

    • [x] add an argument: --num_features
    • [x] describe the argument thoroughly in help text
    • [x] checks on the input types
    • [x] checks on the input ranges
    • [ ] checks on whether all the feature sets have the same number of features
    • [ ] make the new argument interface with all the other methods, if need be.
    • [ ] add option to pass on subset of feature names to visualize.feature_importance_map

    We can add/remove as we realize them.

    enhancement 
    opened by raamana 12
  • How to add multiple features?

    How to add multiple features?

    Hi Pradeep,

    I can see the file structure for adding a single feature is:

    subject1/features.txt subject2/features.txt subject3/features.txt subject4/features.txt

    Where the feature is a vector in each case. To clarify, if I wanted to add a second feature, would this be a second column in the features.txt file of each participant?

    Thanks,

    John

    question 
    opened by johnaeanderson 10
  • Classification blocked with multiple 1-dimensional features

    Classification blocked with multiple 1-dimensional features

    Hello,

    I have problems when using multiple 1-D features with the following command:

          neuropredict -m meta_data.csv \
      -d mask_WMpet_av45_early.ero1.25mm.csv mask_WMpet_av45_early.ero1.5mm.csv \
      -o outdir -t 0.8 -n 250 -k 'all' \
     --gs_level 'exhaustive' --classifier "LinearSVC"

    The CV trial were well launched for the first subgroup CN,MCI for both 1-D features but it seemed that at the end of CV trials the process got stuck at this stage:

    >

>  Python 3.7.3
> > SGE recognized, job set up with 40 slots.
> > Positive class inferred for AUC calculation: CN
> > Running neuropredict 0.5+34.g220af55.dirty
> > 
> > Requested features for analysis:
> > get_data_matrix from /netapp/vol2_agewell/pro/IMAP/imap_mvh/CAT12/pet/Analyses/ML/All/CN_vs_MCI+AD/av45_early/pons/start0_dur4/mask_WMpet_av45_early.ero1.25mm.csv
> > get_data_matrix from /netapp/vol2_agewell/pro/IMAP/imap_mvh/CAT12/pet/Analyses/ML/All/CN_vs_MCI+AD/av45_early/pons/start0_dur4/mask_WMpet_av45_early.ero1.5mm.csv
> > Ignoring imputation strategy chosen, as no missing data were found!
> > 
> > Data import is done.
> > 
> > 
> > Requested processing for the following subgroups:
> > DIS,CN
> > 
> > --------------------------------------------------------------------------------
> > Processing subgroup : DIS,CN (1/1)
> > --------------------------------------------------------------------------------
> > SGE recognized, job set up with 40 slots.
> > Training percentage      : 0.8
> > Number of CV repetitions : 250
> > Classifier chosen        : linearsvc
> > Feature selection chosen : variancethreshold
> > Level of grid search     : exhaustive
> > Number of processors     : 40
> > Saving the results to 
> >   /netapp/vol2_agewell/pro/IMAP/imap_mvh/CAT12/pet/Analyses/ML/All/CN_vs_MCI+AD/av45_early/pons/start0_dur4/linearsvc/binary_WMmasks/CN_DIS
> > 
> > -------------------------
> > All datasets contain:
> >  
> > 101 samples, 2 classes, 1 features
> > Class  CN : 71 samples
> > Class DIS : 30 samples
> > -------------------------
> > 
> > Estimated chance accuracy : 0.500
> > 
> > Different classes in the training set are stratified to match the smallest class!
> > Parallelizing the repetitions of CV with 40 processes ...
> > CV trial 52     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6525 	 weighted AUC: 0.7482
> > CV trial 78     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.8227
> > CV trial 50     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5993 	 weighted AUC: 0.6596
> > CV trial 72     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6082 	 weighted AUC: 0.6596
> > CV trial 46     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7021
> > CV trial 58     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8475
> > CV trial 38     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7518
> > CV trial 16     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7801
> > CV trial 12     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8794
> > CV trial 0      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6738 	 weighted AUC: 0.6950
> > CV trial 68     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7057 	 weighted AUC: 0.8369
> > CV trial 14     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8794
> > CV trial 22     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8333
> > CV trial 6      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7234
> > CV trial 48     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8298
> > CV trial 10     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7624
> > CV trial 64     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8191
> > CV trial 20     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8688
> > CV trial 4      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8688
> > CV trial 56     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.8262
> > CV trial 70     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8440
> > CV trial 74     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7553
> > CV trial 36     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8050
> > CV trial 2      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9043
> > CV trial 54     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8830
> > CV trial 24     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.9043
> > CV trial 76     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.9220
> > CV trial 42     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.8759
> > CV trial 26     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9078
> > CV trial 62     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8103
> > CV trial 66     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6418 	 weighted AUC: 0.7624
> > CV trial 60     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8511 	 weighted AUC: 0.8972
> > CV trial 72     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6294 	 weighted AUC: 0.6418
> > CV trial 40     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.9149
> > CV trial 52     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7340
> > CV trial 16     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7411
> > CV trial 46     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 	 weighted AUC: 0.6489
> > CV trial 38     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6507 	 weighted AUC: 0.7376
> > CV trial 78     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8014
> > CV trial 50     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6099 	 weighted AUC: 0.6454
> > CV trial 58     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8298
> > CV trial 6      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6879
> > CV trial 14     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8688
> > CV trial 36     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6844 	 weighted AUC: 0.8156
> > CV trial 18     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8014
> > CV trial 0      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5904 	 weighted AUC: 0.6454
> > CV trial 20     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8582
> > CV trial 68     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 	 weighted AUC: 0.7979
> > CV trial 4      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8511 	 weighted AUC: 0.8440
> > CV trial 74     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.7358
> > CV trial 56     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8050
> > CV trial 34     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5816 	 weighted AUC: 0.7518
> > CV trial 12     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8723
> > CV trial 66     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7376
> > CV trial 70     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6844 	 weighted AUC: 0.8227
> > CV trial 64     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8121
> > CV trial 24     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8794
> > CV trial 76     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 	 weighted AUC: 0.9007
> > CV trial 79     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5904 	 weighted AUC: 0.7021
> > CV trial 53     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7766
> > CV trial 2      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8936
> > CV trial 73     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9255
> > CV trial 54     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8759
> > CV trial 62     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8014
> > CV trial 17     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7624
> > CV trial 22     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8191
> > CV trial 39     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7695
> > CV trial 60     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9007
> > CV trial 51     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8404 	 weighted AUC: 0.8227
> > CV trial 26     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9007
> > CV trial 42     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 48     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8121
> > CV trial 59     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8014
> > CV trial 69     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7695
> > CV trial 47     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7695
> > CV trial 21     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.9220
> > CV trial 28     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8316 	 weighted AUC: 0.9007
> > CV trial 37     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8475
> > CV trial 57     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8582
> > CV trial 1      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8440
> > CV trial 40     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.9149
> > CV trial 30     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8582
> > CV trial 18     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7589
> > CV trial 65     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7837
> > CV trial 75     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8617
> > CV trial 25     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7943
> > CV trial 13     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8121
> > CV trial 79     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 	 weighted AUC: 0.6915
> > CV trial 61     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.7766
> > CV trial 71     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8156
> > CV trial 32     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8316 	 weighted AUC: 0.9291
> > CV trial 3      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8440
> > CV trial 53     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7411
> > CV trial 55     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8298
> > CV trial 15     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8582
> > CV trial 23     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7872
> > CV trial 34     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5284 	 weighted AUC: 0.7376
> > CV trial 59     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.7713
> > CV trial 43     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7766
> > CV trial 77     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7305
> > CV trial 49     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8528
> > CV trial 27     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8191
> > CV trial 73     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8936
> > CV trial 39     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5798 	 weighted AUC: 0.7305
> > CV trial 47     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7234
> > CV trial 69     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7199
> > CV trial 21     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 	 weighted AUC: 0.9149
> > CV trial 44     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8369
> > CV trial 65     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.7943
> > CV trial 41     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7092
> > CV trial 80     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7305
> > CV trial 51     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6738 	 weighted AUC: 0.8121
> > CV trial 57     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7128 	 weighted AUC: 0.8369
> > CV trial 75     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8156
> > CV trial 1      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8191
> > CV trial 63     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8582
> > CV trial 82     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 3      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8050
> > CV trial 23     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7270
> > CV trial 61     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7553
> > CV trial 55     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8014
> > CV trial 43     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.7270
> > CV trial 77     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7234
> > CV trial 25     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6613 	 weighted AUC: 0.7872
> > CV trial 84     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9362
> > CV trial 30     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8333
> > CV trial 19     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8298
> > CV trial 37     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8298
> > CV trial 88     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8121
> > CV trial 96     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6507 	 weighted AUC: 0.7270
> > CV trial 86     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8759
> > CV trial 13     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.7837
> > CV trial 41     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 	 weighted AUC: 0.6809
> > CV trial 5      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8617 	 weighted AUC: 0.9043
> > CV trial 102    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8333
> > CV trial 80     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7163
> > CV trial 92     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9220
> > CV trial 15     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8227
> > CV trial 104    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6436 	 weighted AUC: 0.7660
> > CV trial 35     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8333
> > CV trial 106    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5585 	 weighted AUC: 0.7163
> > CV trial 108    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6418 	 weighted AUC: 0.7163
> > CV trial 110    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6844
> > CV trial 94     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8617 	 weighted AUC: 0.9255
> > CV trial 98     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8652
> > CV trial 82     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8546
> > CV trial 100    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7890 	 weighted AUC: 0.8652
> > CV trial 49     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8440
> > CV trial 63     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 	 weighted AUC: 0.8475
> > CV trial 112    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8475
> > CV trial 90     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8511 	 weighted AUC: 0.8511
> > CV trial 118    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6525 	 weighted AUC: 0.7411
> > CV trial 27     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.7943
> > CV trial 44     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8014
> > CV trial 124    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8085
> > CV trial 84     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 	 weighted AUC: 0.9255
> > CV trial 110    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4645 	 weighted AUC: 0.6454
> > CV trial 31     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.9149
> > CV trial 96     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 	 weighted AUC: 0.7021
> > CV trial 108    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.6738
> > CV trial 126    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8582
> > CV trial 81     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7589
> > CV trial 116    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.9184
> > CV trial 104    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7163
> > CV trial 106    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5585 	 weighted AUC: 0.7163
> > CV trial 67     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7996 	 weighted AUC: 0.9007
> > CV trial 92     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9043
> > CV trial 88     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7695
> > CV trial 120    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.7695
> > CV trial 122    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8511
> > CV trial 130    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7376
> > CV trial 19     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8085
> > CV trial 102    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8262
> > CV trial 98     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8688
> > CV trial 112    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 	 weighted AUC: 0.8298
> > CV trial 114    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8652
> > CV trial 86     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 35     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8121
> > CV trial 100    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7057 	 weighted AUC: 0.8475
> > CV trial 71     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7996
> > CV trial 118    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 	 weighted AUC: 0.6986
> > CV trial 94     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8901
> > CV trial 83     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6738 	 weighted AUC: 0.8475
> > CV trial 128    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8369
> > CV trial 85     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7695
> > CV trial 17     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7447
> > CV trial 90     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8404 	 weighted AUC: 0.8511
> > CV trial 45     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8617 	 weighted AUC: 0.8191
> > CV trial 111    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6188 	 weighted AUC: 0.8333
> > CV trial 124    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7713
> > CV trial 97     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8582
> > CV trial 109    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8652
> > CV trial 105    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8369
> > CV trial 120    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.7340
> > CV trial 113    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.6702
> > CV trial 87     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.9362
> > CV trial 81     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7305
> > CV trial 136    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5904 	 weighted AUC: 0.8050
> > CV trial 132    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6738 	 weighted AUC: 0.7589
> > CV trial 116    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8901
> > CV trial 99     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8972
> > CV trial 119    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7021 	 weighted AUC: 0.8759
> > CV trial 103    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9113
> > CV trial 95     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6879
> > CV trial 134    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.8014
> > CV trial 130    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.6986
> > CV trial 93     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8121
> > CV trial 125    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7482
> > CV trial 101    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8652
> > CV trial 91     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7021 	 weighted AUC: 0.9255
> > CV trial 85     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7482
> > CV trial 128    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8262
> > CV trial 107    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7801
> > CV trial 111    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6188 	 weighted AUC: 0.7979
> > CV trial 5      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8865
> > CV trial 31     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9220
> > CV trial 138    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8901
> > CV trial 114    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8333
> > CV trial 121    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 113    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.6312
> > CV trial 97     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8546
> > CV trial 28     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8528 	 weighted AUC: 0.8617
> > CV trial 89     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6418 	 weighted AUC: 0.6702
> > CV trial 109    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8688
> > CV trial 122    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8422
> > CV trial 87     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9149
> > CV trial 105    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8191
> > CV trial 83     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8475
> > CV trial 95     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6826 	 weighted AUC: 0.6879
> > CV trial 140    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 7      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8830
> > CV trial 125    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.6844
> > CV trial 142    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8723
> > CV trial 136    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7908
> > CV trial 103    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8759
> > CV trial 119    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 	 weighted AUC: 0.8262
> > CV trial 99     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8759
> > CV trial 132    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6436 	 weighted AUC: 0.7234
> > CV trial 129    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7021
> > CV trial 91     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6915 	 weighted AUC: 0.9184
> > CV trial 126    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8493
> > CV trial 150    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.7376
> > CV trial 93     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.8014
> > CV trial 144    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.7872
> > CV trial 148    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7323
> > CV trial 146    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.7624
> > CV trial 107    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6294 	 weighted AUC: 0.7411
> > CV trial 152    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8723
> > CV trial 101    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6950 	 weighted AUC: 0.8617
> > CV trial 115    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7234 	 weighted AUC: 0.9007
> > CV trial 154    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8759
> > CV trial 134    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 	 weighted AUC: 0.8014
> > CV trial 29     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8582
> > CV trial 158    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7128 	 weighted AUC: 0.8121
> > CV trial 166    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.7872
> > CV trial 137    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7234
> > CV trial 168    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7057
> > CV trial 160    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8440
> > CV trial 123    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8121
> > CV trial 117    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8652
> > CV trial 164    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8369
> > CV trial 45     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6950 	 weighted AUC: 0.7979
> > CV trial 170    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7766
> > CV trial 150    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.7057
> > CV trial 7      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8723
> > CV trial 121    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8050
> > CV trial 174    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8298
> > CV trial 176    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8369
> > CV trial 162    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7660
> > CV trial 156    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7908
> > CV trial 127    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8014
> > CV trial 8      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8723 	 weighted AUC: 0.9326
> > CV trial 172    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6117 	 weighted AUC: 0.7589
> > CV trial 144    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 	 weighted AUC: 0.7589
> > CV trial 148    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5691 	 weighted AUC: 0.7021
> > CV trial 129    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4965 	 weighted AUC: 0.6809
> > CV trial 180    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8121
> > CV trial 140    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8191
> > CV trial 146    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.7624
> > CV trial 142    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8652
> > CV trial 154    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 115    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6915 	 weighted AUC: 0.8865
> > CV trial 29     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8227
> > CV trial 166    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 	 weighted AUC: 0.7553
> > CV trial 89     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.6348
> > CV trial 133    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8298
> > CV trial 168    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.6702
> > CV trial 184    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5780 	 weighted AUC: 0.7589
> > CV trial 137    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.6950
> > CV trial 158    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.7837
> > CV trial 152    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 	 weighted AUC: 0.8262
> > CV trial 151    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5479 	 weighted AUC: 0.7305
> > CV trial 131    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9255
> > CV trial 160    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8227
> > CV trial 123    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7766
> > CV trial 32     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.0709
> > CV trial 135    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.7713
> > CV trial 170    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7270
> > CV trial 188    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6454
> > CV trial 145    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7234 	 weighted AUC: 0.7553
> > CV trial 117    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8369
> > CV trial 156    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.7837
> > CV trial 127    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7890
> > CV trial 186    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7979
> > CV trial 178    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7624
> > CV trial 141    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8511
> > CV trial 147    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8759
> > CV trial 180    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.7979
> > CV trial 174    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8085
> > CV trial 149    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8582
> > CV trial 162    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6507 	 weighted AUC: 0.7163
> > CV trial 172    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5904 	 weighted AUC: 0.7270
> > CV trial 184    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5674 	 weighted AUC: 0.7340
> > CV trial 151    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7163
> > CV trial 176    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 	 weighted AUC: 0.7872
> > CV trial 194    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6525 	 weighted AUC: 0.7092
> > CV trial 188    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4645 	 weighted AUC: 0.5993
> > CV trial 169    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.8688
> > CV trial 155    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.7872
> > CV trial 171    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7766
> > CV trial 159    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6223 	 weighted AUC: 0.7482
> > CV trial 167    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8511
> > CV trial 153    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8652
> > CV trial 133    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7766
> > CV trial 143    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8617
> > CV trial 198    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8333
> > CV trial 161    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8475
> > CV trial 202    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6950
> > CV trial 182    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8333
> > CV trial 164    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6844 	 weighted AUC: 0.8191
> > CV trial 186    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 	 weighted AUC: 0.7411
> > CV trial 8      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8830 	 weighted AUC: 0.0674
> > CV trial 157    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.9574
> > CV trial 141    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 	 weighted AUC: 0.8617
> > CV trial 135    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7695
> > CV trial 145    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.7199
> > CV trial 190    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7553
> > CV trial 181    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.7979
> > CV trial 196    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7766
> > CV trial 33     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8103 	 weighted AUC: 0.8936
> > CV trial 192    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8422 	 weighted AUC: 0.8546
> > CV trial 138    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8901
> > CV trial 200    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.9468
> > CV trial 149    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 173    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 177    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8014
> > CV trial 178    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7340
> > CV trial 204    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8404
> > CV trial 206    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.7872
> > CV trial 163    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8085
> > CV trial 185    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8298
> > CV trial 171    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7801
> > CV trial 202    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 	 weighted AUC: 0.6950
> > CV trial 169    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8440
> > CV trial 167    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8369
> > CV trial 153    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8369
> > CV trial 131    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.9326
> > CV trial 159    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6950 	 weighted AUC: 0.7340
> > CV trial 194    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.6879
> > CV trial 187    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.7553
> > CV trial 155    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6738 	 weighted AUC: 0.7660
> > CV trial 212    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8546
> > CV trial 198    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6826 	 weighted AUC: 0.8191
> > CV trial 189    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8794
> > CV trial 165    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8652
> > CV trial 157    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.9397
> > CV trial 208    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 139    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7855
> > CV trial 190    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7305
> > CV trial 214    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5780 	 weighted AUC: 0.6879
> > CV trial 181    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7128 	 weighted AUC: 0.7660
> > CV trial 210    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8121
> > CV trial 143    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 	 weighted AUC: 0.8404
> > CV trial 9      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8511
> > CV trial 196    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6738 	 weighted AUC: 0.7518
> > CV trial 182    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8227
> > CV trial 147    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8582
> > CV trial 175    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.9113
> > CV trial 216    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7943
> > CV trial 173    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8440
> > CV trial 177    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.7979
> > CV trial 206    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.7518
> > CV trial 179    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8121
> > CV trial 204    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8440
> > CV trial 218    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8759
> > CV trial 185    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8014
> > CV trial 226    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6312 	 weighted AUC: 0.8227
> > CV trial 224    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 220    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.7801
> > CV trial 195    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7872
> > CV trial 200    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8511 	 weighted AUC: 0.9291
> > CV trial 163    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7908
> > CV trial 161    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8156
> > CV trial 230    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6401 	 weighted AUC: 0.8227
> > CV trial 199    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8050
> > CV trial 212    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6613 	 weighted AUC: 0.8262
> > CV trial 187    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7447
> > CV trial 232    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6844 	 weighted AUC: 0.7837
> > CV trial 207    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.4752 	 weighted AUC: 0.6064
> > CV trial 33     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.0975
> > CV trial 228    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8440
> > CV trial 203    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7766
> > CV trial 139    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 	 weighted AUC: 0.7624
> > CV trial 183    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8262
> > CV trial 208    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8404
> > CV trial 165    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8191
> > CV trial 234    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.9468
> > CV trial 214    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 	 weighted AUC: 0.6631
> > CV trial 216    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7589
> > CV trial 191    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8511
> > CV trial 240    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8440
> > CV trial 236    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8262
> > CV trial 210    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.7943
> > CV trial 242    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8582
> > CV trial 224    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 	 weighted AUC: 0.7943
> > CV trial 189    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8387
> > CV trial 220    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7163
> > CV trial 213    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6950 	 weighted AUC: 0.8156
> > CV trial 238    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6844 	 weighted AUC: 0.8085
> > CV trial 218    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8582
> > CV trial 230    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 	 weighted AUC: 0.7730
> > CV trial 226    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6312 	 weighted AUC: 0.8014
> > CV trial 179    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8014
> > CV trial 201    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6507 	 weighted AUC: 0.8298
> > CV trial 207    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4645 	 weighted AUC: 0.5816
> > CV trial 246    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.8121
> > CV trial 248    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6117 	 weighted AUC: 0.7163
> > CV trial 195    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7518
> > CV trial 199    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.7837
> > CV trial 183    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 	 weighted AUC: 0.8298
> > CV trial 244    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8298
> > CV trial 203    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.7270
> > CV trial 209    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8511
> > CV trial 205    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5691 	 weighted AUC: 0.7801
> > CV trial 217    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.9149
> > CV trial 240    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8191
> > CV trial 221    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.9291
> > CV trial 225    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8262
> > CV trial 232    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6436 	 weighted AUC: 0.7660
> > CV trial 211    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8262
> > CV trial 234    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.9326
> > CV trial 236    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7943
> > CV trial 246    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7730
> > CV trial 213    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8404 	 weighted AUC: 0.8050
> > CV trial 209    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8316
> > CV trial 219    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7305
> > CV trial 248    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5071 	 weighted AUC: 0.7092
> > CV trial 201    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6613 	 weighted AUC: 0.7979
> > CV trial 67     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7163 	 weighted AUC: 0.8617
> > CV trial 228    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8121
> > CV trial 9      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8262
> > CV trial 217    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8830
> > CV trial 215    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9255
> > CV trial 244    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8121
> > CV trial 175    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8688
> > CV trial 231    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7979
> > CV trial 227    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8369
> > CV trial 241    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8652
> > CV trial 211    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8014
> > CV trial 205    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7660
> > CV trial 235    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8156
> > CV trial 191    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8475
> > CV trial 249    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.6773
> > CV trial 233    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6206 	 weighted AUC: 0.7234
> > CV trial 225    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7908
> > CV trial 247    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7021 	 weighted AUC: 0.8865
> > CV trial 237    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8652
> > CV trial 242    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8333
> > CV trial 229    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7872
> > CV trial 219    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5904 	 weighted AUC: 0.7199
> > CV trial 221    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8901
> > CV trial 238    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7943
> > CV trial 245    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8528
> > CV trial 222    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8794
> > CV trial 215    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.9078
> > CV trial 249    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.6312
> > CV trial 231    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7447
> > CV trial 235    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7943
> > CV trial 241    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8582
> > CV trial 247    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6915 	 weighted AUC: 0.8511
> > CV trial 229    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7624
> > CV trial 233    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 	 weighted AUC: 0.6915
> > CV trial 237    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8546
> > CV trial 227    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8262
> > CV trial 239    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8901
> > CV trial 245    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8404 	 weighted AUC: 0.8121
> > CV trial 243    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7996 	 weighted AUC: 0.8688
> > CV trial 222    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8723
> > CV trial 239    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8901
> > CV trial 192    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8316 	 weighted AUC: 0.8298
> > CV trial 197    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9539
> > CV trial 197    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9433
> > CV trial 223    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5709 	 weighted AUC: 0.7908
> > CV trial 243    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7890 	 weighted AUC: 0.8617
> > CV trial 193    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9255
> > CV trial 223    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4982 	 weighted AUC: 0.7943
> > CV trial 193    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9397
>

    If I launch the previous command line with only one 1-D feature as follows, everything is going well:

          neuropredict -m meta_data.csv \
      -d mask_WMpet_av45_early.ero1.25mm.csv \
      -o outdir -t 0.8 -n 250 -k 'all' \
      --gs_level 'exhaustive' --classifier "LinearSVC"

    Do you have any idea?

    Best,

    Matthieu

    Sources.zip

    opened by mattvan83 6
  • IO error, unable to load features

    IO error, unable to load features

    Hi Pradeep,

    I finally got neuropredict installed with the dependencies using python 2.7.13! I'm having some issues with running the command, see the error message below,

    Cheers,

    John

    (py2713) Johns-MacBook-Pro-2:KNN John$ neuropredict -m /Users/John/Desktop/KNN/METADATAFILE.csv -o /Users/John/Desktop/KNN/features/results -u /Users/John/Desktop/KNN/features

    Requested features for analysis: get_dir_of_dirs from /Users/John/Desktop/KNN/features Traceback (most recent call last): File "/Users/John/anaconda3/envs/py2713/lib/python2.7/site-packages/neuropredict/neuropredict.py", line 333, in getfeatures data, feat_names = getmethod(featdir, subjid) File "/Users/John/anaconda3/envs/py2713/lib/python2.7/site-packages/neuropredict/neuropredict.py", line 274, in get_dir_of_dirs raise IOError('Unable to load features from \n{}'.format(featfile))

    opened by johnaeanderson 5
  • Unable to launch multi-class classifier

    Unable to launch multi-class classifier

    Hi Pradeep,

    When trying to launch multi-class classifier (whatever the classifier chosen) on a 3-class (CN, MCI, AD) problem, I got the following problem:

    Traceback (most recent call last):
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/bin/neuropredict", line 8, in <module>
    sys.exit(main())
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/lib/python3.7/site-packages/neuropredict/__main__.py", line 11, in main
    run_workflow.cli()
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/lib/python3.7/site-packages/neuropredict/run_workflow.py", line 969, in cli
    grid_search_level, classifier, feat_select_method = parse_args()
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/lib/python3.7/site-packages/neuropredict/run_workflow.py", line 525, in parse_args
    class_set, subgroups, positive_class = validate_class_set(classes, user_args.sub_groups, user_args.positive_class)
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/lib/python3.7/site-packages/neuropredict/run_workflow.py", line 659, in validate_class_set
    ''.format(comb))
ValueError: Subgroup all does not contain 2 unique classes! Each subgroup must contain atleast two classes for classification experiments.

    I don't understand since I have well defined 3-class in the metadata.csv file and used the following command line:

      neuropredict -m meta_data.csv \
  -d metaROI.csv metaROI_split.csv HCP_parcellation.csv \
  -t 0.8 -n 250 -k 'all' --sub_groups 'all' --gs_level 'exhaustive' \
  --feat_select_method 'variancethreshold' --classifier ${classifier} --make_vis ./visu \
  --num_procs ${NCPU} --print_options ./options

    Could you help me? Best, Matthieu

    opened by mattvan83 4
  • Regression output visualizations: residual plots

    Regression output visualizations: residual plots

    A good output in the regression version would be residual plots

    Good example is from yellowbrick: http://www.scikit-yb.org/en/latest/api/regressor/residuals.html

    opened by raamana 2
  • CLI flag to print chosen options from an existing run

    CLI flag to print chosen options from an existing run

    normal run would print only the most important options such as which classifier, cross-validation details etc, whereas verbose option would print which subjects and more info into feature sets etc.

    opened by raamana 1
  • Add documentation to help interpret the results and visualizations

    Add documentation to help interpret the results and visualizations

    Have a separate markdown file

    • [ ] describe each output figure comprehensively
    • [ ] describe each output CSV file
    • [ ] And what else they can do with results
    enhancement help wanted 
    opened by raamana 1
  • feature request: covariates and ability to regress them

    feature request: covariates and ability to regress them

    few ideas:

    CSV input

    • [ ] have column names, specify which columns are class, id, covariates and feature anmes!
    • [ ] get the class ID from a specified colum (e.g. first, or specified)
    • [ ] get the subject ID from a specified column (or the second or specified)
    sbuject_id,class,feature1,feature2,age,sex
acbde_01,healthy,1.2,2.3,3.4,74,female
fghijk_01,healthy,1.2,2.3,3.4,76,male
...
lmnop_01,alzheimer,1.2,2.3,3.4,70,female
...
qrstu_01,depression,1.2,2.3,3.4,71,male

    Directory of directories input

    • [ ] covariates.txt in addition to features.txt within each subject folder
    enhancement Hackathon project 
    opened by raamana 1
  • Enabling plug-in user-chosen models or hyper-parameters

    Enabling plug-in user-chosen models or hyper-parameters

    Sometimes some advanced users would like to choose their model or hyper-parameters for many good reasons including model not existing in sklearn etc or needing deep control over them etc, then having the API or CLI allowing users to do it would be great!

    opened by raamana 0
  • Differences betw. frequency misclassified vs frequently correctly classified

    Differences betw. frequency misclassified vs frequently correctly classified

    besides the misclassification frequency plots (which can be a helpful diagnostic), it may be useful to offer an option to enable the user to compare and study the differences characteristics between frequency misclassified vs. frequently correctly classified..

    Characteristics could be the input features themselves, or the other associated meta data and attributes (e.g. provided as part of pyradigm)

    examples include: those frequently misclassified were mostly the older patients , and those frequently correctly classified has "property X" etc.

    opened by raamana 0
  • Add docs for all API, esp. CVResults

    Add docs for all API, esp. CVResults

    CVResults might be individually even more useful than others in the API. So document it thoroughly align with the rest with a tutorial or two for use by dev and reuse by the replicator or meta-analytic researcher.

    Hackathon project 
    opened by raamana 0
  • update all docs and graphics to indicate users can try different models also

    update all docs and graphics to indicate users can try different models also

    not just compare different feature sets with a fixed builtin model, but users can also input a model of their own choice. it does not limit exploration of new models or pipelines - they can use implementation of best practices while evaluating the such new models on features of their choice

    opened by raamana 1
  • Attribute-contrained performance estimates

    Attribute-contrained performance estimates

    Often, performance is estimated in aggregate on the entire test-set (in a split), regardless of their covariate- or attribute characteristics.. hence looking into performance estimates segregated by certain characteristics of covariates would reveal additional insights - such as difference in AUC in men vs women, or across different age bins, or between sites etc.

    Hackathon project 
    opened by raamana 2
Releases(0.6)

  • 0.6(Jun 8, 2020)

    • Major feature: Ability to predict continuous variables (regression)
    • Major feature: Ability to handle confounds (regress them out, augmenting etc)
    • Redesigned the internal structure for easier extensibility
    • New CVResults class for easier management of a wealth of outputs generated in the Classification and Regression workflows
    • API access is refreshed and easier
    Source code(tar.gz)
    Source code(zip)

  • 0.5(Jan 31, 2019)

    • Good news: neuropredict can handle missing data now (that are encoded with numpy.NaN). This is done respecting the cross-validation splits without any data leakage.
    • Few other useful utilities
    Source code(tar.gz)
    Source code(zip)

  • 0.4.5(Nov 18, 2017)

    • new classifier : SVM
    • new flag to choose a feature selection method
    • user chosen options now saved to disk, to better handle complex interactions between options
    • code clean up and faster tests
    Source code(tar.gz)
    Source code(zip)

  • 0.4.1(Nov 6, 2017)

    • Parallelizing the main the CV loop, leading to great reduction in total time for report generation!
    • More options, including choice of different classifiers (Random Forest and Extra Trees classifiers)
    • Support for dataset in Weka's ARFF format
    • Better visualizations (handling small/nan values in feature importance, layouts and design)
    • auto versioning!
    • Ability to read meta data from pyradigms or ARFF files, without having to specify that separately.
    • Dropping support for Python 2.7 :(
    Source code(tar.gz)
    Source code(zip)

  • v0.3-DOI(Sep 26, 2017)

Owner
Pradeep Reddy Raamana
Neuroscientist trying to bridge the gap between clinic & computer science. Interests: Machine learning, Neuroimaging, Brain disorders, Informatics, Open science
Pradeep Reddy Raamana
GitHub Repository https://raamana.github.io/neuropredict/
Code for Overinterpretation paper Overinterpretation reveals image classification model pathologies

Overinterpretation This repository contains the code for the paper: Overinterpretation reveals image classification model pathologies Authors: Brandon

Gifford Lab, MIT CSAIL 17 Dec 10, 2022
The official PyTorch implementation of recent paper - SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training

This repository is the official PyTorch implementation of SAINT. Find the paper on arxiv SAINT: Improved Neural Networks for Tabular Data via Row Atte

Gowthami Somepalli 284 Dec 21, 2022
DeepCAD: A Deep Generative Network for Computer-Aided Design Models

DeepCAD This repository provides source code for our paper: DeepCAD: A Deep Generative Network for Computer-Aided Design Models Rundi Wu, Chang Xiao,

Rundi Wu 85 Dec 31, 2022
Spatial Intention Maps for Multi-Agent Mobile Manipulation (ICRA 2021)

spatial-intention-maps This code release accompanies the following paper: Spatial Intention Maps for Multi-Agent Mobile Manipulation Jimmy Wu, Xingyua

Jimmy Wu 70 Jan 02, 2023
Official repository of "Investigating Tradeoffs in Real-World Video Super-Resolution"

RealBasicVSR [Paper] This is the official repository of "Investigating Tradeoffs in Real-World Video Super-Resolution, arXiv". This repository contain

Kelvin C.K. Chan 566 Dec 28, 2022
CPPE - 5 (Medical Personal Protective Equipment) is a new challenging object detection dataset

CPPE - 5 CPPE - 5 (Medical Personal Protective Equipment) is a new challenging dataset with the goal to allow the study of subordinate categorization

Rishit Dagli 53 Dec 17, 2022
Official Chainer implementation of GP-GAN: Towards Realistic High-Resolution Image Blending (ACMMM 2019, oral)

GP-GAN: Towards Realistic High-Resolution Image Blending (ACMMM 2019, oral) [Project] [Paper] [Demo] [Related Work: A2RL (for Auto Image Cropping)] [C

Wu Huikai 402 Dec 27, 2022
CLDF dataset derived from Robbeets et al.'s "Triangulation Supports Agricultural Spread" from 2021

CLDF dataset derived from Robbeets et al.'s "Triangulation Supports Agricultural Spread" from 2021 How to cite If you use these data please cite the o

Digital Linguistics 2 Dec 20, 2021
Synthetic structured data generators

Join us on What is Synthetic Data? Synthetic data is artificially generated data that is not collected from real world events. It replicates the stati

YData 850 Jan 07, 2023
A library for answering questions using data you cannot see

A library for computing on data you do not own and cannot see PySyft is a Python library for secure and private Deep Learning. PySyft decouples privat

OpenMined 8.5k Jan 02, 2023
A transformer-based method for Healthcare Image Captioning in Vietnamese

vieCap4H Challenge 2021: A transformer-based method for Healthcare Image Captioning in Vietnamese This repo GitHub contains our solution for vieCap4H

Doanh B C 4 May 05, 2022
Taichi Course Homework Template

太极图形课S1-标题部分 这个作业未来或将是你的开源项目，标题的内容可以来自作业中的核心关键词，让读者一眼看出你所完成的工作/做出的好玩demo 如果暂时未想好，起名时可以参考“太极图形课S1-xxx作业” 如下是作业（项目）展开说明的方法，可以帮大家理清思路，并且也对读者非常友好，请小伙伴们多多参

TaichiCourse 30 Nov 19, 2022
PyTorch implementation of the paper: Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Features

Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Features Estimate the noise transition matrix with f-mutual information. This co

<a href=[email protected]"> 1 Jun 05, 2022
Pytorch Implementation of Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations

NANSY: Unofficial Pytorch Implementation of Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations Notice Papers' D

Dongho Choi 최동호 104 Dec 23, 2022
Tensorflow 2 implementation of the paper: Learning and Evaluating Representations for Deep One-class Classification published at ICLR 2021

Deep Representation One-class Classification (DROC). This is not an officially supported Google product. Tensorflow 2 implementation of the paper: Lea

Google Research 137 Dec 23, 2022
Pipeline for employing a Lightweight deep learning models for LOW-power systems

PL-LOW A high-performance deep learning model lightweight pipeline that gradually lightens deep neural networks in order to utilize high-performance d

POSTECH Data Intelligence Lab 9 Aug 13, 2022
Experiments for Fake News explainability project

fake-news-explainability Experiments for fake news explainability project This repository only contains the notebooks used to train the models and eva

Lorenzo Flores (Lj) 1 Dec 03, 2022
Numenta Platform for Intelligent Computing is an implementation of Hierarchical Temporal Memory (HTM), a theory of intelligence based strictly on the neuroscience of the neocortex.

NuPIC Numenta Platform for Intelligent Computing The Numenta Platform for Intelligent Computing (NuPIC) is a machine intelligence platform that implem

Numenta 6.3k Dec 30, 2022
Get started with Machine Learning with Python - An introduction with Python programming examples

Machine Learning With Python Get started with Machine Learning with Python An engaging introduction to Machine Learning with Python TL;DR Download all

Learn Python with Rune 130 Jan 02, 2023
Automatically erase objects in the video, such as logo, text, etc.

Video-Auto-Wipe Read English Introduction：Here   本人不定期的基于生成技术制作一些好玩有趣的算法模型，这次带来的作品是“视频擦除”方向的应用模型，它实现的功能是自动感知到视频中我们不想看见的部分（譬如广告、水印、字幕、图标等等）然后进行擦除。由于图标擦

seeprettyface.com 141 Dec 26, 2022