Manifoldron: Direct Space Partition via Manifold Discovery

This respository includes implementations on Manifoldron: Direct Space Partition via Manifold Discovery in which we propose a new type of machine learning models referred to as Manifoldron that directly derives decision boundaries from data and partitions the space via manifold structure discovery. Also, we systematically analyze the key characteristics of the Manifoldron including interpretability, manifold characterization capability, and its link to neural networks. The experimental results on 9 small and 11 large datasets demonstrate that the proposed Manifoldron performs competitively compared to the mainstream machine learning models.

Fig. 1 (a) Pipeline of the Manifoldron. (b) The Manifoldron key steps illustration.

Pre-requisites:

• Windows(runned on windows 10, can also run on Ubuntu with the required packages)
• Intell CPU(runned on 12 cores i7-8700 CPU @ 3.20GHZ)
• Python=3.7 (Anaconda), numpy=1.18.5, pandas=0.25.3, scikit-learn=0.22.1, scipy=1.3.2, matplotlib=3.1.1.

Folders

classification: this directory contains the implementations on classfication tasks;
regression: this directory contains implementations on simple regression tasks;
fancy_manifoldron: this directory includes implementations on 3D complex manifolds.

Dataset Preparation

All datasets are publicly available from python scikit-learn package, UCI machine learning repository, Kaggle, and Github: circle, glass, ionosphere, iris, moons, parkinsons, seeds, spirals, wine, banknote, breast, chess, drug, letRecog, magic04, nursery, satimage, semeion, tic-tac-toe, usps5. Most of the datasets can also directly obtain from our shared google drive. https://drive.google.com/drive/folders/14VHR8H7ucp0Loob1PS9yrgTtE9Jm0wsK?usp=sharing.
All datasets need to put under the 'classification/data/' folder to run the Manifoldron on specific data.

Running Experiments

Classification: as a demo, below shows how different versions of the Manifoldron run on tic-tac-toe data.

>> python manifoldron_base.py       # the base manifoldron
>> python manifoldron_bagging.py    # the manifoldron with feature bagging
>> python manifoldron_parallel.py   # the manifoldron with parallel computation

If you would like to run the Manifoldron on other representative classification datasets, go to 'classification/' folder and run cooresponding .py file
Regression: go to 'regression/' folder and run cooresponding .py file to run the manifoldron as regressor.

>> python regressor_function1.py       # the manifoldron regressor.

Experiment Results

Tab. 1 classification results on the Manifoldron and its counterparts.

Fig. 2 Complex simplices.

Tab. 2 Results on complex simplices.