visualize_ML is a python package made to visualize some of the steps involved while dealing with a Machine Learning problem

Overview

visualize_ML

visualize_ML is a python package made to visualize some of the steps involved while dealing with a Machine Learning problem. It is build on libraries like matplotlib for visualization and sklean,scipy for statistical computations.

PyPI version

Table of content:

Requirement

  • python 2.x or python 3.x

Install

Install dependencies needed for matplotlib

sudo apt-get build-dep python-matplotlib

Install it using pip

pip install visualize_ML

Let's Code

While dealing with a Machine Learning problem some of the initial steps involved are data exploration,analysis followed by feature selection.Below are the modules for these tasks.

1) Data Exploration

At this stage, we explore variables one by one using Uni-variate Analysis which depends on whether the variable type is categorical or continuous .To deal with this we have the explore module.

>>> explore module

visualize_ML.explore.plot(data_input,categorical_name=[],drop=[],PLOT_COLUMNS_SIZE=4,bin_size=20,
bar_width=0.2,wspace=0.5,hspace=0.8)

Continuous Variables : In case of continous variables it plots the Histogram for every variable and gives descriptive statistics for them.

Categorical Variables : In case on categorical variables with 2 or more classes it plots the Bar chart for every variable and gives descriptive statistics for them.

Parameters Type Description
data_input Dataframe This is the input Dataframe with all data.(Right now the input can be only be a dataframe input.)
categorical_name list (default=[ ]) Names of all categorical variable columns with more than 2 classes, to distinguish them with the continuous variablesEmply list implies that there are no categorical features with more than 2 classes.
drop list default=[ ] Names of columns to be dropped.
PLOT_COLUMNS_SIZE int (default=4) Number of plots to display vertically in the display window.The row size is adjusted accordingly.
bin_size int (default="auto") Number of bins for the histogram displayed in the categorical vs categorical category.
wspace float32 (default = 0.5) Horizontal padding between subplot on the display window.
hspace float32 (default = 0.8) Vertical padding between subplot on the display window.

Code Snippet

/* The data set is taken from famous Titanic data(Kaggle)*/

import pandas as pd
from visualize_ML import explore
df = pd.read_csv("dataset/train.csv")
explore.plot(df,["Survived","Pclass","Sex","SibSp","Ticket","Embarked"],drop=["PassengerId","Name"])

Alt text

see the dataset

Note: While plotting all the rows with NaN values and columns with Character values are removed(except if values are True and False ),only numeric data is plotted.

2) Feature Selection

This is one of the challenging task to deal with for a ML task.Here we have to do Bi-variate Analysis to find out the relationship between two variables. Here, we look for association and disassociation between variables at a pre-defined significance level.

relation module helps in visualizing the analysis done on various combination of variables and see relation between them.

>>> relation module

visualize_ML.relation.plot(data_input,target_name="",categorical_name=[],drop=[],bin_size=10)

Continuous vs Continuous variables: To do the Bi-variate analysis scatter plots are made as their pattern indicates the relationship between variables. To indicates the strength of relationship amongst them we use Correlation between them.

The graph displays the correlation coefficient along with other information.

Correlation = Covariance(X,Y) / SQRT( Var(X)*Var(Y))
  • -1: perfect negative linear correlation
  • +1:perfect positive linear correlation and
  • 0: No correlation

Categorical vs Categorical variables: Stacked Column Charts are made to visualize the relation.Chi square test is used to derive the statistical significance of relationship between the variables. It returns probability for the computed chi-square distribution with the degree of freedom. For more information on Chi Test see this

Probability of 0: It indicates that both categorical variable are dependent

Probability of 1: It shows that both variables are independent.

The graph displays the p_value along with other information. If it is leass than 0.05 it states that the variables are dependent.

Categorical vs Continuous variables: To explore the relation between categorical and continuous variables,box plots re drawn at each level of categorical variables. If levels are small in number, it will not show the statistical significance. ANOVA test is used to derive the statistical significance of relationship between the variables.

The graph displays the p_value along with other information. If it is leass than 0.05 it states that the variables are dependent.

For more information on ANOVA test see this

Parameters Type Description
data_input Dataframe This is the input Dataframe with all data.(Right now the input can be only be a dataframe input.)
target_name String The name of the target column.
categorical_name list (default=[ ]) Names of all categorical variable columns with more than 2 classes, to distinguish them with the continuous variablesEmply list implies that there are no categorical features with more than 2 classes.
drop list default=[ ] Names of columns to be dropped.
PLOT_COLUMNS_SIZE int (default=4) Number of plots to display vertically in the display window.The row size is adjusted accordingly.
bin_size int (default="auto") Number of bins for the histogram displayed in the categorical vs categorical category.
wspace float32 (default = 0.5) Horizontal padding between subplot on the display window.
hspace float32 (default = 0.8) Vertical padding between subplot on the display window.

Code Snippet

/* The data set is taken from famous Titanic data(Kaggle)*/
import pandas as pd
from visualize_ML import relation
df = pd.read_csv("dataset/train.csv")
relation.plot(df,"Survived",["Survived","Pclass","Sex","SibSp","Ticket","Embarked"],drop=["PassengerId","Name"],bin_size=10)

Alt text

see the dataset

Note: While plotting all the rows with NaN values and columns with Non numeric values are removed only numeric data is plotted.Only categorical taget variable with string values are allowed.

Contribute

If you want to contribute and add new feature feel free to send Pull request here

This project is still under development so to report any bugs or request new features, head over to the Issues page

Tasks To Do

  • Make input compatible with other formats like Numpy.

  • Visualize best fit lines and decision boundaries for various models to make Parameter Tuning task easy.

    and many others!

Licence

Licensed under The MIT License (MIT).

Copyright

ayush1997(c) 2016

You might also like...
Import, visualize, and analyze SpiderFoot OSINT data in Neo4j, a graph database
Import, visualize, and analyze SpiderFoot OSINT data in Neo4j, a graph database

SpiderFoot Neo4j Tools Import, visualize, and analyze SpiderFoot OSINT data in Neo4j, a graph database Step 1: Installation NOTE: This installs the sf

Extract and visualize information from Gurobi log files
Extract and visualize information from Gurobi log files

GRBlogtools Extract information from Gurobi log files and generate pandas DataFrames or Excel worksheets for further processing. Also includes a wrapp

Extract data from ThousandEyes REST API and visualize it on your customized Grafana Dashboard.
Extract data from ThousandEyes REST API and visualize it on your customized Grafana Dashboard.

ThousandEyes Grafana Dashboard Extract data from the ThousandEyes REST API and visualize it on your customized Grafana Dashboard. Deploy Grafana, Infl

This is  a web application to visualize various famous technical indicators and stocks tickers from user
This is a web application to visualize various famous technical indicators and stocks tickers from user

Visualizing Technical Indicators Using Python and Plotly. Currently facing issues hosting the application on heroku. As soon as I am able to I'll like

Visualize the training curve from the *.csv file (tensorboard format).
Visualize the training curve from the *.csv file (tensorboard format).

Training-Curve-Vis Visualize the training curve from the *.csv file (tensorboard format). Feature Custom labels Curve smoothing Support for multiple c

Visualize your pandas data with one-line code
Visualize your pandas data with one-line code

PandasEcharts 简介 基于pandas和pyecharts的可视化工具 安装 pip 安装 $ pip install pandasecharts 源码安装 $ git clone https://github.com/gamersover/pandasecharts $ cd pand

 Flame Graphs visualize profiled code
Flame Graphs visualize profiled code

Flame Graphs visualize profiled code

Visualize data of Vietnam's regions with interactive maps.
Visualize data of Vietnam's regions with interactive maps.

Plotting Vietnam Development Map This is my personal project that I use plotly to analyse and visualize data of Vietnam's regions with interactive map

 Epagneul is a tool to visualize and investigate windows event logs
Epagneul is a tool to visualize and investigate windows event logs

epagneul Epagneul is a tool to visualize and investigate windows event logs. Dep

Comments
  • Can't get graphs to space right

    Can't get graphs to space right

    Not sure what is going on tried looking at the code.. I'm using Jupyter notebook if that is messing stuff up? data: state region age gender race marital_status ptype status-grp 0 IA 3 73 M W M Patient NaN 1 IL 2 57 M W S Patient NaN 2 WI 2 32 F W U Patient NaN 3 WI 2 54 F W U Patient NaN 4 IL 2 56 F W M Patient NaN 5 WI 2 31 F W S Patient

    input line: explore.plot(df2,['state','region','age','gender','race','marital_status','ptype','status-grp'],PLOT_COLUMNS_SIZE=2,bin_size=20, bar_width=0.2,wspace=.75,hspace=.75) result: vizml

    opened by dartdog 6
  • Just installed but it required and executed a downgrade of MPL

    Just installed but it required and executed a downgrade of MPL

    The PIP install downgraded MPL from 1.5.1 to 1.4.2 and also required the installation of "sudo apt-get install blt-dev" for freetype to build,, I had not previously run into that before? Any advice on how to preserve Matplotlib at 1.5.1 and of course MPL 2.0 is about to drop soon as well? The package looks quite useful with some nice ideas!

    opened by dartdog 2
Releases(0.2.2)
Owner
Ayush Singh
Machine Learning | Computer Vision | Data Science | Python
Ayush Singh
metedraw is a project mainly for data visualization projects of Atmospheric Science, Marine Science, Environmental Science or other majors

It is mainly for data visualization projects of Atmospheric Science, Marine Science, Environmental Science or other majors.

Nephele 11 Jul 05, 2022
Make scripted visualizations in blender

Scripted visualizations in blender The goal of this project is to script 3D scientific visualizations using blender. To achieve this, we aim to bring

Praneeth Namburi 10 Jun 01, 2022
HiPlot makes understanding high dimensional data easy

HiPlot - High dimensional Interactive Plotting HiPlot is a lightweight interactive visualization tool to help AI researchers discover correlations and

Facebook Research 2.4k Jan 04, 2023
100 Days of Code The Complete Python Pro Bootcamp for 2022

100-Day-With-Python 100 Days of Code - The Complete Python Pro Bootcamp for 2022. In this course, I spend with python language over 100 days, and I up

Rajdip Das 8 Jun 22, 2022
A little logger for machine learning research

Blinker Blinker provides a fast dispatching system that allows any number of interested parties to subscribe to events, or "signals". Signal receivers

Reinforcement Learning Working Group 27 Dec 03, 2022
Automatization of BoxPlot graph usin Python MatPlotLib and Excel

BoxPlotGraphAutomation Automatization of BoxPlot graph usin Python / Excel. This file is an automation of BoxPlot-Graph using python graph library mat

EricAugustin 1 Feb 07, 2022
Flow-based visual scripting for Python

A simple visual node editor for Python Ryven combines flow-based visual scripting with Python. It gives you absolute freedom for your nodes and a simp

Leon Thomm 3.1k Jan 06, 2023
This Crash Course will cover all you need to know to start using Plotly in your projects.

Plotly Crash Course This course was designed to help you get started using Plotly. If you ever felt like your data visualization skills could use an u

Fábio Neves 2 Aug 21, 2022
Comparing USD and GBP Exchange Rates

Currency Data Visualization Comparing USD and GBP Exchange Rates This is a bar graph comparing GBP and USD exchange rates. I chose blue for the UK bec

5 Oct 28, 2021
Visualise top-rated GitHub repositories in a barchart by keyword

This python script was written for simple purpose -- to visualise top-rated GitHub repositories in a barchart by keyword. Script generates html-page with barchart and information about repository own

Cur1iosity 2 Feb 07, 2022
Geocoding library for Python.

geopy geopy is a Python client for several popular geocoding web services. geopy makes it easy for Python developers to locate the coordinates of addr

geopy 3.8k Jan 02, 2023
Python library that makes it easy for data scientists to create charts.

Chartify Chartify is a Python library that makes it easy for data scientists to create charts. Why use Chartify? Consistent input data format: Spend l

Spotify 3.2k Jan 04, 2023
Python Data Validation for Humans™.

validators Python data validation for Humans. Python has all kinds of data validation tools, but every one of them seems to require defining a schema

Konsta Vesterinen 670 Jan 09, 2023
In-memory Graph Database and Knowledge Graph with Natural Language Interface, compatible with Pandas

CogniPy for Pandas - In-memory Graph Database and Knowledge Graph with Natural Language Interface Whats in the box Reasoning, exploration of RDF/OWL,

Cognitum Octopus 34 Dec 13, 2022
Small project demonstrating the use of Grafana and InfluxDB for monitoring the speed of an internet connection

Speedtest monitor for Grafana A small project that allows internet speed monitoring using Grafana, InfluxDB 2 and Speedtest. Demo Requirements Docker

Joshua Ghali 3 Aug 06, 2021
A visualization tool made in Pygame for various pathfinding algorithms.

Pathfinding-Visualizer 🚀 A visualization tool made in Pygame for various pathfinding algorithms. Pathfinding is closely related to the shortest path

Aysha sana 7 Jul 09, 2022
A data visualization curriculum of interactive notebooks.

A data visualization curriculum of interactive notebooks, using Vega-Lite and Altair. This repository contains a series of Python-based Jupyter notebooks.

UW Interactive Data Lab 1.2k Dec 30, 2022
Create animated and pretty Pandas Dataframe or Pandas Series

Rich DataFrame Create animated and pretty Pandas Dataframe or Pandas Series, as shown below: Installation pip install rich-dataframe Usage Minimal exa

Khuyen Tran 92 Dec 26, 2022
daily report of @arkinvest ETF activity + data collection

ark_invest daily weekday report of @arkinvest ETF activity + data collection This script was created to: Extract and save daily csv's from ARKInvest's

T D 27 Jan 02, 2023
Quickly and accurately render even the largest data.

Turn even the largest data into images, accurately Build Status Coverage Latest dev release Latest release Docs Support What is it? Datashader is a da

HoloViz 2.9k Dec 28, 2022