Create and implement a deep learning library from scratch.

In this project, we create and implement a deep learning library from scratch.

Table of Contents

• Deep Leaning Library
  • Table of Contents
  • About The Project
    • Aim
    • Tech Stack
    • File Structure
  • Approach
  • Theory
    • Loss Function:
    • Cost Function :
    • Gradient Descent : -
  • Getting Started
    • Prerequisites
  • Results
    • Result
  • Future Work
  • Troubleshooting
  • Contributors
  • Acknowledgements
  • Resources
  • License *Results *Demo Video

• Future Work
• Troubleshooting
• Contributors
• Acknowledgements
• Resources
• License

About The Project

Deep learning can be considered as a subset of machine learning. It is a field that is based on learning and improving on its own by examining computer algorithms. Deep learning works with artificial neural networks consisting of many layers. This project, which is creating a Deep Learning Library from scratch, can be further implemented in various kinds of projects that involve Deep Learning. Which include, but are not limited to applications in Image, Natural Language and Speech processing, among others.

Aim

To implement a deep learning library from scratch.

Tech Stack

Technologies used in the project:

• Python and numpy, pandas, matplotlib
• Google Colab

File Structure

.
├── code
|   └── main.py                                   #contains the main code for the library
├── resources                                     #Notes 
|   ├── ImprovingDeepNeuralNetworks
|   |   ├── images
|   |   |   ├── BatchvsMiniBatch.png
|   |   |   ├── Bias.png
|   |   |   └── EWG.png
|   |   └── notes.md
|   ├── Course1.md                               
|   ├── accuracy.jpg
|   ├── error.jpg
|   └── grad_des_graph.jpg
├── LICENSE.txt
├── ProjectReport.pdf                            #Project Report
└── README.md                                    #Readme

Approach

The approach of the project is to basically create a deep learning library, as stated before. The aim of the project was to implement various deep learning algorithms, in order to drive a deep neural network and hence,create a deep learning library, which is modular,and driven on user input so that it can be applied for various deep learning processes, and to train and test it against a model.

Theory

A neural network is a network or circuit of neurons, or in a modern sense, an artificial neural network, composed of artificial neurons or nodes.

There are different types of Neural Networks

• Standard Neural Networks
• Convolutional Neural Networks
• Recurring Neural Networks

Loss Function:

Loss function is defined so as to see how good the output ŷ is compared to output label y.

Cost Function :

Cost Function quantifies the error between predicted values and expected values.

Gradient Descent : -

Gradient descent is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function.

Getting Started

Prerequisites

• Object oriented programming in Python

• Linear Algebra

• Basic knowledge of Neural Networks

• Python 3.6 and above

  You can visit the Python Download Guide for the installation steps.

• Install numpy next

pip install numpy

Installation

1. Clone the repo

git clone [email protected]:https://github.com/Ris-Bali/ARA.git

Results

Training

We trained a model on the iris dataset using ARA here's the video for the same -

As you may have observed we achieved an accuracy of nearly 100% while training the model.

Result

Results obtained during training: (where Y-axis represents the value of the cost function and X axis represents the number of iterations) (where Y-axis represents the accuracy of the prediction wrt the labels and X-axis represents the number of iterations)

Future Work

• Short term
  • Adding class for normalization and regularization
• Near Future
  • Addition of support for linear regression
  • Addition of classes for LSTM and GRU blocks
• Future goal
  • Addition of algorithms to support CNN models.
  • Addition of more Machine Learning algorithms
  • Include algorithms to facilitate Image Recognition, Machine Translation and Natural Language Processing

Troubleshooting

• Numpy library not working so we shifted workspace to colab

Contributors

• Rishabh Bali
• Aditya Mhatre
• Aayush Mehta

Acknowledgements

• SRA VJTI Eklavya 2021
• Mentors:
  • Kush Kothari
  • Aman Chhaparia

Resources

• Coursera course by Andrew NG on Deep Learning and Neural Networks
• Coursera course by Andrew NG on Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization
• On Gradient Descent Optimization
• Essence of Linear Algebra by 3Blue1Brown
• Basics of Neural Networks by 3Blue1Brown
• Basics of Implementing Deep Learning Libraries
• Deep Learning
• Deep Learning Libraries
• Iris Dataset

License

Describe your License for your project.