Deep-learning-roadmap - All You Need to Know About Deep Learning - A kick-starter

Overview

Deep Learning - All You Need to Know

https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat https://badges.frapsoft.com/os/v2/open-source.png?v=103 https://img.shields.io/twitter/follow/machinemindset.svg?label=Follow&style=social

Sponsorship

To support maintaining and upgrading this project, please kindly consider Sponsoring the project developer.

Any level of support is a great contribution here ❤️

Download Free Python Machine Learning Book

Slack Group

Table of Contents

_img/mainpage/logo.gif

Introduction

The purpose of this project is to introduce a shortcut to developers and researcher for finding useful resources about Deep Learning.

Motivation

There are different motivations for this open source project.

What's the point of this open source project?

There are other repositories similar to this repository that are very comprehensive and useful and to be honest they made me ponder if there is a necessity for this repository!

The point of this repository is that the resources are being targeted. The organization of the resources is such that the user can easily find the things he/she is looking for. We divided the resources to a large number of categories that in the beginning one may have a headache!!! However, if someone knows what is being located, it is very easy to find the most related resources. Even if someone doesn't know what to look for, in the beginning, the general resources have been provided.

Papers

_img/mainpage/article.jpeg

This chapter is associated with the papers published in deep learning.

Models

Convolutional Networks
_img/mainpage/convolutional.png
  • Imagenet classification with deep convolutional neural networks : [Paper][Code]

    _img/mainpage/star_5.png
  • Convolutional Neural Networks for Sentence Classification : [Paper][Code]

    _img/mainpage/star_4.png
  • Large-scale Video Classification with Convolutional Neural Networks : [Paper][Project Page]

    _img/mainpage/star_4.png
  • Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks : [Paper]

    _img/mainpage/star_5.png
  • Deep convolutional neural networks for LVCSR : [Paper]

    _img/mainpage/star_3.png
  • Face recognition: a convolutional neural-network approach : [Paper]

    _img/mainpage/star_5.png
Recurrent Networks
  • An empirical exploration of recurrent network architectures : [Paper][Code]

    _img/mainpage/star_4.png
  • LSTM: A search space odyssey : [Paper][Code]

    _img/mainpage/star_3.png
  • On the difficulty of training recurrent neural networks : [Paper][Code]

    _img/mainpage/star_5.png
  • Learning to forget: Continual prediction with LSTM : [Paper]

    _img/mainpage/star_5.png
Autoencoders

_img/mainpage/Autoencoder_structure.png

  • Extracting and composing robust features with denoising autoencoders : [Paper]

    _img/mainpage/star_5.png
  • Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion : [Paper][Code]

    _img/mainpage/star_5.png
  • Adversarial Autoencoders : [Paper][Code]

    _img/mainpage/star_3.png
  • Autoencoders, Unsupervised Learning, and Deep Architectures : [Paper]

    _img/mainpage/star_4.png
  • Reducing the Dimensionality of Data with Neural Networks : [Paper][Code]

    _img/mainpage/star_5.png
Generative Models

_img/mainpage/generative.png

  • Exploiting generative models discriminative classifiers : [Paper]

    _img/mainpage/star_4.png
  • Semi-supervised Learning with Deep Generative Models : [Paper][Code]

    _img/mainpage/star_4.png
  • Generative Adversarial Nets : [Paper][Code]

    _img/mainpage/star_5.png
  • Generalized Denoising Auto-Encoders as Generative Models : [Paper]

    _img/mainpage/star_5.png
  • Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks : [Paper][Code]

    _img/mainpage/star_5.png
Probabilistic Models
  • Stochastic Backpropagation and Approximate Inference in Deep Generative Models : [Paper]

    _img/mainpage/star_4.png
  • Probabilistic models of cognition: exploring representations and inductive biases : [Paper]

    _img/mainpage/star_5.png
  • On deep generative models with applications to recognition : [Paper]

    _img/mainpage/star_5.png

Core

Optimization
  • Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift : [Paper]

    _img/mainpage/star_5.png
  • Dropout: A Simple Way to Prevent Neural Networks from Overfitting : [Paper]

    _img/mainpage/star_5.png
  • Training Very Deep Networks : [Paper]

    _img/mainpage/star_4.png
  • Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification : [Paper]

    _img/mainpage/star_5.png
  • Large Scale Distributed Deep Networks : [Paper]

    _img/mainpage/star_5.png
Representation Learning
  • Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks : [Paper][Code]

    _img/mainpage/star_5.png
  • Representation Learning: A Review and New Perspectives : [Paper]

    _img/mainpage/star_4.png
  • InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets : [Paper][Code]

    _img/mainpage/star_3.png
Understanding and Transfer Learning
  • Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks : [Paper]

    _img/mainpage/star_5.png
  • Distilling the Knowledge in a Neural Network : [Paper]

    _img/mainpage/star_4.png
  • DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition : [Paper][

    _img/mainpage/star_5.png
  • How transferable are features in deep neural networks? : [Paper][Code]

    _img/mainpage/star_5.png
Reinforcement Learning
  • Human-level control through deep reinforcement learning : [Paper][Code]

    _img/mainpage/star_5.png
  • Playing Atari with Deep Reinforcement Learning : [Paper][Code]

    _img/mainpage/star_3.png
  • Continuous control with deep reinforcement learning : [Paper][Code]

    _img/mainpage/star_4.png
  • Deep Reinforcement Learning with Double Q-Learning : [Paper][Code]

    _img/mainpage/star_3.png
  • Dueling Network Architectures for Deep Reinforcement Learning : [Paper][Code]

    _img/mainpage/star_3.png

Applications

Image Recognition
  • Deep Residual Learning for Image Recognition : [Paper][Code]

    _img/mainpage/star_5.png
  • Very Deep Convolutional Networks for Large-Scale Image Recognition : [Paper]

    _img/mainpage/star_5.png
  • Multi-column Deep Neural Networks for Image Classification : [Paper]

    _img/mainpage/star_4.png
  • DeepID3: Face Recognition with Very Deep Neural Networks : [Paper]

    _img/mainpage/star_4.png
  • Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps : [Paper][Code]

    _img/mainpage/star_3.png
  • Deep Image: Scaling up Image Recognition : [Paper]

    _img/mainpage/star_4.png
  • Long-Term Recurrent Convolutional Networks for Visual Recognition and Description : [Paper][Code]

    _img/mainpage/star_5.png
  • 3D Convolutional Neural Networks for Cross Audio-Visual Matching Recognition : [Paper][Code]

    _img/mainpage/star_4.png
Object Recognition
  • ImageNet Classification with Deep Convolutional Neural Networks : [Paper]

    _img/mainpage/star_5.png
  • Learning Deep Features for Scene Recognition using Places Database : [Paper]

    _img/mainpage/star_3.png
  • Scalable Object Detection using Deep Neural Networks : [Paper]

    _img/mainpage/star_4.png
  • Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks : [Paper][Code]

    _img/mainpage/star_4.png
  • OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks : [Paper][Code]

    _img/mainpage/star_5.png
  • CNN Features Off-the-Shelf: An Astounding Baseline for Recognition : [Paper]

    _img/mainpage/star_3.png
  • What is the best multi-stage architecture for object recognition? : [Paper]

    _img/mainpage/star_2.png
Action Recognition
  • Long-Term Recurrent Convolutional Networks for Visual Recognition and Description : [Paper]

    _img/mainpage/star_5.png
  • Learning Spatiotemporal Features With 3D Convolutional Networks : [Paper][Code]

    _img/mainpage/star_5.png
  • Describing Videos by Exploiting Temporal Structure : [Paper][Code]

    _img/mainpage/star_3.png
  • Convolutional Two-Stream Network Fusion for Video Action Recognition : [Paper][Code]

    _img/mainpage/star_4.png
  • Temporal segment networks: Towards good practices for deep action recognition : [Paper][Code]

    _img/mainpage/star_3.png
Caption Generation
  • Show, Attend and Tell: Neural Image Caption Generation with Visual Attention : [Paper][Code]

    _img/mainpage/star_5.png
  • Mind's Eye: A Recurrent Visual Representation for Image Caption Generation : [Paper]

    _img/mainpage/star_2.png
  • Generative Adversarial Text to Image Synthesis : [Paper][Code]

    _img/mainpage/star_3.png
  • Deep Visual-Semantic Al60ignments for Generating Image Descriptions : [Paper][Code]

    _img/mainpage/star_4.png
  • Show and Tell: A Neural Image Caption Generator : [Paper][Code]

    _img/mainpage/star_5.png
Natural Language Processing
  • Distributed Representations of Words and Phrases and their Compositionality : [Paper][Code]

    _img/mainpage/star_5.png
  • Efficient Estimation of Word Representations in Vector Space : [Paper][Code]

    _img/mainpage/star_4.png
  • Sequence to Sequence Learning with Neural Networks : [Paper][Code]

    _img/mainpage/star_5.png
  • Neural Machine Translation by Jointly Learning to Align and Translate : [Paper][Code]

    _img/mainpage/star_4.png
  • Get To The Point: Summarization with Pointer-Generator Networks : [Paper][Code]

    _img/mainpage/star_3.png
  • Attention Is All You Need : [Paper][Code]

    _img/mainpage/star_4.png
  • Convolutional Neural Networks for Sentence Classification : [Paper][Code]

    _img/mainpage/star_4.png
Speech Technology
  • Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups : [Paper]

    _img/mainpage/star_5.png
  • Towards End-to-End Speech Recognition with Recurrent Neural Networks : [Paper]

    _img/mainpage/star_3.png
  • Speech recognition with deep recurrent neural networks : [Paper]

    _img/mainpage/star_4.png
  • Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition : [Paper]

    _img/mainpage/star_3.png
  • Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin : [Paper][Code]

    _img/mainpage/star_4.png
  • A novel scheme for speaker recognition using a phonetically-aware deep neural network : [Paper]

    _img/mainpage/star_3.png
  • Text-Independent Speaker Verification Using 3D Convolutional Neural Networks : [Paper][Code]

    _img/mainpage/star_4.png

Datasets

Image

General
  • MNIST Handwritten digits: [Link]
Face
  • Face Recognition Technology (FERET) The goal of the FERET program was to develop automatic face recognition capabilities that could be employed to assist security, intelligence, and law enforcement personnel in the performance of their duties: [Link]
  • The CMU Pose, Illumination, and Expression (PIE) Database of Human Faces Between October and December 2000 we collected a database of 41,368 images of 68 people: [Link]
  • YouTube Faces DB The data set contains 3,425 videos of 1,595 different people. All the videos were downloaded from YouTube. An average of 2.15 videos are available for each subject: [Link]
  • Grammatical Facial Expressions Data Set Developed to assist the the automated analysis of facial expressions: [Link]
  • FaceScrub A Dataset With Over 100,000 Face Images of 530 People: [Link]
  • IMDB-WIKI 500k+ face images with age and gender labels: [Link]
  • FDDB Face Detection Data Set and Benchmark (FDDB): [Link]
Object Recognition
  • COCO Microsoft COCO: Common Objects in Context: [Link]
  • ImageNet The famous ImageNet dataset: [Link]
  • Open Images Dataset Open Images is a dataset of ~9 million images that have been annotated with image-level labels and object bounding boxes: [Link]
  • Caltech-256 Object Category Dataset A large dataset object classification: [Link]
  • Pascal VOC dataset A large dataset for classification tasks: [Link]
  • CIFAR 10 / CIFAR 100 The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes. CIFAR-100 is similar to CIFAR-10 but it has 100 classes containing 600 images each: [Link]
Action recognition
  • HMDB a large human motion database: [Link]
  • MHAD Berkeley Multimodal Human Action Database: [Link]
  • UCF101 - Action Recognition Data Set UCF101 is an action recognition data set of realistic action videos, collected from YouTube, having 101 action categories. This data set is an extension of UCF50 data set which has 50 action categories: [Link]
  • THUMOS Dataset A large dataset for action classification: [Link]
  • ActivityNet A Large-Scale Video Benchmark for Human Activity Understanding: [Link]

Text and Natural Language Processing

General
  • 1 Billion Word Language Model Benchmark: The purpose of the project is to make available a standard training and test setup for language modeling experiments: [Link]
  • Common Crawl: The Common Crawl corpus contains petabytes of data collected over the last 7 years. It contains raw web page data, extracted metadata and text extractions: [Link]
  • Yelp Open Dataset: A subset of Yelp's businesses, reviews, and user data for use in personal, educational, and academic purposes: [Link]
Text classification
  • 20 newsgroups The 20 Newsgroups data set is a collection of approximately 20,000 newsgroup documents, partitioned (nearly) evenly across 20 different newsgroups: [Link]
  • Broadcast News The 1996 Broadcast News Speech Corpus contains a total of 104 hours of broadcasts from ABC, CNN and CSPAN television networks and NPR and PRI radio networks with corresponding transcripts: [Link]
  • The wikitext long term dependency language modeling dataset: A collection of over 100 million tokens extracted from the set of verified Good and Featured articles on Wikipedia. : [Link]
Question Answering
  • Question Answering Corpus by Deep Mind and Oxford which is two new corpora of roughly a million news stories with associated queries from the CNN and Daily Mail websites. [Link]
  • Stanford Question Answering Dataset (SQuAD) consisting of questions posed by crowdworkers on a set of Wikipedia articles: [Link]
  • Amazon question/answer data contains Question and Answer data from Amazon, totaling around 1.4 million answered questions: [Link]
Sentiment Analysis
  • Multi-Domain Sentiment Dataset TThe Multi-Domain Sentiment Dataset contains product reviews taken from Amazon.com from many product types (domains): [Link]
  • Stanford Sentiment Treebank Dataset The Stanford Sentiment Treebank is the first corpus with fully labeled parse trees that allows for a complete analysis of the compositional effects of sentiment in language: [Link]
  • Large Movie Review Dataset: This is a dataset for binary sentiment classification: [Link]
Machine Translation
  • Aligned Hansards of the 36th Parliament of Canada dataset contains 1.3 million pairs of aligned text chunks: [Link]
  • Europarl: A Parallel Corpus for Statistical Machine Translation dataset extracted from the proceedings of the European Parliament: [Link]
Summarization
  • Legal Case Reports Data Set as a textual corpus of 4000 legal cases for automatic summarization and citation analysis.: [Link]

Speech Technology

  • TIMIT Acoustic-Phonetic Continuous Speech Corpus The TIMIT corpus of read speech is designed to provide speech data for acoustic-phonetic studies and for the development and evaluation of automatic speech recognition systems: [Link]
  • LibriSpeech LibriSpeech is a corpus of approximately 1000 hours of 16kHz read English speech, prepared by Vassil Panayotov with the assistance of Daniel Povey: [Link]
  • VoxCeleb A large scale audio-visual dataset: [Link]
  • NIST Speaker Recognition: [Link]

Courses

_img/mainpage/online.png

  • Machine Learning by Stanford on Coursera : [Link]
  • Neural Networks and Deep Learning Specialization by Coursera: [Link]
  • Intro to Deep Learning by Google: [Link]
  • Introduction to Deep Learning by CMU: [Link]
  • NVIDIA Deep Learning Institute by NVIDIA: [Link]
  • Convolutional Neural Networks for Visual Recognition by Stanford: [Link]
  • Deep Learning for Natural Language Processing by Stanford: [Link]
  • Deep Learning by fast.ai: [Link]
  • Course on Deep Learning for Visual Computing by IITKGP: [Link]

Books

_img/mainpage/books.jpg

  • Deep Learning by Ian Goodfellow: [Link]
  • Neural Networks and Deep Learning : [Link]
  • Deep Learning with Python: [Link]
  • Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems: [Link]

Blogs

_img/mainpage/Blogger_icon.png

  • Colah's blog: [Link]
  • Andrej Karpathy blog: [Link]
  • The Spectator Shakir's Machine Learning Blog: [Link]
  • WILDML: [Link]
  • Distill blog It is more like a journal than a blog because it has a peer review process and only accepted articles will be published on that.: [Link]
  • BAIR Berkeley Artificial Inteliigent Research: [Link]
  • Sebastian Ruder's blog: [Link]
  • inFERENCe: [Link]
  • i am trask A Machine Learning Craftsmanship Blog: [Link]

Tutorials

_img/mainpage/tutorial.png

  • Deep Learning Tutorials: [Link]
  • Deep Learning for NLP with Pytorch by Pytorch: [Link]
  • Deep Learning for Natural Language Processing: Tutorials with Jupyter Notebooks by Jon Krohn: [Link]

Frameworks

Contributing

For typos, unless significant changes, please do not create a pull request. Instead, declare them in issues or email the repository owner. Please note we have a code of conduct, please follow it in all your interactions with the project.

Pull Request Process

Please consider the following criterions in order to help us in a better way:

  1. The pull request is mainly expected to be a link suggestion.
  2. Please make sure your suggested resources are not obsolete or broken.
  3. Ensure any install or build dependencies are removed before the end of the layer when doing a build and creating a pull request.
  4. Add comments with details of changes to the interface, this includes new environment variables, exposed ports, useful file locations and container parameters.
  5. You may merge the Pull Request in once you have the sign-off of at least one other developer, or if you do not have permission to do that, you may request the owner to merge it for you if you believe all checks are passed.

Final Note

We are looking forward to your kind feedback. Please help us to improve this open source project and make our work better. For contribution, please create a pull request and we will investigate it promptly. Once again, we appreciate your kind feedback and support.

Owner
Instill AI
A company offering AI-based solutions to real-world applications.
Instill AI
Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT

CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT CheXbert is an accurate, automated dee

Stanford Machine Learning Group 51 Dec 08, 2022
This repository is for DSA and CP scripts for reference.

dsa-script-collections This Repo is the collection of DSA and CP scripts for reference. Contents Python Bubble Sort Insertion Sort Merge Sort Quick So

Aditya Kumar Pandey 9 Nov 22, 2022
Leveraging Social Influence based on Users Activity Centers for Point-of-Interest Recommendation

SUCP Leveraging Social Influence based on Users Activity Centers for Point-of-Interest Recommendation () Direct Friends (i.e., users who follow each o

Kosar 8 Nov 26, 2022
🔥 TensorFlow Code for technical report: "YOLOv3: An Incremental Improvement"

🆕 Are you looking for a new YOLOv3 implemented by TF2.0 ? If you hate the fucking tensorflow1.x very much, no worries! I have implemented a new YOLOv

3.6k Dec 26, 2022
Qlib is an AI-oriented quantitative investment platform

Qlib is an AI-oriented quantitative investment platform, which aims to realize the potential, empower the research, and create the value of AI technologies in quantitative investment.

Microsoft 10.1k Dec 30, 2022
Wider-Yolo Kütüphanesi ile Yüz Tespit Uygulamanı Yap

WIDER-YOLO : Yüz Tespit Uygulaması Yap Wider-Yolo Kütüphanesinin Kullanımı 1. Wider Face Veri Setini İndir Train Dataset Val Dataset Test Dataset Not:

Kadir Nar 6 Aug 22, 2022
Official PyTorch Code of GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection (CVPR 2021)

GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Mo

Abhinav Kumar 76 Jan 02, 2023
BED: A Real-Time Object Detection System for Edge Devices

BED: A Real-Time Object Detection System for Edge Devices About this project Thi

Data Analytics Lab at Texas A&M University 44 Nov 18, 2022
PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

Under construction... Attention in Attention Network for Image Super-Resolution (A2N) This repository is an PyTorch implementation of the paper "Atten

Haoyu Chen 71 Dec 30, 2022
Deep Learning & 3D Convolutional Neural Networks for Speaker Verification

TensorFlow implementation of 3D Convolutional Neural Networks for Speaker Verification - Official Project Page - Pytorch Implementation This repositor

Amirsina Torfi 753 Dec 17, 2022
Code for the paper Progressive Pose Attention for Person Image Generation in CVPR19 (Oral).

Pose-Transfer Code for the paper Progressive Pose Attention for Person Image Generation in CVPR19(Oral). The paper is available here. Video generation

Tengteng Huang 679 Jan 04, 2023
TuckER: Tensor Factorization for Knowledge Graph Completion

TuckER: Tensor Factorization for Knowledge Graph Completion This codebase contains PyTorch implementation of the paper: TuckER: Tensor Factorization f

Ivana Balazevic 296 Dec 06, 2022
Campsite Reservation Finder

yellowstone-camping UPDATE: yellowstone-camping is being expanded and renamed to camply. The updated tool now interfaces with the Recreation.gov API a

Justin Flannery 233 Jan 08, 2023
xitorch: differentiable scientific computing library

xitorch is a PyTorch-based library of differentiable functions and functionals that can be widely used in scientific computing applications as well as deep learning.

24 Apr 15, 2021
An excellent hash algorithm combining classical sponge structure and RNN.

SHA-RNN Recurrent Neural Network with Chaotic System for Hash Functions Anonymous Authors [摘要] 在这次作业中我们提出了一种新的 Hash Function —— SHA-RNN。其以海绵结构为基础,融合了混

Houde Qian 5 May 15, 2022
Official Tensorflow implementation of U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation (ICLR 2020)

U-GAT-IT — Official TensorFlow Implementation (ICLR 2020) : Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization fo

Junho Kim 6.2k Jan 04, 2023
Contrastive Learning of Structured World Models

Contrastive Learning of Structured World Models This repository contains the official PyTorch implementation of: Contrastive Learning of Structured Wo

Thomas Kipf 371 Jan 06, 2023
maximal update parametrization (µP)

Maximal Update Parametrization (μP) and Hyperparameter Transfer (μTransfer) Paper link | Blog link In Tensor Programs V: Tuning Large Neural Networks

Microsoft 694 Jan 03, 2023
Tianshou - An elegant PyTorch deep reinforcement learning library.

Tianshou (天授) is a reinforcement learning platform based on pure PyTorch. Unlike existing reinforcement learning libraries, which are mainly based on

Tsinghua Machine Learning Group 5.5k Jan 05, 2023
PyTorch implementation of Advantage async actor-critic Algorithms (A3C) in PyTorch

Advantage async actor-critic Algorithms (A3C) in PyTorch @inproceedings{mnih2016asynchronous, title={Asynchronous methods for deep reinforcement lea

LEI TAI 111 Dec 08, 2022