A MNIST-like fashion product database. Benchmark

Overview

Fashion-MNIST

GitHub stars Gitter Readme-CN Readme-JA License: MIT Year-In-Review

Table of Contents

Fashion-MNIST is a dataset of Zalando's article images—consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes. We intend Fashion-MNIST to serve as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms. It shares the same image size and structure of training and testing splits.

Here's an example of how the data looks (each class takes three-rows):

Why we made Fashion-MNIST

The original MNIST dataset contains a lot of handwritten digits. Members of the AI/ML/Data Science community love this dataset and use it as a benchmark to validate their algorithms. In fact, MNIST is often the first dataset researchers try. "If it doesn't work on MNIST, it won't work at all", they said. "Well, if it does work on MNIST, it may still fail on others."

To Serious Machine Learning Researchers

Seriously, we are talking about replacing MNIST. Here are some good reasons:

Get the Data

Many ML libraries already include Fashion-MNIST data/API, give it a try!

You can use direct links to download the dataset. The data is stored in the same format as the original MNIST data.

Name Content Examples Size Link MD5 Checksum
train-images-idx3-ubyte.gz training set images 60,000 26 MBytes Download 8d4fb7e6c68d591d4c3dfef9ec88bf0d
train-labels-idx1-ubyte.gz training set labels 60,000 29 KBytes Download 25c81989df183df01b3e8a0aad5dffbe
t10k-images-idx3-ubyte.gz test set images 10,000 4.3 MBytes Download bef4ecab320f06d8554ea6380940ec79
t10k-labels-idx1-ubyte.gz test set labels 10,000 5.1 KBytes Download bb300cfdad3c16e7a12a480ee83cd310

Alternatively, you can clone this GitHub repository; the dataset appears under data/fashion. This repo also contains some scripts for benchmark and visualization.

git clone [email protected]:zalandoresearch/fashion-mnist.git

Labels

Each training and test example is assigned to one of the following labels:

Label Description
0 T-shirt/top
1 Trouser
2 Pullover
3 Dress
4 Coat
5 Sandal
6 Shirt
7 Sneaker
8 Bag
9 Ankle boot

Usage

Loading data with Python (requires NumPy)

Use utils/mnist_reader in this repo:

import mnist_reader
X_train, y_train = mnist_reader.load_mnist('data/fashion', kind='train')
X_test, y_test = mnist_reader.load_mnist('data/fashion', kind='t10k')

Loading data with Tensorflow

Make sure you have downloaded the data and placed it in data/fashion. Otherwise, Tensorflow will download and use the original MNIST.

from tensorflow.examples.tutorials.mnist import input_data
data = input_data.read_data_sets('data/fashion')

data.train.next_batch(BATCH_SIZE)

Note, Tensorflow supports passing in a source url to the read_data_sets. You may use:

data = input_data.read_data_sets('data/fashion', source_url='http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/')

Also, an official Tensorflow tutorial of using tf.keras, a high-level API to train Fashion-MNIST can be found here.

Loading data with other machine learning libraries

To date, the following libraries have included Fashion-MNIST as a built-in dataset. Therefore, you don't need to download Fashion-MNIST by yourself. Just follow their API and you are ready to go.

You are welcome to make pull requests to other open-source machine learning packages, improving their support to Fashion-MNIST dataset.

Loading data with other languages

As one of the Machine Learning community's most popular datasets, MNIST has inspired people to implement loaders in many different languages. You can use these loaders with the Fashion-MNIST dataset as well. (Note: may require decompressing first.) To date, we haven't yet tested all of these loaders with Fashion-MNIST.

Benchmark

We built an automatic benchmarking system based on scikit-learn that covers 129 classifiers (but no deep learning) with different parameters. Find the results here.

You can reproduce the results by running benchmark/runner.py. We recommend building and deploying this Dockerfile.

You are welcome to submit your benchmark; simply create a new issue and we'll list your results here. Before doing that, please make sure it does not already appear in this list. Visit our contributor guidelines for additional details.

The table below collects the submitted benchmarks. Note that we haven't yet tested these results. You are welcome to validate the results using the code provided by the submitter. Test accuracy may differ due to the number of epoch, batch size, etc. To correct this table, please create a new issue.

Classifier Preprocessing Fashion test accuracy MNIST test accuracy Submitter Code
2 Conv+pooling None 0.876 - Kashif Rasul 🔗
2 Conv+pooling None 0.916 - Tensorflow's doc 🔗
2 Conv+pooling+ELU activation (PyTorch) None 0.903 - @AbhirajHinge 🔗
2 Conv Normalization, random horizontal flip, random vertical flip, random translation, random rotation. 0.919 0.971 Kyriakos Efthymiadis 🔗
2 Conv <100K parameters None 0.925 0.992 @hardmaru 🔗
2 Conv ~113K parameters Normalization 0.922 0.993 Abel G. 🔗
2 Conv+3 FC ~1.8M parameters Normalization 0.932 0.994 @Xfan1025 🔗
2 Conv+3 FC ~500K parameters Augmentation, batch normalization 0.934 0.994 @cmasch 🔗
2 Conv+pooling+BN None 0.934 - @khanguyen1207 🔗
2 Conv+2 FC Random Horizontal Flips 0.939 - @ashmeet13 🔗
3 Conv+2 FC None 0.907 - @Cenk Bircanoğlu 🔗
3 Conv+pooling+BN None 0.903 0.994 @meghanabhange 🔗
3 Conv+pooling+2 FC+dropout None 0.926 - @Umberto Griffo 🔗
3 Conv+BN+pooling None 0.921 0.992 @gchhablani 🔗
5 Conv+BN+pooling None 0.931 - @Noumanmufc1 🔗
CNN with optional shortcuts, dense-like connectivity standardization+augmentation+random erasing 0.947 - @kennivich 🔗
GRU+SVM None 0.888 0.965 @AFAgarap 🔗
GRU+SVM with dropout None 0.897 0.988 @AFAgarap 🔗
WRN40-4 8.9M params standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips) 0.967 - @ajbrock 🔗 🔗
DenseNet-BC 768K params standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips) 0.954 - @ajbrock 🔗 🔗
MobileNet augmentation (horizontal flips) 0.950 - @苏剑林 🔗
ResNet18 Normalization, random horizontal flip, random vertical flip, random translation, random rotation. 0.949 0.979 Kyriakos Efthymiadis 🔗
GoogleNet with cross-entropy loss None 0.937 - @Cenk Bircanoğlu 🔗
AlexNet with Triplet loss None 0.899 - @Cenk Bircanoğlu 🔗
SqueezeNet with cyclical learning rate 200 epochs None 0.900 - @snakers4 🔗
Dual path network with wide resnet 28-10 standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips) 0.957 - @Queequeg 🔗
MLP 256-128-100 None 0.8833 - @heitorrapela 🔗
VGG16 26M parameters None 0.935 - @QuantumLiu 🔗 🔗
WRN-28-10 standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips) 0.959 - @zhunzhong07 🔗
WRN-28-10 + Random Erasing standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips) 0.963 - @zhunzhong07 🔗
Human Performance Crowd-sourced evaluation of human (with no fashion expertise) performance. 1000 randomly sampled test images, 3 labels per image, majority labelling. 0.835 - Leo -
Capsule Network 8M parameters Normalization and shift at most 2 pixel and horizontal flip 0.936 - @XifengGuo 🔗
HOG+SVM HOG 0.926 - @subalde 🔗
XgBoost scaling the pixel values to mean=0.0 and var=1.0 0.898 0.958 @anktplwl91 🔗
DENSER - 0.953 0.997 @fillassuncao 🔗 🔗
Dyra-Net Rescale to unit interval 0.906 - @Dirk Schäfer 🔗 🔗
Google AutoML 24 compute hours (higher quality) 0.939 - @Sebastian Heinz 🔗
Fastai Resnet50+Fine-tuning+Softmax on last layer's activations 0.9312 - @Sayak 🔗

Other Explorations of Fashion-MNIST

Fashion-MNIST: Year in Review

Fashion-MNIST on Google Scholar

Generative adversarial networks (GANs)

Clustering

Video Tutorial

Machine Learning Meets Fashion by Yufeng G @ Google Cloud

Machine Learning Meets Fashion

Introduction to Kaggle Kernels by Yufeng G @ Google Cloud

Introduction to Kaggle Kernels

动手学深度学习 by Mu Li @ Amazon AI

MXNet/Gluon中文频道

Apache MXNet으로 배워보는 딥러닝(Deep Learning) - 김무현 (AWS 솔루션즈아키텍트)

Apache MXNet으로 배워보는 딥러닝(Deep Learning)

Visualization

t-SNE on Fashion-MNIST (left) and original MNIST (right)

PCA on Fashion-MNIST (left) and original MNIST (right)

UMAP on Fashion-MNIST (left) and original MNIST (right)

PyMDE on Fashion-MNIST (left) and original MNIST (right)

Contributing

Thanks for your interest in contributing! There are many ways to get involved; start with our contributor guidelines and then check these open issues for specific tasks.

Contact

To discuss the dataset, please use Gitter.

Citing Fashion-MNIST

If you use Fashion-MNIST in a scientific publication, we would appreciate references to the following paper:

Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. Han Xiao, Kashif Rasul, Roland Vollgraf. arXiv:1708.07747

Biblatex entry:

@online{xiao2017/online,
  author       = {Han Xiao and Kashif Rasul and Roland Vollgraf},
  title        = {Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms},
  date         = {2017-08-28},
  year         = {2017},
  eprintclass  = {cs.LG},
  eprinttype   = {arXiv},
  eprint       = {cs.LG/1708.07747},
}

Who is citing Fashion-MNIST?

License

The MIT License (MIT) Copyright © [2017] Zalando SE, https://tech.zalando.com

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Owner
Zalando Research
Repositories of the research branch of Zalando SE
Zalando Research
GT4SD, an open-source library to accelerate hypothesis generation in the scientific discovery process.

The GT4SD (Generative Toolkit for Scientific Discovery) is an open-source platform to accelerate hypothesis generation in the scientific discovery process. It provides a library for making state-of-t

Generative Toolkit 4 Scientific Discovery 142 Dec 24, 2022
Code for LIGA-Stereo Detector, ICCV'21

LIGA-Stereo Introduction This is the official implementation of the paper LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based

Xiaoyang Guo 75 Dec 09, 2022
Adversarially Learned Inference

Adversarially Learned Inference Code for the Adversarially Learned Inference paper. Compiling the paper locally From the repo's root directory, $ cd p

Mohamed Ishmael Belghazi 308 Sep 24, 2022
Repo for the Tutorials of Day1-Day3 of the Nordic Probabilistic AI School 2021 (https://probabilistic.ai/)

ProbAI 2021 - Probabilistic Programming and Variational Inference Tutorial with Pryo Day 1 (June 14) Slides Notebook: students_PPLs_Intro Notebook: so

PGM-Lab 46 Nov 01, 2022
Fashion Entity Classification

Fashion-Entity-Classification - Fashion-MNIST is a dataset of Zalando's article images—consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grays

ADITYA SHAH 1 Jan 04, 2022
The code succinctly shows how our ensemble learning based on deep learning CNN is used for LAM-avulsion-diagnosis.

deep-learning-LAM-avulsion-diagnosis The code succinctly shows how our ensemble learning based on deep learning CNN is used for LAM-avulsion-diagnosis

1 Jan 12, 2022
Predict and time series avocado hass

RECOMMENDER SYSTEM MARKETING TỔNG QUAN VỀ HỆ THỐNG DỮ LIỆU 1. Giới thiệu - Tiki là một hệ sinh thái thương mại "all in one", trong đó có tiki.vn, là

hieulmsc 3 Jan 10, 2022
PyTorch Implementation of AnimeGANv2

PyTorch implementation of AnimeGANv2

4k Jan 07, 2023
Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity

[ICLR 2022] Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity by Shiwei Liu, Tianlong Chen, Zahra Atashgahi, Xiaohan Chen, Ghada Sokar, Elen

VITA 18 Dec 31, 2022
A set of Deep Reinforcement Learning Agents implemented in Tensorflow.

Deep Reinforcement Learning Agents This repository contains a collection of reinforcement learning algorithms written in Tensorflow. The ipython noteb

Arthur Juliani 2.2k Jan 01, 2023
Save-restricted-v-3 - Save restricted content Bot For telegram

Save restricted content Bot Contact: Telegram A stable telegram bot to get restr

DEVANSH 11 Dec 21, 2022
Genshin-assets - 👧 Public documentation & static assets for Genshin Impact data.

genshin-assets This repo provides easy access to the Genshin Impact assets, primarily for use on static sites. Sources Genshin Optimizer - An Artifact

Zerite Development 5 Nov 22, 2022
Official Pytorch Implementation of 3DV2021 paper: SAFA: Structure Aware Face Animation.

SAFA: Structure Aware Face Animation (3DV2021) Official Pytorch Implementation of 3DV2021 paper: SAFA: Structure Aware Face Animation. Getting Started

QiulinW 122 Dec 23, 2022
Codes for NeurIPS 2021 paper "Adversarial Neuron Pruning Purifies Backdoored Deep Models"

Adversarial Neuron Pruning Purifies Backdoored Deep Models Code for NeurIPS 2021 "Adversarial Neuron Pruning Purifies Backdoored Deep Models" by Dongx

Dongxian Wu 31 Dec 11, 2022
CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation

CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation [arxiv] This is the official repository for CDTrans: Cross-domain Transformer for

238 Dec 22, 2022
TeachMyAgent is a testbed platform for Automatic Curriculum Learning methods in Deep RL.

TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL Paper Website Documentation TeachMyAgent is a testbed platform for Automatic Cu

Flowers Team 51 Dec 25, 2022
Vehicle speed detection with python

Vehicle-speed-detection In the project simulate the tracker.py first then simulate the SpeedDetector.py. Finally, a new window pops up and the output

3 Dec 15, 2022
Compare outputs between layers written in Tensorflow and layers written in Pytorch

Compare outputs of Wasserstein GANs between TensorFlow vs Pytorch This is our testing module for the implementation of improved WGAN in Pytorch Prereq

Hung Nguyen 72 Dec 20, 2022
The official MegEngine implementation of the ICCV 2021 paper: GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning

[ICCV 2021] GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning This is the official implementation of our ICCV2021 paper GyroFlow. Our pres

MEGVII Research 36 Sep 07, 2022
Text-to-Image generation

Generate vivid Images for Any (Chinese) text CogView is a pretrained (4B-param) transformer for text-to-image generation in general domain. Read our p

THUDM 1.3k Dec 29, 2022