A PyTorch Lightning Callback for pushing models to the Hugging Face Hub 🤗⚡️

Last update: Dec 14, 2022

Overview

hf-hub-lightning

A callback for pushing lightning models to the Hugging Face Hub.

Note: I made this package for myself, mostly...if folks seem to be interested in it, we'll move this into huggingface_hub or an official PyTorch Lightning repo directly and archive this one. Since there are a lot of options/considerations, I decided to just post it separately for now as a proof of concept.

Setup

pip install hf-hub-lightning

Usage

To periodically upload your model ckpt and TensorBoard logs while training, you can do the following...

import pytorch_lightning as pl
from hf_hub_lightning import HuggingFaceHubCallback

train_dataloader = ...
model = YourCoolModel()
trainer = pl.Trainer(callbacks=[HuggingFaceHubCallback('your_username/model_id')])
trainer.fit(model, train_dataloader)

To load your model back from the huggingface hub, just do...

from huggingface_hub import hf_hub_download

ckpt_path = hf_hub_download('your_username/model_id', 'lit_model.ckpt')
model = YourCoolModel.load_from_checkpoint(ckpt_path)

You might also like...

Code for HLA-Face: Joint High-Low Adaptation for Low Light Face Detection (CVPR21)

HLA-Face: Joint High-Low Adaptation for Low Light Face Detection The official PyTorch implementation for HLA-Face: Joint High-Low Adaptation for Low L

77 Dec 8, 2022

DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition, TPAMI 2021

DVG-Face: Dual Variational Generation for HFR This repo is a PyTorch implementation of DVG-Face: Dual Variational Generation for Heterogeneous Face Re

52 Dec 30, 2022

[TIP 2021] SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

SADRNet Paper link: SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction Requirements python

Multimedia Computing Group, Nanjing University

99 Dec 30, 2022

Realtime Face Anti Spoofing with Face Detector based on Deep Learning using Tensorflow/Keras and OpenCV

Realtime Face Anti-Spoofing Detection 🤖 Realtime Face Anti Spoofing Detection with Face Detector to detect real and fake faces Please star this repo

86 Aug 3, 2022

Swapping face using Face Mesh with TensorFlow Lite

17 Apr 26, 2022

Face Synthetics dataset is a collection of diverse synthetic face images with ground truth labels.

The Face Synthetics dataset Face Synthetics dataset is a collection of diverse synthetic face images with ground truth labels. It was introduced in ou

608 Jan 2, 2023

Face Library is an open source package for accurate and real-time face detection and recognition

Face Library Face Library is an open source package for accurate and real-time face detection and recognition. The package is built over OpenCV and us

52 Nov 9, 2022

VGGFace2-HQ - A high resolution face dataset for face editing purpose

The first open source high resolution dataset for face swapping!!! A high resolution version of VGGFace2 for academic face editing purpose

232 Dec 29, 2022

A large-scale face dataset for face parsing, recognition, generation and editing.

CelebAMask-HQ [Paper] [Demo] CelebAMask-HQ is a large-scale face image dataset that has 30,000 high-resolution face images selected from the CelebA da

1.7k Dec 26, 2022

Comments

Saving only the best models
Hi, just curious how it is interconnected with PL internal checkpointing callback, and saving only the best models according to some given monitoring criteria like max validation accuracy etc.?

# saves a file like: my/path/sample-mnist-epoch=02-val_loss=0.32.ckpt checkpoint_callback = ModelCheckpoint( monitor="val_loss", dirpath="my/path/", filename="sample-mnist-{epoch:02d}-{val_loss:.2f}", save_top_k=3, mode="min", ) trainer = Trainer(callbacks=[checkpoint_callback])

see: https://pytorch-lightning.readthedocs.io/en/latest/common/weights_loading.html

Thx :rabbit:
opened by Borda 1

Releases(v0.0.2)

v0.0.2(Oct 27, 2021)
fix logger updates so you can view updated TB logs as your model trains

fix args to the callback, adding missing ones I had hardcoded

Source code(tar.gz)
Source code(zip)
v0.0.1(Oct 27, 2021)

Initial Release!
Source code(tar.gz)
Source code(zip)

A PyTorch Lightning Callback for pushing models to the Hugging Face Hub 🤗⚡️

Related tags

Overview

hf-hub-lightning

Setup

Usage

You might also like...

Code for HLA-Face: Joint High-Low Adaptation for Low Light Face Detection (CVPR21)

DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition, TPAMI 2021

[TIP 2021] SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

Realtime Face Anti Spoofing with Face Detector based on Deep Learning using Tensorflow/Keras and OpenCV

Swapping face using Face Mesh with TensorFlow Lite

Face Synthetics dataset is a collection of diverse synthetic face images with ground truth labels.

Face Library is an open source package for accurate and real-time face detection and recognition

VGGFace2-HQ - A high resolution face dataset for face editing purpose

A large-scale face dataset for face parsing, recognition, generation and editing.

Comments

Saving only the best models

Releases(v0.0.2)

v0.0.2(Oct 27, 2021)

v0.0.1(Oct 27, 2021)

Owner

Nathan Raw

YOLOX + ROS(1, 2) object detection package

Short and long time series classification using convolutional neural networks

Website for D2C paper

Jittor implementation of PCT:Point Cloud Transformer

This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

ERISHA is a mulitilingual multispeaker expressive speech synthesis framework. It can transfer the expressivity to the speaker's voice for which no expressive speech corpus is available.

This repository includes different versions of the prescribed-time controller as Simulink blocks and MATLAB script codes for engineering applications.

An open-source Kazakh named entity recognition dataset (KazNERD), annotation guidelines, and baseline NER models.

Code for the Higgs Boson Machine Learning Challenge organised by CERN & EPFL

DeepMind's software stack for physics-based simulation and Reinforcement Learning environments, using MuJoCo.

Visual Adversarial Imitation Learning using Variational Models (VMAIL)

The code repository for "RCNet: Reverse Feature Pyramid and Cross-scale Shift Network for Object Detection" (ACM MM'21)

Fit Fast, Explain Fast

Implementation of "Deep Implicit Templates for 3D Shape Representation"

Pytorch GUI(demo) for iVOS(interactive VOS) and GIS (Guided iVOS)

An Open-Source Package for Information Retrieval.

Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes (CVPR 2021 Oral)

NeurIPS 2021, self-supervised 6D pose on category level

NeoDTI: Neural integration of neighbor information from a heterogeneous network for discovering new drug-target interactions

Code for the ECIR'22 paper "Evaluating the Robustness of Retrieval Pipelines with Query Variation Generators"