Using OpenAI's CLIP to upscale and enhance images

Last update: Jun 14, 2022

Overview

CLIP Upscaler and Enhancer

Using OpenAI's CLIP to upscale and enhance images

Based on nshepperd's JAX CLIP Guided Diffusion v2.4

Sample Results

Viewport	Original Image (512x512)	Using This Repo (scaled to 1024x1024 in 45 minutes)	Using Waifu2x (scaled to 1024x1024 in a few seconds)
Full Image
Zoomed In

How it works

Start with any image.
Find CLIP's embeddings of the image.
Resize the image to the desired resolution and add some noise.
Optimize the resized image for the original image's embeddings (using nshepperd's JAX CLIP Guided Diffusion v2.4).

Owner

Tripp Lyons

Blockchain/Ethereum/Solidity, Web Dev, Security

GitHub Repository

A setup script to generate ITK Python Wheels

ITK Python Package This project provides a setup.py script to build ITK Python binary packages and infrastructure to build ITK external module Python

59 Dec 14, 2022

The Habitat-Matterport 3D Research Dataset - the largest-ever dataset of 3D indoor spaces.

Habitat-Matterport 3D Dataset (HM3D) The Habitat-Matterport 3D Research Dataset is the largest-ever dataset of 3D indoor spaces. It consists of 1,000

62 Dec 27, 2022

mPose3D, a mmWave-based 3D human pose estimation model.

35 Nov 08, 2022

Code for "Diffusion is All You Need for Learning on Surfaces"

Source code for "Diffusion is All You Need for Learning on Surfaces", by Nicholas Sharp Souhaib Attaiki Keenan Crane Maks Ovsjanikov NOTE: the linked

247 Dec 28, 2022

The pytorch implementation of DG-Font: Deformable Generative Networks for Unsupervised Font Generation

DG-Font: Deformable Generative Networks for Unsupervised Font Generation The source code for 'DG-Font: Deformable Generative Networks for Unsupervised

130 Dec 05, 2022

ISBI 2022: Cross-level Contrastive Learning and Consistency Constraint for Semi-supervised Medical Image.

Cross-level Contrastive Learning and Consistency Constraint for Semi-supervised Medical Image Introduction This repository contains the PyTorch implem

25 Nov 09, 2022

Pytorch implementation of the paper Progressive Growing of Points with Tree-structured Generators (BMVC 2021)

PGpoints Pytorch implementation of the paper Progressive Growing of Points with Tree-structured Generators (BMVC 2021) Hyeontae Son, Young Min Kim Pre

9 Jun 06, 2022

More than a hundred strange attractors

dysts Analyze more than a hundred chaotic systems. Basic Usage Import a model and run a simulation with default initial conditions and parameter value

185 Dec 23, 2022

Code release for "Masked-attention Mask Transformer for Universal Image Segmentation"

Mask2Former: Masked-attention Mask Transformer for Universal Image Segmentation Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexander Kirillov, Ro

1.2k Jan 02, 2023

Deep learning (neural network) based remote photoplethysmography: how to extract pulse signal from video using deep learning tools

Deep-rPPG: Camera-based pulse estimation using deep learning tools Deep learning (neural network) based remote photoplethysmography: how to extract pu

138 Dec 17, 2022

GPU-accelerated PyTorch implementation of Zero-shot User Intent Detection via Capsule Neural Networks

GPU-accelerated PyTorch implementation of Zero-shot User Intent Detection via Capsule Neural Networks This repository implements a capsule model Inten

15 Dec 24, 2022

Proto-RL: Reinforcement Learning with Prototypical Representations

Proto-RL: Reinforcement Learning with Prototypical Representations This is a PyTorch implementation of Proto-RL from Reinforcement Learning with Proto

74 Dec 06, 2022

An implementation of "MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing" (ICML 2019).

MixHop and N-GCN ⠀ A PyTorch implementation of "MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing" (ICML 2019)

393 Dec 13, 2022

g2o: A General Framework for Graph Optimization

g2o - General Graph Optimization Linux: Windows: g2o is an open-source C++ framework for optimizing graph-based nonlinear error functions. g2o has bee

2.5k Dec 30, 2022

deep learning model with only python and numpy with test accuracy 99 % on mnist dataset and different optimization choices

deep_nn_model_with_only_python_100%_test_accuracy deep learning model with only python and numpy with test accuracy 99 % on mnist dataset and differen

0 Aug 28, 2022

FedMM: Saddle Point Optimization for Federated Adversarial Domain Adaptation

This repository contains the code accompanying the paper " FedMM: Saddle Point Optimization for Federated Adversarial Domain Adaptation" Paper link: R

20 Jun 29, 2022

The code for the CVPR 2021 paper Neural Deformation Graphs, a novel approach for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.

Neural Deformation Graphs Project Page | Paper | Video Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction Aljaž Božič, Pablo P

134 Dec 16, 2022

PyTorch Implementation of Vector Quantized Variational AutoEncoders.

Pytorch implementation of VQVAE. This paper combines 2 tricks: Vector Quantization (check out this amazing blog for better understanding.) Straight-Th

2 Oct 06, 2021

Original Implementation of Prompt Tuning from Lester, et al, 2021

Prompt Tuning This is the code to reproduce the experiments from the EMNLP 2021 paper "The Power of Scale for Parameter-Efficient Prompt Tuning" (Lest

282 Dec 28, 2022

Trash Sorter Extraordinaire is a software which efficiently detects the different types of waste in a pile of random trash through feeding it pictures or videos.

Trash-Sorter-Extraordinaire Trash Sorter Extraordinaire is a software which efficiently detects the different types of waste in a pile of random trash

1 Nov 07, 2021

Using OpenAI's CLIP to upscale and enhance images

Related tags

Overview

CLIP Upscaler and Enhancer

Sample Results

How it works

Owner

Tripp Lyons

A setup script to generate ITK Python Wheels

The Habitat-Matterport 3D Research Dataset - the largest-ever dataset of 3D indoor spaces.

mPose3D, a mmWave-based 3D human pose estimation model.

Code for "Diffusion is All You Need for Learning on Surfaces"

The pytorch implementation of DG-Font: Deformable Generative Networks for Unsupervised Font Generation

ISBI 2022: Cross-level Contrastive Learning and Consistency Constraint for Semi-supervised Medical Image.

Pytorch implementation of the paper Progressive Growing of Points with Tree-structured Generators (BMVC 2021)

More than a hundred strange attractors

Code release for "Masked-attention Mask Transformer for Universal Image Segmentation"

Deep learning (neural network) based remote photoplethysmography: how to extract pulse signal from video using deep learning tools

GPU-accelerated PyTorch implementation of Zero-shot User Intent Detection via Capsule Neural Networks

Proto-RL: Reinforcement Learning with Prototypical Representations

An implementation of "MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing" (ICML 2019).

g2o: A General Framework for Graph Optimization

deep learning model with only python and numpy with test accuracy 99 % on mnist dataset and different optimization choices

FedMM: Saddle Point Optimization for Federated Adversarial Domain Adaptation

The code for the CVPR 2021 paper Neural Deformation Graphs, a novel approach for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.

PyTorch Implementation of Vector Quantized Variational AutoEncoders.

Original Implementation of Prompt Tuning from Lester, et al, 2021

Trash Sorter Extraordinaire is a software which efficiently detects the different types of waste in a pile of random trash through feeding it pictures or videos.