Fast / fuzzy PostgreSQL counts for Django

Overview

Created by Stephen McDonald

Introduction

Up until PostgreSQL 9.2, COUNT queries generally required scanning every row in a database table. With millions of rows, this can become quite slow. One work-around for this is to query statistics in PostgreSQL for an approximate row count, which in many cases is an acceptable trade-off.

Given a table called bigdata, the following query will return an approximate row count:

SELECT reltuples FROM pg_class WHERE relname = 'bigdata';

You can read more information about slow COUNT queries in PostgreSQL in the PostgreSQL wiki.

What django-postgres-fuzzycount provides is a way of using this approach directly in your Django model managers. It was originally built for displaying statistics in the kouio RSS reader, a popular alternative to Google Reader, that acquired over 5 million news articles in its database during the first week of its launch.

Installation

The easiest way to install django-postgres-fuzzycount is directly from PyPi using pip by running the following command:

$ pip install -U django-postgres-fuzzycount

Otherwise you can download and install it directly from source:

$ python setup.py install

Usage

By using the fuzzycount.FuzzyCountManager on your Django models, its count() method will return an approximate value when querying PostgreSQL tables without any WHERE OR HAVING clauses:

from django.db import models
from fuzzycount import FuzzyCountManager

class BigData(models.Model):

    big = models.BooleanField(default=True)
    data = models.TextField()

    objects = FuzzyCountManager()

BigData.objects.count()  # Uses fuzzycount
BigData.objects.filter(id__gt=9000).count()  # Doesn't use fuzzycount

The fuzzycount.FuzzyCountManager also checks the database engine being used, and only applies the approximate count query when using PostgreSQL, so other database backends can be used and will behave as usual (for varying definitions of usual, depending on the database :-).

Inspiration

Owner
stephenmcd
stephenmcd
Intellicards-backend - A Django project bootstrapped with django-admin startproject mysite

Intellicards-backend - A Django project bootstrapped with django-admin startproject mysite

Fabrizio Torrico 2 Jan 13, 2022
Realworld - Realworld using Django and HTMX

Realworld - Realworld using Django and HTMX

Dan Jacob 53 Jan 05, 2023
REST API with Django and SQLite3

REST API with Django and SQLite3

Luis Quiñones Requelme 1 Nov 07, 2021
Vehicle registration using Python, Django and SQlite3

PythonCrud Cadastro de veículos utilizando Python, Django e SQlite3 Para acessar o deploy no Heroku:

Jorge Thiago 4 May 20, 2022
mirage ~ ♪ extended django admin or manage.py command.

mirage ~ ♪ extended django admin or manage.py command. ⬇️ Installation Installing Mirage with Pipenv is recommended. pipenv install -d mirage-django-l

Shota Shimazu 6 Feb 14, 2022
Per object permissions for Django

django-guardian django-guardian is an implementation of per object permissions [1] on top of Django's authorization backend Documentation Online docum

3.3k Jan 04, 2023
📝 Sticky Notes in Django admin

django-admin-sticky-notes Share notes between superusers. Installation Install via pip: pip install django_admin_sticky_notes Put django_admin_sticky_

Dariusz Choruży 7 Oct 06, 2021
Py-instant-search-redis - Source code example for how to build an instant search with redis in python

py-instant-search-redis Source code example for how to build an instant search (

Giap Le 4 Feb 17, 2022
A Django web application to receive, virus check and validate transfers of digital archival records, and allow archivists to appraise and accession those records.

Aurora Aurora is a Django web application that can receive, virus check and validate transfers of digital archival records, and allows archivists to a

Rockefeller Archive Center 20 Aug 30, 2022
Backend with Django .

BackendCode - Cookies Documentation: https://docs.djangoproject.com/fr/3.2/intro/ By @tcotidiane33 & @yaya Models Premium class Pack(models.Model): n

just to do it 1 Jan 28, 2022
Imparare Django ricreando un sito facsimile a quello Flask

SitoPBG-Django Imparare Django ricreando un sito facsimile a quello Flask Note di utilizzo Necessita la valorizzazione delle seguenti variabili di amb

Mario Nardi 1 Dec 08, 2021
Django-pwned - A collection of django password validators

Django Pwned A collection of django password validators. Compatibility Python: 3

Quera 22 Jun 27, 2022
☄️ Google Forms autofill script

lazrr 'Destroy Them With Lazers' - Knife Party, 2011 Google Forms autofill script Installation: pip3 install -r requirements.txt Usage: python3 lazrr.

Serezha Rakhmanov 12 Jun 04, 2022
A app for managing lessons with Django

Course Notes A app for managing lessons with Django Some Ideas

Motahhar.Mokfi 6 Jan 28, 2022
Analytics services for Django projects

django-analytical The django-analytical application integrates analytics services into a Django project. Using an analytics service with a Django proj

Jazzband 1.1k Dec 31, 2022
Django + NextJS + Tailwind Boilerplate

django + NextJS + Tailwind Boilerplate About A Django project boilerplate/templa

Shayan Debroy 3 Mar 11, 2022
Improved Django model inheritance with automatic downcasting

Polymorphic Models for Django Django-polymorphic simplifies using inherited models in Django projects. When a query is made at the base model, the inh

1.4k Jan 03, 2023
A CBV to handle multiple forms in one view

django-shapeshifter A common problem in Django is how to have a view, especially a class-based view that can display and process multiple forms at onc

Kenneth Love 167 Nov 26, 2022
A Django Demo Project of Students Management System

Django_StudentMS A Django Demo Project of Students Management System. From NWPU Seddon for DB Class Pre. Seddon simplify the code in 2021/10/17. Hope

2 Dec 08, 2021
The friendly PIL fork (Python Imaging Library)

Pillow Python Imaging Library (Fork) Pillow is the friendly PIL fork by Alex Clark and Contributors. PIL is the Python Imaging Library by Fredrik Lund

Pillow 10.4k Jan 03, 2023