Fast / fuzzy PostgreSQL counts for Django

Overview

Created by Stephen McDonald

Introduction

Up until PostgreSQL 9.2, COUNT queries generally required scanning every row in a database table. With millions of rows, this can become quite slow. One work-around for this is to query statistics in PostgreSQL for an approximate row count, which in many cases is an acceptable trade-off.

Given a table called bigdata, the following query will return an approximate row count:

SELECT reltuples FROM pg_class WHERE relname = 'bigdata';

You can read more information about slow COUNT queries in PostgreSQL in the PostgreSQL wiki.

What django-postgres-fuzzycount provides is a way of using this approach directly in your Django model managers. It was originally built for displaying statistics in the kouio RSS reader, a popular alternative to Google Reader, that acquired over 5 million news articles in its database during the first week of its launch.

Installation

The easiest way to install django-postgres-fuzzycount is directly from PyPi using pip by running the following command:

$ pip install -U django-postgres-fuzzycount

Otherwise you can download and install it directly from source:

$ python setup.py install

Usage

By using the fuzzycount.FuzzyCountManager on your Django models, its count() method will return an approximate value when querying PostgreSQL tables without any WHERE OR HAVING clauses:

from django.db import models
from fuzzycount import FuzzyCountManager

class BigData(models.Model):

    big = models.BooleanField(default=True)
    data = models.TextField()

    objects = FuzzyCountManager()

BigData.objects.count()  # Uses fuzzycount
BigData.objects.filter(id__gt=9000).count()  # Doesn't use fuzzycount

The fuzzycount.FuzzyCountManager also checks the database engine being used, and only applies the approximate count query when using PostgreSQL, so other database backends can be used and will behave as usual (for varying definitions of usual, depending on the database :-).

Inspiration

Owner
stephenmcd
stephenmcd
django-quill-editor makes Quill.js easy to use on Django Forms and admin sites

django-quill-editor django-quill-editor makes Quill.js easy to use on Django Forms and admin sites No configuration required for static files! The ent

lhy 139 Dec 05, 2022
Keep track of failed login attempts in Django-powered sites.

django-axes Axes is a Django plugin for keeping track of suspicious login attempts for your Django based website and implementing simple brute-force a

Jazzband 1.1k Dec 30, 2022
A Django web application that shortens long URLs. This is a demo project to show off my tech abilities.

Django URL Shortener This project is just a complete and production-ready URL shortener web application to show off my tech and coding abilities. Impo

Seyyed Ali Ayati 5 Jan 26, 2022
A Django web application to receive, virus check and validate transfers of digital archival records, and allow archivists to appraise and accession those records.

Aurora Aurora is a Django web application that can receive, virus check and validate transfers of digital archival records, and allows archivists to a

Rockefeller Archive Center 20 Aug 30, 2022
Django Advance DumpData

Django Advance Dumpdata Django Manage Command like dumpdata but with have more feature to Output the contents of the database from given fields of a m

EhsanSafir 7 Jul 25, 2022
Yet another Django audit log app, hopefully the simplest one.

django-easy-audit Yet another Django audit log app, hopefully the easiest one. This app allows you to keep track of every action taken by your users.

Natán 510 Jan 02, 2023
Django Login Api With Python

How to run this project Download and extract this project Create an environment and install all the libraries from requiements.txt pip freeze -r requi

Vikash Kisku 1 Dec 10, 2021
Sampling profiler for Python programs

py-spy: Sampling profiler for Python programs py-spy is a sampling profiler for Python programs. It lets you visualize what your Python program is spe

Ben Frederickson 9.5k Jan 01, 2023
Simple alternative to Doodle polls and scheduling (Python 3, Django 3, JavaScript)

What is jawanndenn? jawanndenn is a simple web application to schedule meetings and run polls, a libre alternative to Doodle. It is using the followin

Sebastian Pipping 169 Jan 06, 2023
This is raw connection between redis server and django python app

Django_Redis This repository contains the code for this blogpost. Running the Application Clone the repository git clone https://github.com/xxl4tomxu9

Tom Xu 1 Sep 15, 2022
A simple Blog Using Django Framework and Used IBM Cloud Services for Text Analysis and Text to Speech

ElhamBlog Cloud Computing Course first assignment. A simple Blog Using Django Framework and Used IBM Cloud Services for Text Analysis and Text to Spee

Elham Razi 5 Dec 06, 2022
File and Image Management Application for django

Django Filer django Filer is a file management application for django that makes handling of files and images a breeze. Contributing This is a an open

django CMS Association 1.6k Dec 28, 2022
AUES Student Management System Developed for laboratory works №9 Purpose using Python (Django).

AUES Student Management System (L M S ) AUES Student Management System Developed for laboratory works №9 Purpose using Python (Django). I've created t

ANAS NABIL 2 Dec 06, 2021
Pipeline is an asset packaging library for Django.

Pipeline Pipeline is an asset packaging library for Django, providing both CSS and JavaScript concatenation and compression, built-in JavaScript templ

Jazzband 1.4k Jan 03, 2023
An opinionated Django CMS setup bundled as an Aldryn Addon

Aldryn CMS |PyPI Version| An opinionated django CMS setup bundled as an Aldryn Addon. This package will auto configure django CMS including some extra

Vladimir Bezrukov 1 Nov 12, 2021
A Django/Python web app that functions as a digital diary

My Django Diary Full-stack web application that functions as a digital diary using Django, Python, SQLite, HTML & CSS. Things I learned during this pr

1 Sep 30, 2022
Silk is a live profiling and inspection tool for the Django framework.

Silk is a live profiling and inspection tool for the Django framework. Silk intercepts and stores HTTP requests and database queries before presenting them in a user interface for further inspection:

Jazzband 3.7k Jan 02, 2023
Auto-detecting the n+1 queries problem in Python

nplusone nplusone is a library for detecting the n+1 queries problem in Python ORMs, including SQLAlchemy, Peewee, and the Django ORM. The Problem Man

Joshua Carp 837 Dec 29, 2022
Sistema de tratamento e análise de grandes volumes de dados através de técnicas de Data Science

Sistema de tratamento e análise de grandes volumes de dados através de técnicas de data science Todos os scripts, gráficos e relatórios de todas as at

Arthur Quintanilha Neto 1 Sep 05, 2022
It's the assignment 1 from the Python 2 course, that requires a ToDoApp with authentication using Django

It's the assignment 1 from the Python 2 course, that requires a ToDoApp with authentication using Django

0 Jan 20, 2022