A Python dictionary implementation designed to act as an in-memory cache for FaaS environments

Related tags

Data Structuresfaas-cache-dict
Overview

faas-cache-dict

A Python dictionary implementation designed to act as an in-memory cache for FaaS environments.

Formally you would describe this a memory constrained LRU TTL cache dict.

This is implementation only uses core Python stdlib with no external dependencies.

Background

This was originally designed to be used as an in-memory cache for AWS Lambda.

In most FaaS environments, successive quick calls of the function persists variables in the global scope. We can leverage this to cache data for future calls.

FaaS runtimes have limited RAM capacities so this library allows you to set a max mebibyte size for the dict. It also allows setting an optional max items length.

Items are kept in order with the LRU at the HEAD of the list.

Items are deleted if they expire, or from the head (LRU) if the cache is out of space.

Usage

Limitations

  • Performance degrades with size, you will need to test this for your use case. Though in most circumstances this will be much faster than performing a network call to a cache.
  • The library should be thread-safe, but limited testing has gone into this.

Support

CPython 3.7 or greater. No extra dependencies are required.

Contributions

This code is distributed under an open license. Feel free to fork it or preferably open a PR.

Inspirations

Thanks to mobilityhouse/ttldict for their implementation which served as a starting point.

You might also like...
Leetcode solutions - All algorithms implemented in Python 3 (for education)

Leetcode solutions - All algorithms implemented in Python 3 (for education)

Vineet Dhaimodker 3 Oct 21, 2022
This Repository consists of my solutions in Python 3 to various problems in Data Structures and Algorithms
This Repository consists of my solutions in Python 3 to various problems in Data Structures and Algorithms

Problems and it's solutions. Problem solving, a great Speed comes with a good Accuracy. The more Accurate you can write code, the more Speed you will

SAMIR PAUL 1.3k Jan 1, 2023
Python library for doing things with Grid-like structures

gridthings Python library for doing things with Grid-like structures Development This project uses poetry for dependency management, pre-commit for li

Matt Kafonek 2 Dec 21, 2021
A Python library for electronic structure pre/post-processing
A Python library for electronic structure pre/post-processing

PyProcar PyProcar is a robust, open-source Python library used for pre- and post-processing of the electronic structure data coming from DFT calculati

Romero Group 124 Dec 7, 2022
Programming of a spanning tree algorithm with Python : In depth first with a root node.

ST Algorithm Programming of a spanning tree algorithm with Python : In depth first with a root node. Description This programm reads informations abou

Mathieu Lamon 1 Dec 16, 2021
Data Structure With Python

Data-Structure-With-Python- Python programs also include in this repo Stack A stack is a linear data structure that stores items in a Last-In/First-Ou

Sumit Nautiyal 2 Jan 9, 2022
🔬 Fixed struct serialization system, using Python 3.9 annotated type hints

py-struct Fixed-size struct serialization, using Python 3.9 annotated type hints This was originally uploaded as a Gist because it's not intended as a

Alba Mendez 4 Jan 14, 2022
Final Project for Practical Python Programming and Algorithms for Data Analysis
Final Project for Practical Python Programming and Algorithms for Data Analysis

Final Project for Practical Python Programming and Algorithms for Data Analysis (PHW2781L, Summer 2020) Redlining, Race-Exclusive Deed Restriction Lan

Aislyn Schalck 1 Jan 27, 2022
Basic sort and search algorithms written in python.

Basic sort and search algorithms written in python. These were all developed as part of my Computer Science course to demonstrate understanding so they aren't 100% efficent

Ben Jones 0 Dec 14, 2022
Releases(0.2.3)

  • 0.2.3(Apr 27, 2022)

    Included

    • Fix min version installable on Py 3.8 + (due to pickle protocol versioning)

    Todo:

    • Discover better documents solution

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.2.2(Apr 25, 2022)

    Included

    • Add purge() to empty all contents of cache dict

    Todo:

    • Discover better documents solution

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.2.1(Apr 22, 2022)

    Included

    • Fix set_ttl() should be able to make a value immortal by passing None
    • Fix is_expired() no longer errors if the key has been purged, now returns True if missing
    • Add ability to change the max (LRU) length of the cache once created, purging the LRU as required.
    • Add extensive new documentation

    Todo:

    • Discover better documents solution

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.2.0(Apr 22, 2022)

    Included

    • Add FileBackedFaaSCache
    • Add support for pickling faas cache objects
    • Bump minimum supported version to Python 3.8
    • Fix minor docs

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.1.3(Feb 15, 2022)

    Included

    • Cleanup deprecated leftovers from objsize migration

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.1.2(Feb 15, 2022)

    Included

    • Replaced primitive deep size algorithm with objsize

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.6(Feb 15, 2022)

    Included

    • Fixed "falsey" key causing badly reported item expiry. Thanks @lokhman
    • Fixed get_deep_size incorrectly passing params on deeply nested dicts.
    • Fix handling of attempted nested deletion of already deleted items.
    • Refactor constants to own file and general code cleanup.

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.5(Dec 20, 2021)

    Included

    • Fix usage in README

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.4(Dec 20, 2021)

    Included

    • Add module level python version lockout
    • Improve object __repr__
    • Add usage docstring

    Todo:

    • Add full documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.3(Dec 20, 2021)

    Included

    • Added new support for memory constraint shortcuts, 1M for 1 Mebibyte, 1K, 1G and 1T also supported respectively.
    • Added support for float TTL and float memory constraints.
    • Fixed nasty bug which could cause livelock if memory constraint made too small after data had been added.
    • Added max items to object __repr___.
    • Standardised user input linting.
    • Unit tests for the new functionality.
    • Excluded scripts folder from the distributable.
    • Refactoring of project to split out logic into smaller files.

    Todo:

    • Add usage documentation.

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)

  • 0.0.2(Dec 17, 2021)

    Initial release.

    Included

    • Support implemented for TTL, LRU and max mebibyte size.
    • Unit tests.

    Todo:

    • Add usage documentation

    This has not yet been tested in a prod environment, and as such should be considered beta software for now.

    Source code(tar.gz)
    Source code(zip)
Owner
Juan
Security Engineer, Python charmer, JSON wizard.
Juan
GitHub Repository
Python tree data library

Links Documentation PyPI GitHub Changelog Issues Contributors If you enjoy anytree Getting started Usage is simple. Construction from anytree impo

776 Dec 28, 2022
Array is a functional mutable sequence inheriting from Python's built-in list.

funct.Array Array is a functional mutable sequence inheriting from Python's built-in list. Array provides 100+ higher-order methods and more functiona

182 Nov 21, 2022
schemasheets - structuring your data using spreadsheets

schemasheets - structuring your data using spreadsheets Create a data dictionary / schema for your data using simple spreadsheets - no coding required

Linked data Modeling Language 23 Dec 01, 2022
nocasedict - A case-insensitive ordered dictionary for Python

nocasedict - A case-insensitive ordered dictionary for Python Overview Class NocaseDict is a case-insensitive ordered dictionary that preserves the or

PyWBEM Projects 2 Dec 12, 2021
Supporting information (calculation outputs, structures)

Supporting information (calculation outputs, structures)

Eric Berquist 2 Feb 02, 2022
Leetcode solutions - All algorithms implemented in Python 3 (for education)

Leetcode solutions - All algorithms implemented in Python 3 (for education)

Vineet Dhaimodker 3 Oct 21, 2022
IADS 2021-22 Algorithm and Data structure collection

A collection of algorithms and datastructures introduced during UoE's Introduction to Datastructures and Algorithms class.

Artemis Livingstone 20 Nov 07, 2022
A HDF5-based python pickle replacement

Hickle Hickle is an HDF5 based clone of pickle, with a twist: instead of serializing to a pickle file, Hickle dumps to an HDF5 file (Hierarchical Data

Danny Price 450 Dec 21, 2022
An command-line utility that schedules your exams preparation routines

studyplan A tiny utility that schedules your exams preparation routines. You only need to specify the tasks and the deadline. App will output a iCal f

Ilya Breitburg 3 May 18, 2022
Webtesting for course Data Structures & Algorithms

Selenium job to automate queries to check last posts of Module Data Structures & Algorithms Web-testing for course Data Structures & Algorithms Struct

1 Dec 15, 2021
pyprobables is a pure-python library for probabilistic data structures

pyprobables is a pure-python library for probabilistic data structures. The goal is to provide the developer with a pure-python implementation of common probabilistic data-structures to use in their

Tyler Barrus 86 Dec 25, 2022
Python library for doing things with Grid-like structures

gridthings Python library for doing things with Grid-like structures Development This project uses poetry for dependency management, pre-commit for li

Matt Kafonek 2 Dec 21, 2021
Al-Quran dengan Terjemahan Indonesia

Al-Quran Rofi Al-Quran dengan Terjemahan / Tafsir Jalalayn Instalasi Al-Quran Rofi untuk Archlinux untuk pengguna distro Archlinux dengan paket manage

Nestero 4 Dec 20, 2021
This repository contains code for CTF platform.

CTF-platform Repository for backend of CTF hosting website For starting the project first time : Clone the repo in which you have to work in your syst

Yash Jain 3 Feb 18, 2022
Solutions for leetcode problems.

Leetcode-solution This is an repository for storring new algorithms that I am learning form the LeetCode for future use. Implemetations Two Sum (pytho

Shrutika Borkute 1 Jan 09, 2022
Programming of a spanning tree algorithm with Python : In depth first with a root node.

ST Algorithm Programming of a spanning tree algorithm with Python : In depth first with a root node. Description This programm reads informations abou

Mathieu Lamon 1 Dec 16, 2021
Datastructures such as linked list, trees, graphs etc

datastructures datastructures such as linked list, trees, graphs etc Made a public repository for coding enthusiasts. Those who want to collaborate on

0 Dec 01, 2021
This repo represents all we learned and are learning in Data Structure course.

DataStructure Journey This repo represents all we learned and are learning in Data Structure course which is based on CLRS book and is being taught by

Aprime Afr (Alireza Afroozi) 3 Jan 22, 2022
This Repository consists of my solutions in Python 3 to various problems in Data Structures and Algorithms

Problems and it's solutions. Problem solving, a great Speed comes with a good Accuracy. The more Accurate you can write code, the more Speed you will

SAMIR PAUL 1.3k Jan 01, 2023
A JSON-friendly data structure which allows both object attributes and dictionary keys and values to be used simultaneously and interchangeably.

A JSON-friendly data structure which allows both object attributes and dictionary keys and values to be used simultaneously and interchangeably.

Peter F 93 Dec 01, 2022