Python document object mapper (load python object from JSON and vice-versa)

Overview

lupin is a Python JSON object mapper

Build Status

lupin is meant to help in serializing python objects to JSON and unserializing JSON data to python objects.

Installation

pip install lupin

Usage

lupin uses schemas to create a representation of a python object.

A schema is composed of fields which represents the way to load and dump an attribute of an object.

Define schemas

from datetime import datetime
from lupin import Mapper, Schema, fields as f


# 1) Define your models
class Thief(object):
    def __init__(self, name, stolen_items):
        self.name = name
        self.stolen_items = stolen_items


class Painting(object):
    def __init__(self, name, author):
        self.name = name
        self.author = author


class Artist(object):
    def __init__(self, name, birth_date):
        self.name = name
        self.birth_date = birth_date


# 2) Create schemas
artist_schema = Schema({
    "name": f.String(),
    "birthDate": f.DateTime(binding="birth_date", format="%Y-%m-%d")
}, name="artist")

painting_schema = Schema({
    "name": f.String(),
    "author": f.Object(artist_schema)
}, name="painting")

thief_schema = Schema({
    "name": f.String(),
    "stolenItems": f.List(painting_schema, binding="stolen_items")
}, name="thief")

# 3) Create a mapper and register a schema for each of your models you want to map to JSON objects
mapper = Mapper()

mapper.register(Artist, artist_schema)
mapper.register(Painting, painting_schema)
mapper.register(Thief, thief_schema)


# 4) Create some sample data
leonardo = Artist(name="Leonardo da Vinci", birth_date=datetime(1452, 4, 15))
mona_lisa = Painting(name="Mona Lisa", author=leonardo)
arsene = Thief(name="Arsène Lupin", stolen_items=[mona_lisa])

Dump objects

# use mapper to dump python objects
assert mapper.dump(leonardo) == {
    "name": "Leonardo da Vinci",
    "birthDate": "1452-04-15"
}

assert mapper.dump(mona_lisa) == {
    "name": "Mona Lisa",
    "author": {
        "name": "Leonardo da Vinci",
        "birthDate": "1452-04-15"
    }
}

assert mapper.dump(arsene) == {
    "name": "Arsène Lupin",
    "stolenItems": [
        {
            "name": "Mona Lisa",
            "author": {
                "name": "Leonardo da Vinci",
                "birthDate": "1452-04-15"
            }
        }
    ]
}

Load objects

# use mapper to load JSON data
data = {
    "name": "Mona Lisa",
    "author": {
        "name": "Leonardo da Vinci",
        "birthDate": "1452-04-15"
    }
}
painting = mapper.load(data, "painting")  # "painting" is the name of the schame you want to use
artist = painting.author

assert isinstance(painting, Painting)
assert painting.name == "Mona Lisa"

assert isinstance(artist, Artist)
assert artist.name == "Leonardo da Vinci"
assert artist.birth_date == datetime(1452, 4, 15)

Polymorphic lists

Sometimes a list can contain multiple type of objects. In such cases you will have to use a PolymorphicList, you will also need to add a key in the items schema to store the type of the object (you can use a Constant field).

Say that our thief has level up and has stolen a diamond.

class Diamond(object):
    def __init__(self, carat):
        self.carat = carat


mapper = Mapper()

# Register a schema for diamonds
diamond_schema = Schema({
    "carat": f.Field(),
    "type": f.Constant("diamond")  # this will be used to know which schema to used while loading JSON
}, name="diamond")
mapper.register(Diamond, diamond_schema)

# Change our painting schema in order to include a `type` field
painting_schema = Schema({
    "name": f.String(),
    "type": f.Constant("painting"),
    "author": f.Object(artist_schema)
}, name="painting")
mapper.register(Painting, painting_schema)

# Use `PolymorphicList` for `stolen_items`
thief_schema = Schema({
    "name": f.String(),
    "stolenItems": f.PolymorphicList(on="type",  # JSON key to lookup for the polymorphic type
                                     binding="stolen_items",
                                     schemas={
                                         "painting": painting_schema,  # if `type == "painting"` then use painting_schema
                                         "diamond": diamond_schema  # if `type == "diamond"` then use diamond_schema
                                     })
}, name="thief")
mapper.register(Thief, thief_schema)


diamond = Diamond(carat=20)
arsene.stolen_items.append(diamond)

# Dump object
data = mapper.dump(arsene)
assert data == {
    "name": "Arsène Lupin",
    "stolenItems": [
        {
            "name": "Mona Lisa",
            "type": "painting",
            "author": {
                "name": "Leonardo da Vinci",
                "birthDate": "1452-04-15"
            }
        },
        {
            "carat": 20,
            "type": "diamond"
        }
    ]
}

# Load data
thief = mapper.load(data, "thief")
assert isinstance(thief.stolen_items[0], Painting)
assert isinstance(thief.stolen_items[1], Diamond)

Validation

Lupin provides a set of builtin validators, you can find them in the lupin/validators folder.

While creating your schemas you can assign validators to the fields. Before loading a document lupin will validate its format. If one field is invalid, an InvalidDocument is raised with all the error detected in the data.

Example :

from lupin import Mapper, Schema, fields as f, validators as v
from lupin.errors import InvalidDocument, InvalidLength
from models import Artist

mapper = Mapper()

artist_schema = Schema({
    "name": f.String(validators=v.Length(max=10)),
}, name="artist")
mapper.register(Artist, artist_schema)

data = {
    "name": "Leonardo da Vinci"
}

try:
    mapper.load(data, artist_schema, allow_partial=True)
except InvalidDocument as errors:
    error = errors[0]
    assert isinstance(error, InvalidLength)
    assert error.path == ["name"]

Current validators are :

  • DateTimeFormat (validate that value is a valid datetime format)
  • Equal (validate that value is equal to a predefined one)
  • In (validate that a value is contained in a set of value)
  • Length (validate the length of a value)
  • Match (validate the format of a value with a regex)
  • Type (validate the type of a value, this validator is already included in all fields to match the field type)
  • URL (validate an URL string format)
  • IsNone (validate that value is None)
  • Between (validate that value belongs to a range)

Combination

You can build validators combinations using the & and | operator.

Example :

from lupin import validators as v
from lupin.errors import ValidationError

validators = v.Equal("Lupin") | v.Equal("Andrésy")
# validators passes only if value is "Lupin" or "Andrésy"

validators("Lupin", [])

try:
    validators("Holmes", [])
except ValidationError:
    print("Validation error")
Owner
Aurélien Amilin
Aurélien Amilin
Mozilla Campus Club CCEW is a student committee working to spread awareness on Open Source software.

Mozilla Campus Club CCEW is a student committee working to spread awareness on Open Source software. We organize webinars and workshops on different technical topics and making Open Source contributi

Mozilla-Campus-Club-Cummins 8 Jun 15, 2022
Elliptic curve cryptography (ed25519) beginner tutorials in Python 3

ed25519_tutorials Elliptic curve cryptography (ed25519) beginner tutorials in Python 3 Instructions Just download the repo and read the tutorial files

6 Dec 27, 2022
Repository for learning Python (Python Tutorial)

Repository for learning Python (Python Tutorial) Languages and Tools 🧰 Overview 📑 Repository for learning Python (Python Tutorial) Languages and Too

Swiftman 2 Aug 22, 2022
Generating a report CSV and send it to an email - Python / Django Rest Framework

Generating a report in CSV format and sending it to a email How to start project. Create a folder in your machine Create a virtual environment python3

alexandre Lopes 1 Jan 17, 2022
Modified fork of CPython's ast module that parses `# type:` comments

Typed AST typed_ast is a Python 3 package that provides a Python 2.7 and Python 3 parser similar to the standard ast library. Unlike ast up to Python

Python 217 Dec 06, 2022
A simple document management REST based API for collaboratively interacting with documents

documan_api A simple document management REST based API for collaboratively interacting with documents.

Shahid Yousuf 1 Jan 22, 2022
Fastest Git client for Emacs.

EAF Git Client EAF Git is git client application for the Emacs Application Framework. The advantages of EAF Git are: Large log browse: support 1 milli

Emacs Application Framework 31 Dec 02, 2022
🐱‍🏍 A curated list of awesome things related to Hugo themes.

awesome-hugo-themes Automated deployment @ 2021-10-12 06:24:07 Asia/Shanghai &sorted=updated Theme Author License GitHub Stars Updated Blonde wamo MIT

13 Dec 12, 2022
Ultimaker Cura 2 Mooraker Upload Plugin

Klipper & Cura - Cura2MoonrakerPlugin Allows you to upload Gcode directly from Cura to your Klipper-based 3D printer (Fluidd, Mainsailos etc.) using t

214 Jan 03, 2023
Data-science-on-gcp - Source code accompanying book: Data Science on the Google Cloud Platform, Valliappa Lakshmanan, O'Reilly 2017

data-science-on-gcp Source code accompanying book: Data Science on the Google Cloud Platform, 2nd Edition Valliappa Lakshmanan O'Reilly, Jan 2022 Bran

Google Cloud Platform 1.2k Dec 28, 2022
Material for the ros2 crash course

Material for the ros2 crash course

Emmanuel Dean 1 Jan 22, 2022
Project documentation with Markdown.

MkDocs Project documentation with Markdown. View the MkDocs documentation. Project release notes. Visit the MkDocs wiki for community resources, inclu

MkDocs 15.6k Jan 02, 2023
A simple flask application to collect annotations for the Turing Change Point Dataset, a benchmark dataset for change point detection algorithms

AnnotateChange Welcome to the repository of the "AnnotateChange" application. This application was created to collect annotations of time series data

The Alan Turing Institute 16 Jul 21, 2022
Dev Centric Tools for Mkdocs Based Documentation

docutools MkDocs Documentation Tools For Developers This repo is providing a set of plugins for mkdocs material compatible documentation. It is meant

Axiros GmbH 14 Sep 10, 2022
The Python Dict that's better than heroin.

addict addict is a Python module that gives you dictionaries whose values are both gettable and settable using attributes, in addition to standard ite

Mats Julian Olsen 2.3k Dec 22, 2022
FireEye Related Projects

FireEye FireEye Related Projects Tor-IP-Collector Simple python script that will collect a list of TOR IPs from the SecOps Institute Github and inject

Taran Ulrich 2 Nov 12, 2022
A plugin to introduce a generic API for Decompiler support in GEF

decomp2gef A plugin to introduce a generic API for Decompiler support in GEF. Like GEF, the plugin is battery-included and requires no external depend

Zion 379 Jan 08, 2023
More detailed upload statistics for Nicotine+

More Upload Statistics A small plugin for Nicotine+ 3.1+ to create more detailed upload statistics. ⚠ No data previous to enabling this plugin will be

Nick 1 Dec 17, 2021
Pystm32ai - A Python wrapper for the stm32ai command-line tool

PySTM32.AI A python wrapper for the stm32ai command-line tool to analyse deep le

Thibaut Vercueil 5 Jul 28, 2022
A next-generation curated knowledge sharing platform for data scientists and other technical professions.

Knowledge Repo The Knowledge Repo project is focused on facilitating the sharing of knowledge between data scientists and other technical roles using

Airbnb 5.2k Dec 27, 2022