A fast unobtrusive MongoDB ODM for Python.

Overview

MongoFrames logo

MongoFrames

Build Status Join the chat at https://gitter.im/GetmeUK/ContentTools

MongoFrames is a fast unobtrusive MongoDB ODM for Python designed to fit into a workflow not dictate one. Documentation is available at MongoFrames.com and includes:

Installation

We recommend you use virtualenv or virtualenvwrapper to create a virtual environment for your install. There are several options for installing MongoFrames:

  • pip install MongoFrames (recommended)
  • easy_install MongoFrames
  • Download the source and run python setup.py install

Dependencies

10 second example

from mongoframes import *

# Define some document frames (a.k.a models)
class Dragon(Frame):
    _fields = {'name', 'loot'}

class Item(Frame):
    _fields = {'desc', 'value'}

# Create a dragon and loot to boot
Item(desc='Sock', value=1).insert()
Item(desc='Diamond', value=100).insert()
Dragon(name='Burt', loot=Item.many()).insert()

# Have Burt boast about his loot
burt = Dragon.one(Q.name == 'Burt', projection={'loot': {'$ref': Item}})
for item in burt.loot:
    print('I have a {0.name} worth {0.value} crown'.format(item))

Testing

To test the library you'll need to be running a local instance of MongoDB on the standard port.

To run the test suite: py.test To run the test suite on each supported version of Python: tox

Helpful organizations

MongoFrames is developed using a number of tools & services provided for free by nice folks at organizations committed to supporting open-source projects including GitHub and Travis CI.

Comments
  • MongoFrames doesn't delete document

    MongoFrames doesn't delete document

    Hi.

    A strange issue: the Frame class doesn't delete the document from the db. Insert and update works fine.

                str_id = '{}_{}_{}_{}_{}'.format(
                    data_before.get('CodImpianto'),
                    data_before.get('Armadio'),
                    data_before.get('DateVar'),
                    data_before.get('ShelfEvent'),
                    data_before.get('Counter'),
                )
                log_id = sha224(str_id.encode()).hexdigest()
                previous_record = SDClosetLogs().by_id(log_id)
    
                if previous_record is not None:
                    logger.debug('Deleting document {}.'.format(previous_record._id))
                    try:
                        # previous_record.delete()
                        SDClosetLogs.delete(SDClosetLogs().by_id(log_id))
                    except Exception as e:
                        logger.error('Cannot delete document {}. Error: {}'.format(previous_record._id, e))
                        return False
    

    This is the log: 2016-12-11 16:30:36 DEBUG 3700 spy_replicator.etl_modules.etl_spydoc etl_spydoc.py:219 => Deleting document c4e599fb7467db9f868746631b7daea81a32fc63a2be05871b5710d8. (no exception logged) I tried using both the previous_record instance and the general Frame class.

    This is the documents in collections pre and after the delete operation: image

    This is the Frame class

    class SDClosetLogs(Frame):
        _db = 'spy'
        _collection = 'logs'
        _fields = {
            '_id',
            'date',
            'type',
            'subtype',
            'installation',
            'closet',
            'shelf_event',
            'tag_id',
            'tag_user',
            'description',
            'counter'
        }
    
    question 
    opened by njordr 11
  • frameless with check for defined fields

    frameless with check for defined fields

    Hi.

    I've created a frameless class as stated here: http://mongoframes.com/snippets/frameless and it works like a charm :)

    What about to introduce a check for the defined fields?

    Here is an example of my frameless class:

    class InfluxMetric(Frameless):
        _collection = 'influxdb_metrics'
        _fields = {
            '_id',
            'test'
        }
    

    Is it possible to force the new class to fill-in at least the two defined fields (_id and test)?

    Thanks, Giovanni

    question 
    opened by njordr 5
  • edit/remove SubFrame

    edit/remove SubFrame

    Hi.

    How to delete/edit SubFrame?

    Let me explain. I have these Frames/SubFrames:

    class Room(SubFrame):
        _fields = {
            'room_id',
            'name'
        }
    
    
    class Floor(SubFrame):
        _default_projection = {'rooms': {'$sub': Room}}
        _fields = {
            'floor_id',
            'name',
            'rooms'
        }
    
    
    class Building(Frame):
        _db = 'spy'
        _collection = 'buildings'
        _default_projection = {'floors': {'$sub': Floor}}
        _fields = {
            '_id',
            'building_id',
            'name',
            'street',
            'city',
            'zipcode',
            'province',
            'state',
            'phone',
            'lat_lon',
            'notes',
            'floors'
        }
    
    

    Then create the building

    building = Building(
        _id='id',
        building_id='building_id',
        name='test',
        city='Milan'
    )
    

    Create rooms and floors

    room01 = Room(
        name='test room 1',
        room_id=1
    )
    
    room02 = Room(
        name='test room 2',
        room_id=2
    )
    
    room03 = Room(
        name='test room 3',
        room_id=3
    )
    
    room04 = Room(
        name='test room 4',
        room_id=4
    )
    
    floor01 = Floor(
        floor_id=1,
        name='First floor',
        rooms=[room01, room02]
    )
    
    floor02 = Floor(
        floor_id=2,
        name='Second floor',
        rooms=[room03, room04]
    )
    
    building.floors = [floor01, floor02]
    

    Now what if:

    1. I want to edit a floor/room with a specific floor_id/room_id
    2. I want to delete a floor/room with a specific floor_id/room_id

    Thanks

    question 
    opened by njordr 3
  • issue with Frame.by_id

    issue with Frame.by_id

    Hi.

    I've this document: { "_id" : "report_xyz", "name" : "XYZ.COM", "site24" : { "monitors" : [ NumberLong(27498547952905495), NumberLong(18975984759794899), NumberLong(19875894954998994), NumberLong(14987598437950899), NumberLong(17498574975919809) ] } }

    If I try to get it with by_id this is the error: bson.errors.InvalidId: 'report_xyz' is not a valid ObjectId, it must be a 12-byte input or a 24-character hex string

    Giovanni

    bug 
    opened by njordr 3
  • ObjectIds are distinct from document _id fields

    ObjectIds are distinct from document _id fields

    Hi, I notice the guide says, "Projections also allow reference fields (fields containing an ObjectID) to be dereferenced." This implies that a field is a reference to another document if and only if it contains an ObjectId. In fact, a field can contain an ObjectId and not be a reference to another document. Additionally, a field could contain any other type (int, string, subdocument, ...) except array, and yet be a reference to another document.

    Documents' _id fields can contain any type but array:

    https://docs.mongodb.com/manual/core/document/#the-id-field

    question 
    opened by ajdavis 3
  • documents with random fields name

    documents with random fields name

    Hi.

    How can I map a document like this?

    { "_id" : "locations_names", "1" : "California - US", "2" : "New Jersey - US", "3" : "Singapore - SG", "4" : "Texas - US", "5" : "Rotterdam - NL", "6" : "London - UK", "7" : "Dallas - US", "8" : "Seattle - US", "9" : "Chicago - US", "10" : "Cologne - DE", "11" : "Johannesburg - ZA", "12" : "Melbourne - AUS", "13" : "Nagano - JP", "14" : "Shanghai - CHN", "15" : "Chennai - IN", "16" : "Rio de Janeiro - BR", "17" : "Stockholm - SWE", "18" : "Paris - FR", "19" : "Virginia - US", "20" : "Ireland - IE", "21" : "Hong Kong - HK", "22" : "Sao Paulo - BR", "23" : "Barcelona - ES", "24" : "Milano - IT", "25" : "New York - US", "26" : "Los Angeles - US", "27" : "Denver - US", "28" : "Kansas - US", "29" : "Munich - DE", "30" : "Washington - US", "31" : "Montreal - CA", "32" : "Phoenix - US", "33" : "Mumbai - IN", "34" : "Istanbul - TR", "35" : "Tel Aviv - IL", "36" : "Sydney - AUS", "37" : "Auckland - NZ", "38" : "Atlanta - US", "39" : "Brussels - BE", "40" : "Toronto - CA", "41" : "Copenhagen - DA", "42" : "Vienna - AT", "43" : "Zurich - CH", "44" : "Warsaw - PL", "45" : "Bucharest - RO", "46" : "Moscow - RU", "47" : "Beijing - CHN", "48" : "Hangzhou City - CHN", "49" : "Qingdao City - CHN", "50" : "Miami - US", "52" : "Tokyo - JP", "55" : "Dubai - UAE", "56" : "Queretaro - MEX", "57" : "Falkenstein - DE", "58" : "Strasbourg - FR", "59" : "Bengaluru - IN", "source" : "site24" }

    The fields name are not static and they can grow in the future.

    If in the class I only map the ones that are static class LocationInfo(Frame): _collection = 'locations_info' _fields = { 'source' }

    Than I've got only them in the query result {'_document': {'_id': 'locations_names', 'source': 'site24'}}

    Thanks

    question 
    opened by njordr 2
  • Performance improvement by using ctypes

    Performance improvement by using ctypes

    Hi, I had the idea to integrate ctypes/Structure. I imagined that somehow:

    1. Get the meta data of the collection (field -> type)
    2. Build a object template by mongodb -> ctypes mapping definition
    3. Use the ctypes object template to construct the objects

    I've already started my own experiment and I have to say that the performance gain is enormous:

    class Attribute(Frame):
        __metaclass__ = ctypes.Structure
        _collection = 'system_attribute'
        _fields = {
            'key',
            'label',
            'description',
            'metadata',
            'field_class',
            'multivalue',
            'tags',
        }
    
        _fields_ = [
            ('key', ctypes.c_char),
            ('label', ctypes.c_char),
            ('description', ctypes.c_char),
            ('field_class', ctypes.c_char),
            ('multivalue', ctypes.c_bool),
            ('tags', ctypes.Union),
        ]
    

    Let me know what you think

    opened by sly8787 1
  • Add support to query builder for creating sort arguments

    Add support to query builder for creating sort arguments

    At the moment to specify the order of a set of results we'd do something like:

    articles = Ariticle.many(sort=[('publish_date', DESC), ('title', ASC)])
    

    I think it would be a useful addition to the query builder if we would instead build these sort lists along the lines that we build queries, e.g:

    articles = Ariticle.many(sort=[Q.publish_date.desc, Q.title])
    
    enhancement 
    opened by anthonyjb 1
  • Add a Gitter chat badge to README.md

    Add a Gitter chat badge to README.md

    GetmeUK/MongoFrames now has a Chat Room on Gitter

    @anthonyjb has just created a chat room. You can visit it here: https://gitter.im/GetmeUK/MongoFrames.

    This pull-request adds this badge to your README.md:

    Gitter

    If my aim is a little off, please let me know.

    Happy chatting.

    PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

    opened by gitter-badger 0
  • Inconsistent calling of listen against Frame by Factory Blueprint

    Inconsistent calling of listen against Frame by Factory Blueprint

    The Blueprint class triggers fake and faked events against Frame instances inconsistently with the way that other Frame methods trigger events.

    The work around currently is to use a classmethod (to receive the sender) over using a staticmethod. Fixing this issue will break backwards compatibility so I look into a solution that can cater for both in the short term (it's a pretty simple fix to upgrade, however, switch classmethods to staticmethods).

    Note This bug doesn't not affect documented functionality.

    bug 
    opened by anthonyjb 0
  • Support for `TypedFrame` class

    Support for `TypedFrame` class

    Hi. It would be great if MongoFrame implements also a check of field type/structure.

    Let me say I want to have all the documents of a certain type with the same _id structure:

    class Test(Frame):
        _db = 'testdb'
        _collection = 'testcollection'
        _fields = {
            '_id': {
                'template': 'test_id_'
                'type': str
            },
    

    so when I create a new Test class, i could do the following:

    Test(_id=1234).insert()
    

    and the result _id will be "test_id_1234"

    In this way it's more simple also to document the structure of the document.

    Let me know if this could be in the philosophy of MongoFrames or it's better to use an external validator. In the latter case, any advice?

    Thanks

    enhancement 
    opened by njordr 2
Releases(1.3.6)
  • 1.3.6(Jun 6, 2020)

    Thanks to @tylerganter for the pull request to significantly improve the performance of update_many.

    In this release:

    • update_many now uses bulk writes which significantly improves the performance of calling update_many.
    • unset and unset_methods have now been added to the Frame class.
    • A fix has been applied to the ElemMatch function which previously would raise an error if a raw condition was given instead of a Condition instance.
    Source code(tar.gz)
    Source code(zip)
  • 1.3.5(Jan 16, 2020)

    It's now possible to set the collection options for a frame within a given context, such as:

    
    with MyFrame.with_options(read_preference=ReadPreference.SECONDARY):
        my_frame = MyFrame.one()
    
    

    This release also sees a move to pymongo>=3.9.0 requirements.

    Source code(tar.gz)
    Source code(zip)
  • 1.3.4(May 25, 2019)

    This release has moved the minimum pymongo requirement to 3.8+. To support this change we've replaced a number of pymongo deprecated method calls, including update with update_one and count with count_documents or estimated_document_count. This does not change theupdate and count methods / API for MongoFrames.

    In addition I fixed an issue where using $slice in a reference projection wasn't correctly, for example in the expression:

        projection={
            'name': True,
            'show_date_str': True,
            'leads': {
                '$ref': User,
                '$slice': 1,
                'first_name': True,
                'last_name': True
            }
        }
    

    This now correctly selects the first Id in the leads field to use as a reference when selecting the assoicated users

    Source code(tar.gz)
    Source code(zip)
  • 1.3.0(Feb 6, 2019)

    Previous to this release projections within $sub or $sub. where simply ignored and the projection was converted to True for the key, for example:

    lairs = Lair.many(
        projection={
            '_id': True,
            'inventory': {
                '$sub': Inventory,
                'gold': True
        }
    )
    

    Previously used the projection:

    {
         '_id: True, 
         'inventory': True
    }
    

    In this release this will now project to:

    {
         '_id: True, 
         'inventory.gold': True
    }
    

    NOTE: This release is a minor release as whilst this should effect existing code if you've previous been setting projections (which simply were ignored before) this could potentially break this code.

    Source code(tar.gz)
    Source code(zip)
  • 1.2.13(Aug 15, 2018)

    For example this is now possible:

    projection={
        'employees': {
            '$ref': Employee,
            '$slice': 2,
           'name': True
        }
    }
    

    So get the first 2 employee Ids in an array of employee Ids stored against the field employees and project them as Employee documents selecting only the name field for each.

    Source code(tar.gz)
    Source code(zip)
  • 1.2.10(Apr 29, 2018)

    As Collation is now passed through the standard mongoframes import the pymongo dependency need to be updated (thanks to the @TheGent for flagging this).

    Source code(tar.gz)
    Source code(zip)
  • 1.2.9(Mar 23, 2018)

  • 1.2.8(Oct 24, 2017)

  • 1.2.7(Aug 28, 2017)

    New features

    • The $sub. projection keyword now supports for dictionaries containing list values not just dictionary values.
    • New functionality added to make sort by instructions easier to define, e.g:
    articles = Article.many(Q.published == True, sort=SortBy(Q.published_date.desc, Q.title))
    
    Source code(tar.gz)
    Source code(zip)
  • 1.2.6(Feb 16, 2017)

  • 1.2.4(Aug 16, 2016)

    This really should have been a minor release instead of a bug fix version increment as it breaks compatibility with previous releases - however since the changes relate entirely to the factory model which is not currently documented or promoted we should be safe as I believe that's only being used internally at Getme.

    Changes:

    • Presets have been removed as a concept from factories (with hindsight they were more trouble than they were worth).
    • Added document property to makers which is set to the current target document when blueprints are using makers to assemble or finish a document.
    Source code(tar.gz)
    Source code(zip)
  • 1.2.3(Aug 12, 2016)

    Support for reassembling fields in previously assembled documents/dictionaries has been added to the Factory and Blueprint classes. This helps make it simpler to update fake data with new instructions without having to assemble everything from scratch which is more time consuming and can break existing tests.

    Source code(tar.gz)
    Source code(zip)
  • 1.2.2(Aug 10, 2016)

    Changes:

    • Added RandomReference maker to allow the _id or a document from a specified collection to be selected at random, optionally with a constraint.
    • Removed the issue where insert_many would only accept documents that did not have _id values.
    Source code(tar.gz)
    Source code(zip)
  • 1.2.0(Aug 8, 2016)

    Changes:

    • New interface for blueprints, Blueprint classes are now defined as static classes and instructions as attributes not a separate dictionary.
    • Added Int maker class.
    • Added Float maker class.
    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Aug 3, 2016)

    This release of MongoFrames sees the introduction of the factory sub-module which provides a set of classes and functions to simplify the process of generating fake data.

    The documentation for the release has not yet been completed and so I'll update the release with a link once this is in place.

    Source code(tar.gz)
    Source code(zip)
  • 1.0.2(Jul 11, 2016)

  • 1.0.1(Jul 6, 2016)

    This makes the following project possible:

    web_order = WebOrder.one(Q._id == ObjectID(...), projection={
            'items': {
                '$sub': Item,
                'product': {
                    '$ref': Product,
                    'name': True 
                }
            }
        })
    

    In the example above we select a single web order (Frame) and map all items for the order to an item (SubFrame). Each item has a reference to a product (Frame) which we select limiting the projection to just the product's name.

    Source code(tar.gz)
    Source code(zip)
Owner
getme
a small team of boffins and creatives obsessed with all things web
getme
Py2neo is a comprehensive toolkit for working with Neo4j from within Python applications or from the command line.

Py2neo v3 Py2neo is a client library and toolkit for working with Neo4j from within Python applications and from the command line. The core library ha

64 Oct 14, 2022
a small, expressive orm -- supports postgresql, mysql and sqlite

peewee Peewee is a simple and small ORM. It has few (but expressive) concepts, making it easy to learn and intuitive to use. a small, expressive ORM p

Charles Leifer 9.7k Dec 30, 2022
The JavaScript Database, for Node.js, nw.js, electron and the browser

The JavaScript Database Embedded persistent or in memory database for Node.js, nw.js, Electron and browsers, 100% JavaScript, no binary dependency. AP

Louis Chatriot 13.2k Jan 02, 2023
#crypto #cipher #encode #decode #hash

🌹 CYPHER TOOLS 🌹 Written by TMRSWRR Version 1.0.0 All in one tools for CRYPTOLOGY. Instagram: Capture the Root 🖼️ Screenshots 🖼️ 📹 How to use 📹

50 Dec 23, 2022
Python script to clone SQL dashboard from one workspace to another

Databricks dashboard clone Unofficial project to allow Databricks SQL dashboard copy from one workspace to another. Resource clone Setup: Create a fil

Quentin Ambard 12 Jan 01, 2023
Asynchronous interface for peewee ORM powered by asyncio

peewee-async Asynchronous interface for peewee ORM powered by asyncio. Important notes Since version 0.6.0a only peewee 3.5+ is supported If you still

05Bit 666 Dec 30, 2022
Asynchronous Python client for InfluxDB

aioinflux Asynchronous Python client for InfluxDB. Built on top of aiohttp and asyncio. Aioinflux is an alternative to the official InfluxDB Python cl

Gustavo Bezerra 159 Dec 27, 2022
Pure-python PostgreSQL driver

pg-purepy pg-purepy is a pure-Python PostgreSQL wrapper based on the anyio library. A lot of this library was inspired by the pg8000 library. Credits

Lura Skye 11 May 23, 2022
Simplest SQL mapper in Python, probably

SQL MAPPER Basically what it does is: it executes some SQL thru a database connector you fed it, maps it to some model and gives to u. Also it can cre

2 Nov 07, 2022
GINO Is Not ORM - a Python asyncio ORM on SQLAlchemy core.

GINO - GINO Is Not ORM - is a lightweight asynchronous ORM built on top of SQLAlchemy core for Python asyncio. GINO 1.0 supports only PostgreSQL with

GINO Community 2.5k Dec 27, 2022
Amazon S3 Transfer Manager for Python

s3transfer - An Amazon S3 Transfer Manager for Python S3transfer is a Python library for managing Amazon S3 transfers. Note This project is not curren

the boto project 158 Jan 07, 2023
Pystackql - Python wrapper for StackQL

pystackql - Python Library for StackQL Python wrapper for StackQL Usage from pys

StackQL Studios 6 Jul 01, 2022
PyRemoteSQL is a python SQL client that allows you to connect to your remote server with phpMyAdmin installed.

PyRemoteSQL Python MySQL remote client Basically this is a python SQL client that allows you to connect to your remote server with phpMyAdmin installe

ProbablyX 3 Nov 04, 2022
DBMS Mini-project: Recruitment Management System

# Hire-ME DBMS Mini-project: Recruitment Management System. 💫 ✨ Features Python + MYSQL using mysql.connector library Recruiter and Client Panel Beau

Karan Gandhi 35 Dec 23, 2022
A simple wrapper to make a flat file drop in raplacement for mongodb out of TinyDB

Purpose A simple wrapper to make a drop in replacement for mongodb out of tinydb. This module is an attempt to add an interface familiar to those curr

180 Jan 01, 2023
Databank is an easy-to-use Python library for making raw SQL queries in a multi-threaded environment.

Databank Databank is an easy-to-use Python library for making raw SQL queries in a multi-threaded environment. No ORM, no frills. Thread-safe. Only ra

snapADDY GmbH 4 Apr 04, 2022
Example Python codes that works with MySQL and Excel files (.xlsx)

Python x MySQL x Excel by Zinglecode Example Python codes that do the processes between MySQL database and Excel spreadsheet files. YouTube videos MyS

Potchara Puttawanchai 1 Feb 07, 2022
Pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

AWS Data Wrangler Pandas on AWS Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretMana

Amazon Web Services - Labs 3.3k Dec 31, 2022
Application which allows you to make PostgreSQL databases with Python

Automate PostgreSQL Databases with Python Application which allows you to make PostgreSQL databases with Python I used the psycopg2 library which is u

Marc-Alistair Coffi 0 Dec 31, 2021
Async ORM based on PyPika

PyPika-ORM - ORM for PyPika SQL Query Builder The package gives you ORM for PyPika with asycio support for a range of databases (SQLite, PostgreSQL, M

Kirill Klenov 7 Jun 04, 2022