erdantic is a simple tool for drawing entity relationship diagrams (ERDs) for Python data model classes

Overview

erdantic: Entity Relationship Diagrams

Docs Status PyPI conda-forge tests codecov

erdantic is a simple tool for drawing entity relationship diagrams (ERDs) for Python data model classes. Diagrams are rendered using the venerable Graphviz library. Supported data modeling frameworks are:

Features include a convenient CLI, automatic native rendering in Jupyter notebooks, and easy extensibility to other data modeling frameworks. Docstrings are even accessible as tooltips for SVG outputs. Great for adding a simple and clean data model reference to your documentation.

Example diagram created by erdantic

Installation

erdantic's graph modeling depends on pygraphviz and Graphviz, an open-source C library. If you are on Linux or macOS, the easiest way to install everything together is to use conda and conda-forge:

conda install erdantic -c conda-forge

If not using conda, Graphviz must be installed first (before you can install pygraphviz). For recommended options and installation troubleshooting, see the pygraphviz docs. Then to install erdantic and its Python dependencies from PyPI:

pip install erdantic

Development version

You can get the development version from GitHub with:

pip install https://github.com/drivendataorg/erdantic.git#egg=erdantic

Quick Usage

The fastest way to produce a diagram like the above example is to use the erdantic CLI. Simply specify the full dotted path to your data model class and an output path. The rendered format is interpreted from the filename extension.

erdantic erdantic.examples.pydantic.Party -o diagram.png

You can also import the erdantic Python library and use its functions.

import erdantic as erd
from erdantic.examples.pydantic import Party

# Easy one-liner
erd.draw(Party, out="diagram.png")

# Or create a diagram object that you can inspect and do stuff with
diagram = erd.create(Party)
diagram.models
#> [PydanticModel(Adventurer), PydanticModel(Party), PydanticModel(Quest), PydanticModel(QuestGiver)]
diagram.draw("diagram.png")

Check out the "Usage Examples" section of our docs to see more.

Comments
  • Fix issue when a model has a Literal

    Fix issue when a model has a Literal

    A Literal is special in that the argument to the literal is the actual collection of exact values that are permissible to that field.


    Bug description. Start with a file named erdbug.py with the following contents:

    from pydantic import BaseModel
    from typing import Literal
    
    class Species(BaseModel):
        name: str
    
    class PigmentedFlower(BaseModel):
        color: str
        species: Species
    
    class BlackFlower(PigmentedFlower):
        color: Literal["black"] = "black"
    
    class BlueFlower(PigmentedFlower):
        color: Literal["blue"] = "blue"
    

    If we run the following command:

    $ PYTHONPATH=`pwd` erdantic -o diagram.png erdbug.PigmentedFlower erdbug.BlackFlower erdbug.BlueFlower erdbug.Species
    

    We get the following error:

    Traceback (most recent call last):
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/bin/erdantic", line 33, in <module>
        sys.exit(load_entry_point('erdantic', 'console_scripts', 'erdantic')())
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/lib/python3.8/site-packages/typer/main.py", line 214, in __call__
        return get_command(self)(*args, **kwargs)
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
        return self.main(*args, **kwargs)
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/lib/python3.8/site-packages/click/core.py", line 1055, in main
        rv = self.invoke(ctx)
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/lib/python3.8/site-packages/click/core.py", line 760, in invoke
        return __callback(*args, **kwargs)
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/lib/python3.8/site-packages/typer/main.py", line 500, in wrapper
        return callback(**use_params)  # type: ignore
      File "/Users/swails/src/entos/erdantic/erdantic/cli.py", line 87, in main
        diagram = create(*model_classes, termini=termini_classes)
      File "/Users/swails/src/entos/erdantic/erdantic/erd.py", line 192, in create
        search_composition_graph(model=model, seen_models=seen_models, seen_edges=seen_edges)
      File "/Users/swails/src/entos/erdantic/erdantic/erd.py", line 245, in search_composition_graph
        raise StringForwardRefError(
    erdantic.exceptions.StringForwardRefError: Forward reference 'black' for field 'color' on model 'BlackFlower' is a string literal and not a typing.ForwardRef object. erdantic is unable to handle forward references that aren't transformed into typing.ForwardRef. Declare explicitly with 'typing.ForwardRef("black", is_argument=False)'.
    

    The problem is that it's trying to evaluate the value passed to the Literal as a forward reference if it's a string, when in reality Literal is a special typing utility to indicate that the variable carries a specific value.

    After this PR, the following diagram is generated:

    diagram

    opened by swails 7
  • Support docstrings from pydantic.Field descriptions

    Support docstrings from pydantic.Field descriptions

    Adds support for showing field documentation from Pydantic models with descriptions set with Field(description=...) in SVG tooltips. This will add an "Attributes" section to the tooltip using Google-style docstring format and lists fields where the description keyword argument is used.

    Here's an example:

    from typing import Any, List, Optional
    
    import erdantic as erd
    from pydantic import BaseModel, Field
    
    class MyClassWithDescriptions(BaseModel):
        """This is the docstring for my class with descriptions."""
    
        hint_only: str
        has_descr_no_default: List[int] = Field(description="An array of numbers.")
        has_descr_ellipsis_default: List[int] = Field(..., description="Another array of numbers.")
        no_descr_has_default: Any = Field(10)
        has_descr_has_default: Optional[str] = Field(None, description="An optional string.")
    
    diagram = erd.create(MyClassWithDescriptions)
    
    print(diagram.models[0].docstring)
    #> __reprex__.MyClassWithDescriptions
    #> 
    #> This is the docstring for my class with descriptions.
    #> 
    #> Attributes:
    #>     has_descr_no_default (List[int]): An array of numbers.
    #>     has_descr_ellipsis_default (List[int]): Another array of numbers.
    #>     has_descr_has_default (Optional[str]): An optional string. Default is None.
    
    diagram.draw("myclasswithdescriptions.svg")
    #> None
    

    Created at 2021-11-06 11:17:05 EDT by reprexlite v0.4.3

    It doesn't seem like GitHub lets me upload an SVG. Here's a Google Drive link to the output: https://drive.google.com/uc?id=1hGhaq-pLcBP7EanA7uBos8QBuzreSEVv&export=download

    opened by jayqi 5
  • Handle forward references

    Handle forward references

    • Fixed handling of forward references in field type declarations. Evaluated forward references will be properly identified. Forward references not converted to typing.ForwardRef will throw a StringForwardRefError with instructions for how to resolve. Unevaluated forward references will throw an UnevaluatedForwardRefError with instructions for how to resolve.
    • Changed name of erdantic.errors module to erdantic.exceptions.
    • Added new ErdanticException base class from which other exceptions raised within the erdantic library are subclassed from. Changed several existing ValueError exceptions to new exception classes that subclass both ErdanticException and ValueError.
    • Changed __lt__ method on Model and Edge to return NotImplemented instead of raising an exception to follow typical convention for unsupported input types.

    Example

    import dataclasses
    import typing
    
    import erdantic as erd
    
    
    @dataclasses.dataclass
    class A:
        bees: typing.List['B']
    
    
    @dataclasses.dataclass
    class B:
        x: int
    
    # Unevaluated forward ref will error
    diagram = erd.create(A)
    #> Traceback (most recent call last):
    #>   File "<string>", line 2, in <module>
    #>   File "/Users/jqi/repos/erdantic/erdantic/erd.py", line 190, in create
    #>     search_composition_graph(model=model, seen_models=seen_models, seen_edges=seen_edges)
    #>   File "/Users/jqi/repos/erdantic/erdantic/erd.py", line 239, in search_composition_graph
    #>     raise UnevaluatedForwardRefError(
    #> erdantic.exceptions.UnevaluatedForwardRefError: Unevaluated forward reference 'B' for field bees on model A. Call 'typing.get_type_hints' on your dataclass after creating it to resolve.
    
    # Force evaluation with typing.get_type_hints and then it will be fine after that
    _ = typing.get_type_hints(A, localns=locals())
    diagram = erd.create(A)
    diagram.edges
    #> [ Edge(source=DataClassModel(A), source_field=<DataClassField: 'bees', List[B]>, target=DataClassModel(B))]
    diagram.draw("diagram.png")
    

    Created at 2021-10-28 00:48:39 EDT by reprexlite v0.4.2

    diagram

    Closes #40

    opened by jayqi 4
  • ERROR: Cannot unpack file pip-unpack-fpbmoqaz\erdantic.git (content-type: text/html; charset=utf-8); cannot detect archive format

    ERROR: Cannot unpack file pip-unpack-fpbmoqaz\erdantic.git (content-type: text/html; charset=utf-8); cannot detect archive format

    Microsoft Windows [Version 10.0.22000.1098]
    (c) Microsoft Corporation. All rights reserved.
    
    C:\Users\Max>pip install https://github.com/drivendataorg/erdantic.git#egg=erdantic
    Collecting erdantic
      Downloading https://github.com/drivendataorg/erdantic.git
         | 214.4 kB 29.0 kB/s 0:00:07
      ERROR: Cannot unpack file C:\Users\Max\AppData\Local\Temp\pip-unpack-fpbmoqaz\erdantic.git (downloaded from C:\Users\Max\AppData\Local\Temp\pip-install-6ohmio06\erdantic_d45a27d4d8f1499baaee813a654749bf, content-type: text/html; charset=utf-8); cannot detect archive format
    ERROR: Cannot determine archive format of C:\Users\Max\AppData\Local\Temp\pip-install-6ohmio06\erdantic_d45a27d4d8f1499baaee813a654749bf
    
    opened by BaseMax 3
  • Follow edges in a graph automatically

    Follow edges in a graph automatically

    test case:

    https://github.com/adsharma/fquery/blob/master/tests/mock_user.py

    Right now, I get only one class in the diagram, namely: MockUser. Would like to see "friends" and "reviews" edges/objects in the output.

    opened by adsharma 3
  • Show docstrings in rendered tables

    Show docstrings in rendered tables

    Sometimes data classes have docstrings, for the overall class or for individual attributes. It may be useful to be able to show those docstrings.

    How the visual design will work is an open question. The tables are currently compact and easy to read—it may require some creativity to add docstrings in a way that don't detract too much from that.

    enhancement help wanted 
    opened by jayqi 3
  • Hold back nbconvert until rendering SVG is fixed

    Hold back nbconvert until rendering SVG is fixed

    This is not an mkdocs-jupyter problem (https://github.com/danielfrg/mkdocs-jupyter/issues/92), instead it is a problem with nbconvert.

    Looks like there was a partial fix here: https://github.com/jupyter/nbconvert/pull/1837 And finishing that is tracked here: https://github.com/jupyter/nbconvert/issues/1849

    If repro'd this locally with 6.5, but the docs built successfully at 6.4 Pinning to a lower version for now and will update #68 to track removing this pin when the nbconvert issue is fixed.

    opened by pjbull 2
  • Clean up old Python 3.6 code paths

    Clean up old Python 3.6 code paths

    There are a few code paths for handling Python 3.6 typing library quirks. These should no longer be necessary.

    Also minor documentation changes:

    • Switches mkdocstrings handler back to python-legacy. The new python handler doesn't yet properly handle properties. https://github.com/mkdocstrings/python/issues/9
    • Adds TOC entries for modules so they show up in the Intersphinx inventory file
    opened by jayqi 2
  • Add python extra for mkdocstrings

    Add python extra for mkdocstrings

    mkdocstrings 0.19.0 requires that the python extra be explicitly installed.

    Also, to make tests pass with this version, we needed to do #51 as well and drop support for Python 3.6.

    Closes #55 Closes #51

    opened by pjbull 2
  • Support for Pandera

    Support for Pandera

    Pandera allows to create schema's and validations for Pandas dataframes. Is it possible that the erdantic tool supports the Pandera schema's? These can be exported to YAML, so perhaps that's an accessible way to read the schema's?

    opened by ba-tno 2
  • Add versioned docs

    Add versioned docs

    Adds versioned docs using mike.

    • Merges to main deploy as the dev version.
    • Releases deploy as <major>.<minor> version, overwriting previous versions that have the same major and minor. The default alias stable is updated to point at the new release.
    opened by jayqi 2
  • No module found error without setting PYTHONPATH

    No module found error without setting PYTHONPATH

    Logging this issue so that other people don't have to suffer as much as I did.

    I copied the same dataclass example into a newdataclass.py python file and ran erdantic newdataclass -o diagram.png and got the following error.

    After about 30 mins of debugging by running

    from importlib import import_module
    import_module("newdataclass")
    

    I figured it was because PYTHONPATH wasn't set to the current directory.

    Solutions

    1. Updating the documentation to mention this explicitly
    2. Better error message. Only on scrolling I found a whole traceback of exceptions. Importing module, importing class, etc.,

    Error Log

    Traceback (most recent call last):
    
      File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/erdantic/cli.py", line 131, in import_object_from_name
        return import_module(full_obj_name)
    
      File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    
      File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
    
      File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
    
      File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
    
    ModuleNotFoundError: No module named 'newdataclass'
    
    
    During handling of the above exception, another exception occurred:
    
    
    Traceback (most recent call last):
    
      File "/Users/bhavaniravi/.virtualenvs/python-everyday/bin/erdantic", line 8, in <module>
        sys.exit(app())
    
      File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/erdantic/cli.py", line 108, in main
        model_or_module_objs = [import_object_from_name(mm) for mm in models_or_modules]
    
      File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/erdantic/cli.py", line 108, in <listcomp>
        model_or_module_objs = [import_object_from_name(mm) for mm in models_or_modules]
    
      File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/erdantic/cli.py", line 135, in import_object_from_name
        module_name, obj_name = full_obj_name.rsplit(".", 1)
    
    ValueError: not enough values to unpack (expected 2, got 1)
    
    opened by bhavaniravi 1
  • NBConvert does not handle SVG correctly causing docs build to fail

    NBConvert does not handle SVG correctly causing docs build to fail

    opened by github-actions[bot] 10
  • PlantUML generation?

    PlantUML generation?

    When I stumbled on this project, what I was really looking for was a way to generate an ERD in PlantUML. I've implemented that for personal use, but I'm happy to contribute the work back if it would be considered for inclusion. The changes are quite small.

    This was a remarkably easy code base to grok and dig into, so kudos on a useful project!

    opened by swails 1
  • Separating class and its inherited class

    Separating class and its inherited class

    As a user I want to see the relation between a class and its inherited class and also separate the members where it should be So that you will get a correct UML view of the classes.

    example: https://github.com/drivendataorg/erdantic/blob/main/erdantic/examples/pydantic.py

    Additional class:

    class PlannedParty(Party):
        """A planned group of adventurers finding themselves doing and saying things altogether unexpected.
        Attributes:
            name (str): Name that party is known by
            formed_datetime (datetime): Timestamp of when the party was formed
            members (List[Adventurer]): Adventurers that belong to this party
            active_quest (Optional[Quest]): Current quest that party is actively tackling
            planned_quests (List[Quest]): A list of quest that party which can be tackled
        """
    
        planned_quests: List[Quest] = []
    

    (Although this is an erd package, it can have added value. I understand if this issue is closed as it might not fit the vision.)

    opened by zamiramir 0
  • Write tests against static rendered png and svg outputs

    Write tests against static rendered png and svg outputs

    Right now we don't fully check rendered content on diagrams, but we should.

    ~We should do this in a parameterized way to reduce the amount of testing code to be maintained. We can use pytest's fixtures to load the example classes and/or created diagram objects.~ Addressed in #21.

    #21 addressed creating the static outputs, and has a test that checks the against the static DOT files. Still need tests that check png and svg formats.

    enhancement 
    opened by jayqi 0
Releases(v0.5.0)
  • v0.5.0(Jul 30, 2022)

    • Removed support for Python 3.6. (Issue #51, PR #56)
    • Added support for modules as inputs to all entrypoints to diagram creation (create, draw, to_dot, CLI). For all modules passed, erdantic will find all supported data model classes in each module. (Issue #23, PR #58)
      • Added new parameter limit_search_models_to to all entrypoints to allow for limiting which data model classes will be yielded from searching a module.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Apr 8, 2022)

  • v0.4.0(Nov 6, 2021)

    • Added support for showing field documentation from Pydantic models with descriptions set with Field(description=...) in SVG tooltips. This will add an "Attributes" section to the tooltip using Google-style docstring format and lists fields where the description keyword argument is used. (Issue #8, PR #42)
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Oct 29, 2021)

    • Fixed handling of forward references in field type declarations. Evaluated forward references will be properly identified. Forward references not converted to typing.ForwardRef will throw a StringForwardRefError with instructions for how to resolve. Unevaluated forward references will throw an UnevaluatedForwardRefError with instructions for how to resolve. See new documentation for more details. (Issue #40, PR #41)
    • Changed name of erdantic.errors module to erdantic.exceptions. (PR #41)
    • Added new ErdanticException base class from which other exceptions raised within the erdantic library are subclassed from. Changed several existing ValueError exceptions to new exception classes that subclass both ErdanticException and ValueError. (PR #41)
    • Changed __lt__ method on Model and Edge to return NotImplemented instead of raising an exception to follow typical convention for unsupported input types. (PR #41)
    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Feb 17, 2021)

  • v0.2.0(Feb 15, 2021)

  • v0.1.2(Feb 11, 2021)

    • Fixed bug where Pydantic fields were missing generics in their type annotations. (#19)
    • Add tests against static rendered DOT output. Change adapter tests to use parameterized fixtures. (#21)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.1(Feb 11, 2021)

  • v0.1.0(Feb 10, 2021)

Owner
DrivenData
Data science competitions for social good.
DrivenData
Create Badges with stats of Scratch User, Project and Studio. Use those badges in Github readmes, etc.

Scratch-Stats-Badge Create customized Badges with stats of Scratch User, Studio or Project. Use those badges in Github readmes, etc. Examples Document

Siddhesh Chavan 5 Aug 28, 2022
A Simple Flask-Plotly Example for NTU 110-1 DSSI Class

A Simple Flask-Plotly Example for NTU 110-1 DSSI Class Live Demo Prerequisites We will use Flask and Ploty to build a Flask application. If you haven'

Ting Ni Wu 1 Dec 11, 2021
The plottify package is makes matplotlib plots more legible

plottify The plottify package is makes matplotlib plots more legible. It's a thin wrapper around matplotlib that automatically adjusts font sizes, sca

Andy Jones 97 Nov 04, 2022
Simple plotting for Python. Python wrapper for D3xter - render charts in the browser with simple Python syntax.

PyDexter Simple plotting for Python. Python wrapper for D3xter - render charts in the browser with simple Python syntax. Setup $ pip install PyDexter

D3xter 31 Mar 06, 2021
FURY - A software library for scientific visualization in Python

Free Unified Rendering in Python A software library for scientific visualization in Python. General Information • Key Features • Installation • How to

169 Dec 21, 2022
Piglet-shaders - PoC of custom shaders for Piglet

Piglet custom shader PoC This is a PoC for compiling Piglet fragment shaders usi

6 Mar 10, 2022
Render Jupyter notebook in the terminal

jut - JUpyter notebook Terminal viewer. The command line tool view the IPython/Jupyter notebook in the terminal. Install pip install jut Usage $jut --

Kracekumar 169 Dec 27, 2022
Collection of data visualizing projects through Tableau, Data Wrapper, and Power BI

Data-Visualization-Projects Collection of data visualizing projects through Tableau, Data Wrapper, and Power BI Indigenous-Brands-Social-Movements Pyt

Jinwoo(Roy) Yoon 1 Feb 05, 2022
Here I plotted data for the average test scores across schools and class sizes across school districts.

HW_02 Here I plotted data for the average test scores across schools and class sizes across school districts. Average Test Score by Race This graph re

7 Oct 27, 2021
`charts.css.py` brings `charts.css` to Python. Online documentation and samples is available at the link below.

charts.css.py charts.css.py provides a python API to convert your 2-dimension data lists into html snippet, which will be rendered into charts by CSS,

Ray Luo 3 Sep 23, 2021
The Metabolomics Integrator (MINT) is a post-processing tool for liquid chromatography-mass spectrometry (LCMS) based metabolomics.

MINT (Metabolomics Integrator) The Metabolomics Integrator (MINT) is a post-processing tool for liquid chromatography-mass spectrometry (LCMS) based m

Sören Wacker 0 May 04, 2022
Visualization Library

CamViz Overview // Installation // Demos // License Overview CamViz is a visualization library developed by the TRI-ML team with the goal of providing

Toyota Research Institute - Machine Learning 67 Nov 24, 2022
Handout for the tutorial "Creating publication-quality figures with matplotlib"

Handout for the tutorial "Creating publication-quality figures with matplotlib"

JB Mouret 1.9k Jan 02, 2023
649 Pokémon palettes as CSVs, with a Python lib to turn names/IDs into palettes, or MatPlotLib compatible ListedColormaps.

PokePalette 649 Pokémon, broken down into CSVs of their RGB colour palettes. Complete with a Python library to convert names or Pokédex IDs into eithe

11 Dec 05, 2022
Declarative statistical visualization library for Python

Altair http://altair-viz.github.io Altair is a declarative statistical visualization library for Python. With Altair, you can spend more time understa

Altair 8k Jan 05, 2023
Implement the Perspective open source code in preparation for data visualization

Task Overview | Installation Instructions | Link to Module 2 Introduction Experience Technology at JP Morgan Chase Try out what real work is like in t

Abdulazeez Jimoh 1 Jan 23, 2022
Runtime analysis of code with plotting

Runtime analysis of code with plotting A quick comparison among Python, Cython, and the C languages A Programming Assignment regarding the Programming

Cena Ashoori 2 Dec 24, 2021
Visualizations of linear algebra algorithms for people who want a deep understanding

Visualising algorithms on symmetric matrices Examples QR algorithm and LR algorithm Here, we have a GIF animation of an interactive visualisation of t

ogogmad 3 May 05, 2022
High-level geospatial data visualization library for Python.

geoplot: geospatial data visualization geoplot is a high-level Python geospatial plotting library. It's an extension to cartopy and matplotlib which m

Aleksey Bilogur 1k Jan 01, 2023
Make scripted visualizations in blender

Scripted visualizations in blender The goal of this project is to script 3D scientific visualizations using blender. To achieve this, we aim to bring

Praneeth Namburi 10 Jun 01, 2022