erdantic is a simple tool for drawing entity relationship diagrams (ERDs) for Python data model classes

Overview

erdantic: Entity Relationship Diagrams

Docs Status PyPI conda-forge tests codecov

erdantic is a simple tool for drawing entity relationship diagrams (ERDs) for Python data model classes. Diagrams are rendered using the venerable Graphviz library. Supported data modeling frameworks are:

Features include a convenient CLI, automatic native rendering in Jupyter notebooks, and easy extensibility to other data modeling frameworks. Docstrings are even accessible as tooltips for SVG outputs. Great for adding a simple and clean data model reference to your documentation.

Example diagram created by erdantic

Installation

erdantic's graph modeling depends on pygraphviz and Graphviz, an open-source C library. If you are on Linux or macOS, the easiest way to install everything together is to use conda and conda-forge:

conda install erdantic -c conda-forge

If not using conda, Graphviz must be installed first (before you can install pygraphviz). For recommended options and installation troubleshooting, see the pygraphviz docs. Then to install erdantic and its Python dependencies from PyPI:

pip install erdantic

Development version

You can get the development version from GitHub with:

pip install https://github.com/drivendataorg/erdantic.git#egg=erdantic

Quick Usage

The fastest way to produce a diagram like the above example is to use the erdantic CLI. Simply specify the full dotted path to your data model class and an output path. The rendered format is interpreted from the filename extension.

erdantic erdantic.examples.pydantic.Party -o diagram.png

You can also import the erdantic Python library and use its functions.

import erdantic as erd
from erdantic.examples.pydantic import Party

# Easy one-liner
erd.draw(Party, out="diagram.png")

# Or create a diagram object that you can inspect and do stuff with
diagram = erd.create(Party)
diagram.models
#> [PydanticModel(Adventurer), PydanticModel(Party), PydanticModel(Quest), PydanticModel(QuestGiver)]
diagram.draw("diagram.png")

Check out the "Usage Examples" section of our docs to see more.

Comments
  • Fix issue when a model has a Literal

    Fix issue when a model has a Literal

    A Literal is special in that the argument to the literal is the actual collection of exact values that are permissible to that field.


    Bug description. Start with a file named erdbug.py with the following contents:

    from pydantic import BaseModel
    from typing import Literal
    
    class Species(BaseModel):
        name: str
    
    class PigmentedFlower(BaseModel):
        color: str
        species: Species
    
    class BlackFlower(PigmentedFlower):
        color: Literal["black"] = "black"
    
    class BlueFlower(PigmentedFlower):
        color: Literal["blue"] = "blue"
    

    If we run the following command:

    $ PYTHONPATH=`pwd` erdantic -o diagram.png erdbug.PigmentedFlower erdbug.BlackFlower erdbug.BlueFlower erdbug.Species
    

    We get the following error:

    Traceback (most recent call last):
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/bin/erdantic", line 33, in <module>
        sys.exit(load_entry_point('erdantic', 'console_scripts', 'erdantic')())
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/lib/python3.8/site-packages/typer/main.py", line 214, in __call__
        return get_command(self)(*args, **kwargs)
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
        return self.main(*args, **kwargs)
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/lib/python3.8/site-packages/click/core.py", line 1055, in main
        rv = self.invoke(ctx)
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/lib/python3.8/site-packages/click/core.py", line 760, in invoke
        return __callback(*args, **kwargs)
      File "/Applications/Anaconda/anaconda3/envs/foundry-dev/lib/python3.8/site-packages/typer/main.py", line 500, in wrapper
        return callback(**use_params)  # type: ignore
      File "/Users/swails/src/entos/erdantic/erdantic/cli.py", line 87, in main
        diagram = create(*model_classes, termini=termini_classes)
      File "/Users/swails/src/entos/erdantic/erdantic/erd.py", line 192, in create
        search_composition_graph(model=model, seen_models=seen_models, seen_edges=seen_edges)
      File "/Users/swails/src/entos/erdantic/erdantic/erd.py", line 245, in search_composition_graph
        raise StringForwardRefError(
    erdantic.exceptions.StringForwardRefError: Forward reference 'black' for field 'color' on model 'BlackFlower' is a string literal and not a typing.ForwardRef object. erdantic is unable to handle forward references that aren't transformed into typing.ForwardRef. Declare explicitly with 'typing.ForwardRef("black", is_argument=False)'.
    

    The problem is that it's trying to evaluate the value passed to the Literal as a forward reference if it's a string, when in reality Literal is a special typing utility to indicate that the variable carries a specific value.

    After this PR, the following diagram is generated:

    diagram

    opened by swails 7
  • Support docstrings from pydantic.Field descriptions

    Support docstrings from pydantic.Field descriptions

    Adds support for showing field documentation from Pydantic models with descriptions set with Field(description=...) in SVG tooltips. This will add an "Attributes" section to the tooltip using Google-style docstring format and lists fields where the description keyword argument is used.

    Here's an example:

    from typing import Any, List, Optional
    
    import erdantic as erd
    from pydantic import BaseModel, Field
    
    class MyClassWithDescriptions(BaseModel):
        """This is the docstring for my class with descriptions."""
    
        hint_only: str
        has_descr_no_default: List[int] = Field(description="An array of numbers.")
        has_descr_ellipsis_default: List[int] = Field(..., description="Another array of numbers.")
        no_descr_has_default: Any = Field(10)
        has_descr_has_default: Optional[str] = Field(None, description="An optional string.")
    
    diagram = erd.create(MyClassWithDescriptions)
    
    print(diagram.models[0].docstring)
    #> __reprex__.MyClassWithDescriptions
    #> 
    #> This is the docstring for my class with descriptions.
    #> 
    #> Attributes:
    #>     has_descr_no_default (List[int]): An array of numbers.
    #>     has_descr_ellipsis_default (List[int]): Another array of numbers.
    #>     has_descr_has_default (Optional[str]): An optional string. Default is None.
    
    diagram.draw("myclasswithdescriptions.svg")
    #> None
    

    Created at 2021-11-06 11:17:05 EDT by reprexlite v0.4.3

    It doesn't seem like GitHub lets me upload an SVG. Here's a Google Drive link to the output: https://drive.google.com/uc?id=1hGhaq-pLcBP7EanA7uBos8QBuzreSEVv&export=download

    opened by jayqi 5
  • Handle forward references

    Handle forward references

    • Fixed handling of forward references in field type declarations. Evaluated forward references will be properly identified. Forward references not converted to typing.ForwardRef will throw a StringForwardRefError with instructions for how to resolve. Unevaluated forward references will throw an UnevaluatedForwardRefError with instructions for how to resolve.
    • Changed name of erdantic.errors module to erdantic.exceptions.
    • Added new ErdanticException base class from which other exceptions raised within the erdantic library are subclassed from. Changed several existing ValueError exceptions to new exception classes that subclass both ErdanticException and ValueError.
    • Changed __lt__ method on Model and Edge to return NotImplemented instead of raising an exception to follow typical convention for unsupported input types.

    Example

    import dataclasses
    import typing
    
    import erdantic as erd
    
    
    @dataclasses.dataclass
    class A:
        bees: typing.List['B']
    
    
    @dataclasses.dataclass
    class B:
        x: int
    
    # Unevaluated forward ref will error
    diagram = erd.create(A)
    #> Traceback (most recent call last):
    #>   File "<string>", line 2, in <module>
    #>   File "/Users/jqi/repos/erdantic/erdantic/erd.py", line 190, in create
    #>     search_composition_graph(model=model, seen_models=seen_models, seen_edges=seen_edges)
    #>   File "/Users/jqi/repos/erdantic/erdantic/erd.py", line 239, in search_composition_graph
    #>     raise UnevaluatedForwardRefError(
    #> erdantic.exceptions.UnevaluatedForwardRefError: Unevaluated forward reference 'B' for field bees on model A. Call 'typing.get_type_hints' on your dataclass after creating it to resolve.
    
    # Force evaluation with typing.get_type_hints and then it will be fine after that
    _ = typing.get_type_hints(A, localns=locals())
    diagram = erd.create(A)
    diagram.edges
    #> [ Edge(source=DataClassModel(A), source_field=<DataClassField: 'bees', List[B]>, target=DataClassModel(B))]
    diagram.draw("diagram.png")
    

    Created at 2021-10-28 00:48:39 EDT by reprexlite v0.4.2

    diagram

    Closes #40

    opened by jayqi 4
  • ERROR: Cannot unpack file pip-unpack-fpbmoqaz\erdantic.git (content-type: text/html; charset=utf-8); cannot detect archive format

    ERROR: Cannot unpack file pip-unpack-fpbmoqaz\erdantic.git (content-type: text/html; charset=utf-8); cannot detect archive format

    Microsoft Windows [Version 10.0.22000.1098]
    (c) Microsoft Corporation. All rights reserved.
    
    C:\Users\Max>pip install https://github.com/drivendataorg/erdantic.git#egg=erdantic
    Collecting erdantic
      Downloading https://github.com/drivendataorg/erdantic.git
         | 214.4 kB 29.0 kB/s 0:00:07
      ERROR: Cannot unpack file C:\Users\Max\AppData\Local\Temp\pip-unpack-fpbmoqaz\erdantic.git (downloaded from C:\Users\Max\AppData\Local\Temp\pip-install-6ohmio06\erdantic_d45a27d4d8f1499baaee813a654749bf, content-type: text/html; charset=utf-8); cannot detect archive format
    ERROR: Cannot determine archive format of C:\Users\Max\AppData\Local\Temp\pip-install-6ohmio06\erdantic_d45a27d4d8f1499baaee813a654749bf
    
    opened by BaseMax 3
  • Follow edges in a graph automatically

    Follow edges in a graph automatically

    test case:

    https://github.com/adsharma/fquery/blob/master/tests/mock_user.py

    Right now, I get only one class in the diagram, namely: MockUser. Would like to see "friends" and "reviews" edges/objects in the output.

    opened by adsharma 3
  • Show docstrings in rendered tables

    Show docstrings in rendered tables

    Sometimes data classes have docstrings, for the overall class or for individual attributes. It may be useful to be able to show those docstrings.

    How the visual design will work is an open question. The tables are currently compact and easy to read—it may require some creativity to add docstrings in a way that don't detract too much from that.

    enhancement help wanted 
    opened by jayqi 3
  • Hold back nbconvert until rendering SVG is fixed

    Hold back nbconvert until rendering SVG is fixed

    This is not an mkdocs-jupyter problem (https://github.com/danielfrg/mkdocs-jupyter/issues/92), instead it is a problem with nbconvert.

    Looks like there was a partial fix here: https://github.com/jupyter/nbconvert/pull/1837 And finishing that is tracked here: https://github.com/jupyter/nbconvert/issues/1849

    If repro'd this locally with 6.5, but the docs built successfully at 6.4 Pinning to a lower version for now and will update #68 to track removing this pin when the nbconvert issue is fixed.

    opened by pjbull 2
  • Clean up old Python 3.6 code paths

    Clean up old Python 3.6 code paths

    There are a few code paths for handling Python 3.6 typing library quirks. These should no longer be necessary.

    Also minor documentation changes:

    • Switches mkdocstrings handler back to python-legacy. The new python handler doesn't yet properly handle properties. https://github.com/mkdocstrings/python/issues/9
    • Adds TOC entries for modules so they show up in the Intersphinx inventory file
    opened by jayqi 2
  • Add python extra for mkdocstrings

    Add python extra for mkdocstrings

    mkdocstrings 0.19.0 requires that the python extra be explicitly installed.

    Also, to make tests pass with this version, we needed to do #51 as well and drop support for Python 3.6.

    Closes #55 Closes #51

    opened by pjbull 2
  • Support for Pandera

    Support for Pandera

    Pandera allows to create schema's and validations for Pandas dataframes. Is it possible that the erdantic tool supports the Pandera schema's? These can be exported to YAML, so perhaps that's an accessible way to read the schema's?

    opened by ba-tno 2
  • Add versioned docs

    Add versioned docs

    Adds versioned docs using mike.

    • Merges to main deploy as the dev version.
    • Releases deploy as <major>.<minor> version, overwriting previous versions that have the same major and minor. The default alias stable is updated to point at the new release.
    opened by jayqi 2
  • No module found error without setting PYTHONPATH

    No module found error without setting PYTHONPATH

    Logging this issue so that other people don't have to suffer as much as I did.

    I copied the same dataclass example into a newdataclass.py python file and ran erdantic newdataclass -o diagram.png and got the following error.

    After about 30 mins of debugging by running

    from importlib import import_module
    import_module("newdataclass")
    

    I figured it was because PYTHONPATH wasn't set to the current directory.

    Solutions

    1. Updating the documentation to mention this explicitly
    2. Better error message. Only on scrolling I found a whole traceback of exceptions. Importing module, importing class, etc.,

    Error Log

    Traceback (most recent call last):
    
      File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/erdantic/cli.py", line 131, in import_object_from_name
        return import_module(full_obj_name)
    
      File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    
      File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
    
      File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
    
      File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
    
    ModuleNotFoundError: No module named 'newdataclass'
    
    
    During handling of the above exception, another exception occurred:
    
    
    Traceback (most recent call last):
    
      File "/Users/bhavaniravi/.virtualenvs/python-everyday/bin/erdantic", line 8, in <module>
        sys.exit(app())
    
      File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/erdantic/cli.py", line 108, in main
        model_or_module_objs = [import_object_from_name(mm) for mm in models_or_modules]
    
      File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/erdantic/cli.py", line 108, in <listcomp>
        model_or_module_objs = [import_object_from_name(mm) for mm in models_or_modules]
    
      File "/Users/bhavaniravi/.virtualenvs/python-everyday/lib/python3.9/site-packages/erdantic/cli.py", line 135, in import_object_from_name
        module_name, obj_name = full_obj_name.rsplit(".", 1)
    
    ValueError: not enough values to unpack (expected 2, got 1)
    
    opened by bhavaniravi 1
  • NBConvert does not handle SVG correctly causing docs build to fail

    NBConvert does not handle SVG correctly causing docs build to fail

    opened by github-actions[bot] 10
  • PlantUML generation?

    PlantUML generation?

    When I stumbled on this project, what I was really looking for was a way to generate an ERD in PlantUML. I've implemented that for personal use, but I'm happy to contribute the work back if it would be considered for inclusion. The changes are quite small.

    This was a remarkably easy code base to grok and dig into, so kudos on a useful project!

    opened by swails 1
  • Separating class and its inherited class

    Separating class and its inherited class

    As a user I want to see the relation between a class and its inherited class and also separate the members where it should be So that you will get a correct UML view of the classes.

    example: https://github.com/drivendataorg/erdantic/blob/main/erdantic/examples/pydantic.py

    Additional class:

    class PlannedParty(Party):
        """A planned group of adventurers finding themselves doing and saying things altogether unexpected.
        Attributes:
            name (str): Name that party is known by
            formed_datetime (datetime): Timestamp of when the party was formed
            members (List[Adventurer]): Adventurers that belong to this party
            active_quest (Optional[Quest]): Current quest that party is actively tackling
            planned_quests (List[Quest]): A list of quest that party which can be tackled
        """
    
        planned_quests: List[Quest] = []
    

    (Although this is an erd package, it can have added value. I understand if this issue is closed as it might not fit the vision.)

    opened by zamiramir 0
  • Write tests against static rendered png and svg outputs

    Write tests against static rendered png and svg outputs

    Right now we don't fully check rendered content on diagrams, but we should.

    ~We should do this in a parameterized way to reduce the amount of testing code to be maintained. We can use pytest's fixtures to load the example classes and/or created diagram objects.~ Addressed in #21.

    #21 addressed creating the static outputs, and has a test that checks the against the static DOT files. Still need tests that check png and svg formats.

    enhancement 
    opened by jayqi 0
Releases(v0.5.0)
  • v0.5.0(Jul 30, 2022)

    • Removed support for Python 3.6. (Issue #51, PR #56)
    • Added support for modules as inputs to all entrypoints to diagram creation (create, draw, to_dot, CLI). For all modules passed, erdantic will find all supported data model classes in each module. (Issue #23, PR #58)
      • Added new parameter limit_search_models_to to all entrypoints to allow for limiting which data model classes will be yielded from searching a module.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Apr 8, 2022)

  • v0.4.0(Nov 6, 2021)

    • Added support for showing field documentation from Pydantic models with descriptions set with Field(description=...) in SVG tooltips. This will add an "Attributes" section to the tooltip using Google-style docstring format and lists fields where the description keyword argument is used. (Issue #8, PR #42)
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Oct 29, 2021)

    • Fixed handling of forward references in field type declarations. Evaluated forward references will be properly identified. Forward references not converted to typing.ForwardRef will throw a StringForwardRefError with instructions for how to resolve. Unevaluated forward references will throw an UnevaluatedForwardRefError with instructions for how to resolve. See new documentation for more details. (Issue #40, PR #41)
    • Changed name of erdantic.errors module to erdantic.exceptions. (PR #41)
    • Added new ErdanticException base class from which other exceptions raised within the erdantic library are subclassed from. Changed several existing ValueError exceptions to new exception classes that subclass both ErdanticException and ValueError. (PR #41)
    • Changed __lt__ method on Model and Edge to return NotImplemented instead of raising an exception to follow typical convention for unsupported input types. (PR #41)
    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Feb 17, 2021)

  • v0.2.0(Feb 15, 2021)

  • v0.1.2(Feb 11, 2021)

    • Fixed bug where Pydantic fields were missing generics in their type annotations. (#19)
    • Add tests against static rendered DOT output. Change adapter tests to use parameterized fixtures. (#21)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.1(Feb 11, 2021)

  • v0.1.0(Feb 10, 2021)

Owner
DrivenData
Data science competitions for social good.
DrivenData
Matplotlib colormaps from the yt project !

cmyt Matplotlib colormaps from the yt project ! Colormaps overview The following colormaps, as well as their respective reversed (*_r) versions are av

The yt project 5 Sep 16, 2022
By default, networkx has problems with drawing self-loops in graphs.

By default, networkx has problems with drawing self-loops in graphs. It makes it hard to draw a graph with self-loops or to make a nicely looking chord diagram. This repository provides some code to

Vladimir Shitov 5 Jan 06, 2022
Simple Inkscape Scripting

Simple Inkscape Scripting Description In the Inkscape vector-drawing program, how would you go about drawing 100 diamonds, each with a random color an

Scott Pakin 140 Dec 27, 2022
Friday Night Funkin - converts a chart from 4/4 time to 6/8 time, or from regular to swing tempo.

Chart to swing converter As seen in https://twitter.com/i_winxd/status/1462220493558366214 A program written in python that converts a chart from 4/4

5 Dec 23, 2022
Seismic Waveform Inversion Toolbox-1.0

Seismic Waveform Inversion Toolbox (SWIT-1.0)

Haipeng Li 98 Dec 29, 2022
Simulation du problème de Monty Hall avec Python et matplotlib

Le problème de Monty Hall C'est un jeu télévisé où il y a trois portes sur le plateau de jeu. Seule une de ces portes cache un trésor. Il n'y a rien d

ETCHART YANG 1 Jan 06, 2022
🌀❄️🌩️ This repository contains some examples for creating 2d and 3d weather plots using matplotlib and cartopy libraries in python3.

Weather-Plotting 🌀 ❄️ 🌩️ This repository contains some examples for creating 2d and 3d weather plots using matplotlib and cartopy libraries in pytho

Giannis Dravilas 21 Dec 10, 2022
Active Transport Analytics Model (ATAM) is a new strategic transport modelling and data visualization framework for Active Transport as well as emerging micro-mobility modes

{ATAM} Active Transport Analytics Model Active Transport Analytics Model (“ATAM”) is a new strategic transport modelling and data visualization framew

Peter Stephan 0 Jan 12, 2022
PyPassword is a simple follow up to PyPassphrase

PyPassword PyPassword is a simple follow up to PyPassphrase. After finishing that project it occured to me that while some may wish to use that option

Scotty 2 Jan 22, 2022
NumPy and Pandas interface to Big Data

Blaze translates a subset of modified NumPy and Pandas-like syntax to databases and other computing systems. Blaze allows Python users a familiar inte

Blaze 3.1k Jan 01, 2023
Lumen provides a framework for visual analytics, which allows users to build data-driven dashboards from a simple yaml specification

Lumen project provides a framework for visual analytics, which allows users to build data-driven dashboards from a simple yaml specification

HoloViz 120 Jan 04, 2023
a simple REPL display lib for circuitpython

Circuitpython-termio-lib a simple REPL display lib for circuitpython Fonctions cls clear terminal screen and set cursor on top left : coords 0,0 usage

BeBoXoS 1 Nov 17, 2021
Data Visualization Guide for Presentations, Reports, and Dashboards

This is a highly practical and example-based guide on visually representing data in reports and dashboards.

Anton Zhiyanov 395 Dec 29, 2022
Streamlit dashboard examples - Twitter cashtags, StockTwits, WSB, Charts, SQL Pattern Scanner

streamlit-dashboards Streamlit dashboard examples - Twitter cashtags, StockTwits, WSB, Charts, SQL Pattern Scanner Tutorial Video https://ww

122 Dec 21, 2022
Shaded 😎 quantile plots

shadyquant 😎 This python package allows you to quantile and plot lines where you have multiple samples, typically for visualizing uncertainty. Your d

Mehrad Ansari 13 Sep 29, 2022
A python script to visualise explain plans as a graph using graphviz

README Needs to be improved Prerequisites Need to have graphiz installed on the machine. Refer to https://graphviz.readthedocs.io/en/stable/manual.htm

Edward Mallia 1 Sep 28, 2021
100 Days of Code The Complete Python Pro Bootcamp for 2022

100-Day-With-Python 100 Days of Code - The Complete Python Pro Bootcamp for 2022. In this course, I spend with python language over 100 days, and I up

Rajdip Das 8 Jun 22, 2022
Script to create an animated data visualisation for categorical timeseries data - GIF choropleth map with annotations.

choropleth_ldn Simple script to create a chloropleth map of London with categorical timeseries data. The script in main.py creates a gif of the most f

1 Oct 07, 2021
Standardized plots and visualizations in Python

Standardized plots and visualizations in Python pltviz is a Python package for standardized visualization. Routine and novel plotting approaches are f

Andrew Tavis McAllister 0 Jul 09, 2022
Show Data: Show your dataset in web browser!

Show Data is to generate html tables for large scale image dataset, especially for the dataset in remote server. It provides some useful commond line tools and fully customizeble API reference to gen

Dechao Meng 83 Nov 26, 2022