A static analysis library for computing graph representations of Python programs suitable for use with graph neural networks.

Last update: Dec 29, 2022

Related tags

Overview

python_graphs

This package is for computing graph representations of Python programs for machine learning applications. It includes the following modules:

control_flow For computing control flow graphs statically from Python programs.
data_flow For computing data flow analyses of Python programs.
program_graph For computing graphs statically to represent arbitrary Python programs or functions.
cyclomatic_complexity For computing the cyclomatic complexity of a Python function.

Installation

To install python_graphs with pip, run: pip install python_graphs.

To install python_graphs from source, run: python setup.py develop.

Common Tasks

Generate a control flow graph from a function fn:

from python_graphs import control_flow
graph = control_flow.get_control_flow_graph(fn)

Generate a program graph from a function fn:

from python_graphs import program_graph
graph = program_graph.get_program_graph(fn)

Compute the cyclomatic complexity of a function fn:

from python_graphs import control_flow
from python_graphs import cyclomatic_complexity
graph = control_flow.get_control_flow_graph(fn)
value = cyclomatic_complexity.cyclomatic_complexity(graph)

This is not an officially supported Google product.

Comments

Can you provide a quick start example?

Super cool project! Love the idea and think it has a lot of potential.

it would be awesome to have an examples/ directory containing some sample usages - maybe even just plotting the graphs with networkX and matplotlib.
question

opened by LukeWood 5
How do we solve the error when installing python-graphs?

Hello,

I encountered an error "fatal error: 'graphviz/cgraph.h' file not found" when trying to install python_graphs. How do I solve this issue, please? Thanks.
question

opened by fraolBatole 2
How to generate a Holistic Data Flow Graph for a given Function ?

@dbieber, Thanks for this awesome work.

Question

control_flow.get_control_flow_graph, returns a Control Flow Graph for a given Function Object. There is one data_flow class, Is there a way to generate a complete Data Flow Graph given a Function Object?

Thanks.

opened by reshinthadithyan 2
Rename fn to get_test_components to eliminate extra test from logs

The function test_components was being registered as an unsupported test, when in reality it was meant as a helper function for tests. Renaming resolves this.

opened by dbieber 0
get_start_control_flow_node, next_from_end, raise edges, and labels in branches
Adds get_start_control_flow_node to ControlFlowGraph

Adds next_from_end to ControlFlowNode

Uses labels (e.g. '' and '' strings) to indicate these special nodes

Support keyword only arguments without defaults

Add non-interrupting edges from raise statements

Bump version number
opened by dbieber 0
Separate branch kinds

Splits "branches" into branches, except_branches, and reraise_branches.

branches are you're usual branch decisions: ifs, fors, and whiles. except_branches are at "except E:" statements, with True indicating the exception matches and False indicating it does not reraise_branches are at the end of "finally:" blocks, with True indicating the path taken after finally if an error has been raised previously, and False indicating the path taken if there's nothing to reraise at the end of the finally.

opened by dbieber 0
Add module frame to catch raises in top-level code.

Add module frame to catch raises in top-level code. Also marks except expressions and finally blocks as branch points.

An "except A:"'s branch decision is whether the current exception matches A. At the end of a finally block, the branch decision is whether an exception is currently being raised.

This includes https://github.com/google-research/python-graphs/pull/3: Splits "branches" into branches, except_branches, and reraise_branches.

branches are your usual branch decisions: ifs, fors, and whiles. except_branches are at "except E:" statements, with True indicating the exception matches and False indicating it does not reraise_branches are at the end of "finally:" blocks, with True indicating the path taken after finally if an error has been raised previously, and False indicating the path taken if there's nothing to reraise at the end of the finally.

opened by dbieber 0
KeyError when trying to get program_graph
When I try to create a program graph, I encounter a KeyError. If I remove all the and and or expressions from the python file (buggy.py) the error does not occur.

This is how I use the library:

graph = program_graph.get_program_graph(code) program_graph_graphviz.render(graph, path='source.png')

where code is simply the code in the attached file buggy.py.txt.

I have also attached the log file log.txt.

buggy.py.txt

log.txt

More information: python 3.9.5 commit head=44c15b92197f374c3550353ff827997ef1c1d857 gast 0.5.3
opened by ppashakhanloo 1

Releases(v1.2.3)

v1.2.3(Oct 7, 2021)

get_start_control_flow_node, next_from_end, raise edges, and labels in branches (#6)

* Adds get_start_control_flow_node to ControlFlowGraph
* Adds next_from_end to ControlFlowNode
* Uses labels (e.g. '<exit>' and '<raise>' strings) to indicate these special nodes
* Support keyword only arguments without defaults
* Add non-interrupting edges from raise statements
* Bump version number

Source code(tar.gz)
Source code(zip)

v1.2.0(Oct 5, 2021)

Introduce get_branches API on control flow nodes. Previously the new branch types (except_branches and reraise_branches) were only accessible on basic blocks, not on individual nodes.
Source code(tar.gz)
Source code(zip)
v1.1.0(Oct 5, 2021)
Adds a module frame to catch raises in top-level code.

Also marks except expressions and finally blocks as branch points.

The branch kinds are: branches, except_branches, and reraise_branches.

branches are your usual branch decisions: ifs, fors, and whiles.

except_branches are at "except E:" statements, with True indicating the exception matches and False indicating it does not

reraise_branches are at the end of "finally:" blocks, with True indicating the path taken after finally if an error has been raised previously, and False indicating the path taken if there's nothing to reraise at the end of the finally.

Source code(tar.gz)
Source code(zip)
v1.0.1(May 7, 2021)

Includes __init__.py files to allow use of python_graphs library from Python 2.
Source code(tar.gz)
Source code(zip)
v1.0.0(Apr 12, 2021)
v1.0.0

Initial public release of the python_graphs library.

Core features:

control flow graph generation

data flow analyses

program graph construction

cyclomatic complexity calculation

a solid test suite for all the above

visualizations using graphviz for each of the graph representations

Source code(tar.gz)
Source code(zip)

Owner

Google Research

GitHub Repository

FFCV: Fast Forward Computer Vision (and other ML workloads!)

Fast Forward Computer Vision: train models at a fraction of the cost with accele

2.3k Jan 03, 2023

Code for MarioNette: Self-Supervised Sprite Learning, in NeurIPS 2021

MarioNette | Webpage | Paper | Video MarioNette: Self-Supervised Sprite Learning Dmitriy Smirnov, Michaël Gharbi, Matthew Fisher, Vitor Guizilini, Ale

28 Nov 18, 2022

Over-the-Air Ensemble Inference with Model Privacy

Over-the-Air Ensemble Inference with Model Privacy This repository contains simulations for our private ensemble inference method. Installation Instal

1 Jun 29, 2022

Synthetic Humans for Action Recognition, IJCV 2021

SURREACT: Synthetic Humans for Action Recognition from Unseen Viewpoints Gül Varol, Ivan Laptev and Cordelia Schmid, Andrew Zisserman, Synthetic Human

59 Dec 14, 2022

Tensorflow implementation for "Improved Transformer for High-Resolution GANs" (NeurIPS 2021).

HiT-GAN Official TensorFlow Implementation HiT-GAN presents a Transformer-based generator that is trained based on Generative Adversarial Networks (GA

78 Oct 31, 2022

[ICLR 2022 Oral] F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization

F8Net Fixed-Point 8-bit Only Multiplication for Network Quantization (ICLR 2022 Oral) OpenReview | arXiv | PDF | Model Zoo | BibTex PyTorch implementa

76 Dec 13, 2022

Project ArXiv Citation Network

Project ArXiv Citation Network Overview This project involved the analysis of the ArXiv citation network. Usage The complete code of this project is i

5 Oct 20, 2022

Convenient tool for speeding up the intern/officer review process.

icpc-app-screen Convenient tool for speeding up the intern/officer applicant review process. Eliminates the pain from reading application responses of

1 Oct 30, 2021

Code accompanying the paper on "An Empirical Investigation of Domain Generalization with Empirical Risk Minimizers" published at NeurIPS, 2021

Code for "An Empirical Investigation of Domian Generalization with Empirical Risk Minimizers" (NeurIPS 2021) Motivation and Introduction Domain Genera

15 Dec 27, 2022

A framework that allows people to write their own Rocket League bots.

YOU PROBABLY SHOULDN'T PULL THIS REPO Bot Makers Read This! If you just want to make a bot, you don't need to be here. Instead, start with one of thes

543 Dec 20, 2022

License Plate Detection Application

LicensePlate_Project 🚗 🚙 [Project] 2021.02 ~ 2021.09 License Plate Detection Application Overview 1. 데이터 수집 및 라벨링 차량 번호판 이미지를 직접 수집하여 각 이미지에 대해 '번호판

4 Oct 10, 2022

🏆 The 1st Place Submission to AICity Challenge 2021 Natural Language-Based Vehicle Retrieval Track (Alibaba-UTS submission)

AI City 2021: Connecting Language and Vision for Natural Language-Based Vehicle Retrieval 🏆 The 1st Place Submission to AICity Challenge 2021 Natural

82 Dec 29, 2022

PyTorch framework for Deep Learning research and development.

Accelerated DL & RL PyTorch framework for Deep Learning research and development. It was developed with a focus on reproducibility, fast experimentati

29 Jul 13, 2022

Repository of Jupyter notebook tutorials for teaching the Deep Learning Course at the University of Amsterdam (MSc AI), Fall 2020

1.1k Jan 07, 2023

A pure PyTorch implementation of the loss described in "Online Segment to Segment Neural Transduction"

ssnt-loss ℹ️ This is a WIP project. the implementation is still being tested. A pure PyTorch implementation of the loss described in "Online Segment t

1 Feb 09, 2022

Fuwa-http - The http client implementation for the fuwa eco-system

Fuwa HTTP The HTTP client implementation for the fuwa eco-system Example import

2 Feb 16, 2022

Official PyTorch implementation of U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation

U-GAT-IT — Official PyTorch Implementation : Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Imag

2.4k Jan 04, 2023