Python Example Project Structure

Overview

Python Example Project Structure

Example of statuses that can be in readme:

Go Report Card CII Best Practices Gitpod ready-to-code Fuzzing Status

Visit my docs for the full documentation, examples and guides.

With this project you get:

  • a minimal setup.py file
  • testing with PyTest
  • documentation (HTML and PDF) generated using Sphinx
  • a CLI entry point

Project Structure

example_project/
 |-- docs/
 |-- |-- build/
 |-- |-- source/
 |-- example_project/
 |-- |-- __init__.py
 |-- |-- __version__.py
 |-- |-- example_module.py
 |-- tests/
 |-- |-- test_data/
 |-- |   |-- example_class_data.json
 |-- |   __init__.py
 |-- |   conftest.py
 |-- |   test_example_class.py
 |-- .env
 |-- .gitignore
 |-- Pipfile
 |-- Pipfile.lock
 |-- README.md
 |-- setup.py

Example Project

  • example_module.py
  • cli.py

The example_module.py module contains sample code. tests folder contains tests using PyTest.

The cli.py module is referenced in the setup.py file via the entry_points definitions:

entry_points={
    'console_scripts': ['py-package-template=example_project.cli:main'],
}

Project Dependencies

Using pipenv. Use --dev flag for pkgs only needed for dev or test. This gives a deterministic build. Note pipenv is a reference implementation recommened by Python. I fully expect pip to eventually implement it internally.

Installing Pipenv

Assuming you have python installed (duh). On Mac (I exclusively code on mac now) I use brew to manage stuff as mac comes with python 2.x but the world has moved on and you MUST use 3.x, latest version at time of writing is 3.10.

Anyway Install pipenv

pip3 install pipenv

Initialise Your Pipenv shell!

pipenv shell

do this from the source folder where Pipfile is present i.e. root folder.

this will also create your virtual env if its not there, i suggest reading up a bit on pipenv (just a quick brush) so you know the fundamentals as its quite different to virtualenv

Installing this Projects' Dependencies

Make sure that you're in the project's root directory

pipenv install --dev

Running Python and IPython from the Project's Virtual Environment

I find using IPython in command line really useful when I am not in PyCharm IDE. I have included ipython as part of the --dev install above so you should be able to get into it by just doing

❯ ipython
Python 3.10.0 (default, Oct 13 2021, 06:45:00) [Clang 13.0.0 (clang-1300.0.29.3)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.28.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: 

Automatic Loading of Environment Variables

Pipenv will automatically pickup any environment variables declared in the .env file, located in root directory. For example, adding,

DILLY=VANILLY

Will enable access to this variable from python

os.environ['DILLY']

Running Unit Tests

All test have been written using the PyTest package. Tests are kept in the tests folder and can be run from the command line

cd tests
pytest

The conftest.py module is used by PyTest - I've used it to add fixtures which is really cool feature of pytest, I recommend.

Linting Code

I used flake8 for linting code.

pipenv run flake8 example_project

And black for formatting.

 pipenv run black example_project

And you can use pre-commit to hook it all up (incl docs) so you never have to actually do anything manually by hand.

Static Type Checking

I think this is very useful. Think of all the times we said ah it might break some import or something but we wont know until we run, sure we can do extensive tests (we should) but this is like being able to do a compile of python and find problems.

it will barf about pandas/numpy etc which doesnt have stubs, so ignore it for now. am using MyPy package. You can configure what it does with mypy.ini options should be in their docs.

Also note Data Science Types is trying to fix above problem - but I have not tried it.

To run mypy do >

pipenv run python -m mypy example_project/*.py

MyPy options for this project can be defined in the mypy.ini file that MyPy will look for by default. For more information on the full set of options, see the mypy documentation.

Examples of type annotation and type checking for library development can be found in the py_pkg.curves.py module. This should also be cross-referenced with the improvement to readability (and usability) that this has on package documentation.

some terminal output from running above stuff

❯ pipenv run python -m mypy example_project/*.py
Loading .env environment variables...
example_project/example_module.py:12: error: Skipping analyzing "pandas": found module but no type hints or library stubs
example_project/example_module.py:12: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports


❯ pipenv run flake8 example_project
Loading .env environment variables...
example_project/__version__.py:11:13: W292 no newline at end of file
example_project/cli.py:14:10: E211 whitespace before '('
example_project/cli.py:14:20: W292 no newline at end of file

❯ pipenv run black example_project
Loading .env environment variables...
reformatted example_project/__version__.py
reformatted example_project/cli.py
reformatted example_project/example_module.py

Documentation

The documentation in the docs folder has been built using Sphinx. Its a powerful framework that can be used to generate docs on the fly into various formats.

I generated the initial source using:

sphinx-quickstart docs

And then you'd hope to see auto generated docs based on docstring but sadly it isnt so auto-magical... I had to add them in manually per module in docs/source/index.rst where you see I reference 2 files modules.rst and modules_test.rst

And then I did:

cd docs
make html

I like that sphinx ships with make, and we can definitely look to using make as our over-arching tool, plays in nicely with c++.

But alternatively you could also generate the docs by doing:

pipenv run sphinx-build -b html docs/source docs/build/html

Also you obviously should source pipenv shell, and you dont have to actually run pipenv run, but I just show the foolproof way here.

The resulting HTML documentation can be accessed by opening docs/build/html/index.html in a web browser.

If you are curious you'll see I've had to do quite bit of customisation for the config file so that it could generate the docs > docs/source/config.py

I also explored creating PDF docs, I think these are really important if wanting to send to end users, and that can be done using an addon called LatEx, but I havent set it up as yet - but I have explored how to do it.

Building Deployable Distributions

Finally to package this into a wheel! do the following:

pipenv run python setup.py bdist_wheel

This will create build, example_package.egg-info and dist directories. whl should be in dist.

Annoyingly, you cant use pipfile for setup.py requirements, so I had to take a shortcut of generate the requirements.txt by doing

pipenv run pip freeze > requirements.txt

and i wrote a func in setup.py to read the file and use the it to generate install_require, so that the generated wheel installs all the dependencies.

But I make persist requirements.txt in git intentionally, so you generate it everytime you want to create a distributable, i suppose it could be made part of a make command.

a package that provides a marketstrategy for whitelisting on golem

filterms a package that provides a marketstrategy for whitelisting on golem watching requestor logs distribute 10 tasks asynchronously is fun. but you

KJM 3 Aug 03, 2022
Better firefox bookmarks script for rofi

rofi-bookmarks Small python script to open firefox bookmarks with rofi. Features Icons! Only show bookmarks in a specified bookmark folder Show entire

32 Nov 10, 2022
Dungeon Dice Rolls is an aplication that the user can roll dices (d4, d6, d8, d10, d12, d20 and d100) and store the results in one of the 6 arrays.

Dungeon Dice Rolls is an aplication that the user can roll dices (d4, d6, d8, d10, d12, d20 and d100) and store the results in one of the 6 arrays.

Bracero 1 Dec 31, 2021
Leveraging pythonic forces to defeat different coding challenges 🐍

Pyforces Leveraging pythonic forces to defeat different coding challenges! Table of Contents Pyforces Tests Pyforces Pyforces is a study repo with a c

Igor Grillo Peternella 8 Dec 14, 2022
Flask-built web application that simulates a time and cost calculator for charging Electric Vehicles.

ev_charging_calculator Flask-built web application that simulates a time and cost calculator for charging Electric Vehicles. The project aims to simul

1 Nov 03, 2021
simple password manager.

simple password manager.

1 Nov 18, 2021
BDD base project: Python + Behave

BDD base project: Python + Behave Basic example of using Python with Behave (BDD). This Gherkin example includes: Basic Scenario Scenario Outline Tagg

eccanto 1 Dec 08, 2021
easy_sbatch - Batch submitting Slurm jobs with script templates

easy_sbatch - Batch submitting Slurm jobs with script templates

Wei Shen 13 Oct 11, 2022
UniPD exam dates finder

UniPD exam dates finder Find dates for exams at UniPD Usage ./finder.py courses.csv It's suggested to save output to a file: ./finder.py courses.csv

Davide Peressoni 1 Jan 25, 2022
Python interface to IEX and IEX cloud APIs

Python interface to IEX Cloud Referral Please subscribe to IEX Cloud using this referral code. Getting Started Install Install from pip pip install py

IEX Cloud 41 Dec 21, 2022
Wordle Solver

Wordle Solver Installation Install the following onto your computer: Python 3.10.x Download Page Run pip install -r requirements.txt Instructions To r

John Bucknam 1 Feb 15, 2022
Here is my Senior Design Project that I implemented to graduate from Computer Engineering.

Here is my Senior Design Project that I implemented to graduate from Computer Engineering. It is a chatbot made in RASA and helps the user to plan their vacation in the Turkish language. In order to

Ezgi Subaşı 25 May 31, 2022
Pylexa - Artificial Assistant made with Python

Pylexa - Artificial Assistant made with Python Alexa is a famous artificial assistant used massively across the world. It is a substitute of Alexa whi

\_PROTIK_/ 4 Nov 03, 2021
to learn how to do pull request and do contribution to other's repo

Hacktoberfest-2021 - open-source-contribution An Open Source repository to Teach people How to contribute to open sources. 💥 🔥 JOIN PVX PROGRAMMING

Shubham Rawat 82 Dec 26, 2022
Checkers Project Built Using Python

Checkers Project Built Using Python

Meekness Anyaeche 1 Nov 08, 2021
A weekly dive into commonly used modules in the Rust ecosystem, with story flavor!

The goal of this project is to bring the same concept as PyMOTW to the Rust world. PyMOTW was an invaluable resource for me when I was learning Python years ago, and I hope that I can help someone in

Scott Lyons 20 Aug 26, 2022
Standalone PyQGIS application for executing custom scripts without a QGIS GUI.

PyQGIS Standalone Script Executer Standalone PyQGIS application that is able to run a custom script, in this case Proximity.py without the need of a G

6 Sep 23, 2022
🌈Python cheatsheet for all standard libraries(Continuously Updated)

Python Standard Libraries Cheatsheet Depend on Python v3.9.8 All code snippets have been tested to ensure they work properly. Fork me on GitHub. 中文 En

nick 12 Dec 27, 2022
dotfiles - Cristian Valero Abundio

In this repository you can find various configurations to configure your Linux operating system, preferably ArchLinux and its derivatives.

Cristian Valero Abundio 1 Jan 09, 2022
NCAR/UCAR virtual Python Tutorial Seminar Series lesson on MetPy.

The Project Pythia Python Tutorial Seminar Series continues with a lesson on MetPy on Wednesday, 2 February 2022 at 1 PM Mountain Standard Time.

Project Pythia Tutorials 6 Oct 09, 2022