poetry2nix turns Poetry projects into Nix derivations without the need to actually write Nix expressions

Overview

poetry2nix

poetry2nix turns Poetry projects into Nix derivations without the need to actually write Nix expressions. It does so by parsing pyproject.toml and poetry.lock and converting them to Nix derivations on the fly.

For more information, see the announcement post on the Tweag blog. Development of poetry2nix has been supported by Tweag.

API

The poetry2nix public API consists of the following attributes:

  • mkPoetryApplication: Creates a Python application.
  • mkPoetryEnv: Creates a Python environment with an interpreter and all packages from poetry.lock.
  • mkPoetryPackages: Creates an attribute set providing access to the generated packages and other artifacts.
  • mkPoetryScriptsPackage: Creates a package containing the scripts from tool.poetry.scripts of the pyproject.toml.
  • mkPoetryEditablePackage: Creates a package containing editable sources. Changes in the specified paths will be reflected in an interactive nix-shell session without the need to restart it.
  • defaultPoetryOverrides: A set of bundled overrides fixing problems with Python packages.
  • overrides.withDefaults: A convenience function for specifying overrides on top of the defaults.
  • overrides.withoutDefaults: A convenience function for specifying overrides without defaults.
  • cleanPythonSources: A function to create a source filter for python projects.

mkPoetryApplication

Creates a Python application using the Python interpreter specified based on the designated poetry project and lock files. mkPoetryApplication takes an attribute set with the following attributes (attributes without default are mandatory):

  • projectDir: path to the root of the project.
  • src: project source (default: cleanPythonSources { src = projectDir; }).
  • pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
  • poetrylock: poetry.lock file path (default: projectDir + "/poetry.lock").
  • overrides: Python overrides to apply (default: [defaultPoetryOverrides]).
  • meta: application meta data (default: {}).
  • python: The Python interpreter to use (default: pkgs.python3).

Other attributes are passed through to buildPythonApplication.

Example

poetry2nix.mkPoetryApplication {
    projectDir = ./.;
}

See ./pkgs/poetry/default.nix for a working example.

Dependency environment

The resulting derivation also has the passthru attribute dependencyEnv, which is an environment with a python interpreter, all non-development dependencies and your application. This can be used if your application doesn't provide any binaries on its own and instead relies on dependencies binaries to call its modules (as is often the case with celery or gunicorn). For example, if your application defines a CLI for the module admin and a gunicorn app for the module web, a working default.nix would contain

let
    app = poetry2nix.mkPoetryApplication {
        projectDir = ./.;
    };
in app.dependencyEnv

After building this expression, your CLI and app can be called with these commands

$ result/bin/python -m admin
$ result/bin/gunicorn web:app

Note: If you need to perform overrides on the application, use app.dependencyEnv.override { app = app.override { ... }; }. See ./tests/dependency-environment/default.nix for a full example.

mkPoetryEnv

Creates an environment that provides a Python interpreter along with all dependencies declared by the designated poetry project and lock files. Also allows package sources of an application to be installed in editable mode for fast development. mkPoetryEnv takes an attribute set with the following attributes (attributes without default are mandatory):

  • projectDir: path to the root of the project.
  • pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
  • poetrylock: poetry.lock file path (default: projectDir + "/poetry.lock").
  • overrides: Python overrides to apply (default: [defaultPoetryOverrides]).
  • python: The Python interpreter to use (default: pkgs.python3).
  • editablePackageSources: A mapping from package name to source directory, these will be installed in editable mode (default: {}).

Example

poetry2nix.mkPoetryEnv {
    projectDir = ./.;
}

See ./tests/env/default.nix for a working example.

Example with editable packages

poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    editablePackageSources = {
        my-app = ./src;
    };
}

See ./tests/editable/default.nix for a working example of an editable package.

Example shell.nix

The env attribute of the attribute set created by mkPoetryEnv contains a shell environment.

{ pkgs ? import 
    {} }:
let
  myAppEnv = pkgs.poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    editablePackageSources = {
      my-app = ./src;
    };
  };
in myAppEnv.env

Example shell.nix with external dependencies

For a shell environment including external dependencies, override the env to add dependency packages (for example, pkgs.hello) as build inputs.

{ pkgs ? import 
    {} }:
let
  myAppEnv = pkgs.poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    editablePackageSources = {
      my-app = ./src;
    };
  };
in myAppEnv.env.overrideAttrs (oldAttrs: {
  buildInputs = [ pkgs.hello ]:
})

mkPoetryPackages

Creates an attribute set of the shape { python, poetryPackages, pyProject, poetryLock }. Where python is the interpreter specified, poetryPackages is a list of all generated python packages, pyProject is the parsed pyproject.toml and poetryLock is the parsed poetry.lock file. mkPoetryPackages takes an attribute set with the following attributes (attributes without default are mandatory):

  • projectDir: path to the root of the project.
  • pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
  • poetrylock: poetry.lock file path (default: projectDir + "/poetry.lock").
  • overrides: Python overrides to apply (default: [defaultPoetryOverrides]).
  • python: The Python interpreter to use (default: pkgs.python3).
  • editablePackageSources: A mapping from package name to source directory, these will be installed in editable mode (default: {}).

Example

poetry2nix.mkPoetryPackages {
    projectDir = ./.;
    python = python35;
}

mkPoetryScriptsPackage

Creates a package containing the scripts from tool.poetry.scripts of the pyproject.toml:

  • projectDir: path to the root of the project.
  • pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
  • python: The Python interpreter to use (default: pkgs.python3).

Example

poetry2nix.mkPoetryScriptsPackage {
    projectDir = ./.;
    python = python35;
}

mkPoetryEditablePackage

Creates a package containing editable sources. Changes in the specified paths will be reflected in an interactive nix-shell session without the need to restart it:

  • projectDir: path to the root of the project.
  • pyproject: path to pyproject.toml (default: projectDir + "/pyproject.toml").
  • python: The Python interpreter to use (default: pkgs.python3).
  • editablePackageSources: A mapping from package name to source directory, these will be installed in editable mode (default: {}).

Example

poetry2nix.mkPoetryEditablePackage {
    projectDir = ./.;
    python = python35;
    editablePackageSources = {
        my-app = ./src;
    };
}

defaultPoetryOverrides

poetry2nix bundles a set of default overrides that fix problems with various Python packages. These overrides are implemented in overrides.nix.

overrides.withDefaults

Returns a list containing the specified overlay and defaultPoetryOverrides.

Takes an attribute set with the following attributes (attributes without default are mandatory):

  • src: project source directory

Example

poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    overrides = poetry2nix.overrides.withDefaults (self: super: { foo = null; });
}

See ./tests/override-support/default.nix for a working example.

overrides.withoutDefaults

Returns a list containing just the specified overlay, ignoring defaultPoetryOverrides.

Example

poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    overrides = poetry2nix.overrides.withoutDefaults (self: super: { foo = null; });
}

cleanPythonSources

Provides a source filtering mechanism that:

  • Filters gitignore's
  • Filters pycache/pyc files
  • Uses cleanSourceFilter to filter out .git/.hg, .o/.so, editor backup files & nix result symlinks

Example

poetry2nix.cleanPythonSources {
    src = ./.;
}

Creating a custom Poetry2nix instance

Sometimes when it can be convenient to create a custom instance of poetry2nix with a different set of default overrides.

Example

let
  # self & super refers to poetry2nix
  p2nix = poetry2nix.overrideScope' (self: super: {

    # pyself & pysuper refers to python packages
    defaultPoetryOverrides = super.defaultPoetryOverrides.extend (pyself: pysuper: {

      my-custom-pkg = super.my-custom-pkg.overridePythonAttrs (oldAttrs: { });

    });

  });

in
p2nix.mkPoetryApplication {
  projectDir = ./.;
}

or as a nixpkgs overlay:

let
  pkgs = import 
    {
    overlays = [
      # self & super refers to nixpkgs
      (self: super: {

        # p2self & p2super refers to poetry2nix
        poetry2nix = super.poetry2nix.overrideScope' (p2nixself: p2nixsuper: {

          # pyself & pysuper refers to python packages
          defaultPoetryOverrides = p2nixsuper.defaultPoetryOverrides.extend (pyself: pysuper: {

            my-custom-pkg = super.my-custom-pkg.overridePythonAttrs (oldAttrs: { });

          });

        });
      })

    ];
  };

in pkgs.poetry2nix.mkPoetryApplication {
  projectDir = ./.;
}

Using the flake

For the experimental flakes functionality we provide poetry2nix as a flake providing an overlay to use with nixpkgs. Additionally, the flake provides a flake template to quickly start using poetry2nix in a project:

nix flake init --template github:nix-community/poetry2nix

Contributing

Contributions to this project are welcome in the form of GitHub PRs. Please consider the following before creating PRs:

  • This project uses nixpkgs-fmt for formatting the Nix code. You can use nix-shell --run "nixpkgs-fmt ." to format everything.
  • If you are planning to make any considerable changes, you should first present your plans in a GitHub issue so it can be discussed.
  • If you add new features please consider adding tests. You can run them locally as follows:
nix-build --keep-going --show-trace tests/default.nix

License

poetry2nix is released under the terms of the MIT license.

Owner
Nix community projects
A project incubator that works in parallel of the @NixOS org
Nix community projects
A simple API to upload notes or files to KBFS

This API can be used to upload either secure notes or files to a secure KeybaseFS folder.

Dakota Brown 1 Oct 08, 2021
Игра реализована с помощью языке python3.9, библиотеки pygame

Игра в танки Игра реализована с помощью языке python3.9, библиотеки pygame. Игра имеет несколько уровней. Правила: есть танки, которые стреляют, есть

1 Jan 01, 2022
Choice Coin 633 Dec 23, 2022
Team Curie is a group of people working together to achieve a common aim

Team Curie is a group of people working together to achieve a common aim. We are enthusiasts!.... We are setting the pace!.... We offer encouragement and motivation....And we believe TeamWork makes t

4 Aug 07, 2021
Randomly distribute members by groups making sure that every sector is represented

Generate Groups Randomly distribute members by groups making sure that every sector is represented The Scenario Imagine that you have a large group of

Jorge Gomes 1 Oct 22, 2021
Gitlab py scripts

Gitlab py scripts The code can be used to gather the list of GitHub groups/projects and the permissions of the users in those groups/projects. group/p

Roghuchi 1 Aug 29, 2022
Store Simulation

Almacenes Para clonar el Repositorio: Vaya a la terminal de Linux o Mac, o a la cmd en Windows y ejecute:

Johan Posada 1 Nov 12, 2021
Launcher program to select which version of the Q-Sys software to launch.

QSC-QSYS Launcher Launcher program to select which version of the Q-Sys software to launch. Instructions To use the application simply save the "Q-Sys

Zach Lisko 2 Sep 28, 2022
HPomb Is Socail Engineering Tool , Used For Bombing , Spoofing and Anonymity Available For Linux And Android(Termux)

HPomb v2020.02 Coming Soon Created By Secanonm HPomb Is Socail Engineering Tool , Used For Bombing , Spoofing and Anonymity Available For Linux And An

Secanonm 10 Jul 25, 2022
Runs macOS on linux with qemu.

mac-on-linux-with-qemu Runs macOS on linux with qemu. Pre-requisites qemu-system-x86_64 dmg2img pulseaudio python[click] Usage After cloning the repos

Arindam Das 177 Dec 26, 2022
Minimalist BERT implementation assignment for CS11-747

minbert Assignment by Zhengbao Jiang, Shuyan Zhou, and Ritam Dutt This is an exercise in developing a minimalist version of BERT, part of Carnegie Mel

Graham Neubig 51 Jan 03, 2023
Dotfiles for my configurations!

Dotfiles Repo Welcome! Over here, you can find my dotfiles for various applications, including QTile, Alacritty, Kitty, LunarVim, and more! Make sure

class PythonAddict 3 Jan 10, 2022
Eros is an expiremental programming language built using simple Python code.

Eros is an expiremental programming language built using simple Python code. Featuring an easy syntax and unique features like type slicing, the language remains an expirement that grows in down time

zxro 2 Nov 21, 2021
One destination for all the developer's learning resources.

DevResources One destination for all the developer's learning resources. Find all of your learning resources under one roof and add your own. Live ✨ Y

Gaurav Sharma 33 Oct 21, 2022
A platform for developers 👩‍💻 who wants to share their programs and projects.

Fest-Practice-2021 This project is excluded from Hacktoberfest 2021. Please use this as a testing repo/project. A platform for developers 👩‍💻 who wa

Mayank Choudhary 40 Nov 07, 2022
CMPE 204 Modelling Project

CISC/CMPE 204 Modelling Project Welcome to the major project for CISC/CMPE 204 (Fall 2021)! Change this README.md file to summarize your project (few

totallyrin 2 May 16, 2022
Expense-manager - Expense manager with python

Expense_manager TO-DO Source extractor: Credit Card, Wallet Destination extracto

1 Feb 13, 2022
RxPY - The Reactive Extensions for Python (RxPY)

The Reactive Extensions for Python (RxPY) A library for composing asynchronous and event-based programs using observable collections and query operato

ReactiveX 4.4k Dec 29, 2022
Prometheus exporter for chess.com player data

chess-exporter Prometheus exporter for chess.com player data implemented via chess.com's published data API and Prometheus Python Client Example use c

Mário Uhrík 7 Feb 28, 2022
A submodule of rmcrkd/ODE-Uniqueness

Heston-ODE This repo contains the Heston-related code that accompanies the article One-sided maximal uniqueness for a class of spatially irregular ord

0 Jan 05, 2022