A Python package which supports global logfmt formatted logging.

Overview

Python Logfmter

pre-commit test

A Python package which supports global logfmt formatted logging.

Install

$ pip install logfmter

Usage

Before integrating this library, you should be familiar with Python's logging functionality. I recommend reading the Basic Logging Tutorial.

This package exposes a single Logfmter class that can be integrated into the standard library logging system similar to any logging.Formatter.

The provided formatter will logfmt encode all logs. Key value pairs are provided via the extra keyword argument or by passing a dictionary as the log message.

Basic

import logging
from logfmter import Logfmter

handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(Logfmter())

logger = logging.getLogger()
logger.addHandler(handler)

logger.error("hello", extra={"alpha": 1}) # at=ERROR msg=hello alpha=1
logger.error({"token": "Hello, World!"}) # at=ERROR token="Hello, World!"

Customize

You can subclass the formatter to change its behavior.

import logging
from logfmter import Logfmter


class CustomLogfmter(Logfmter):
    """
    Provide a custom logfmt formatter which formats
    booleans as "yes" or "no" strings.
    """

    @classmethod
    def format_value(cls, value):
        if isinstance(value, bool):
            return "yes" if value else "no"

	return super().format_value(value)

handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(CustomLogfmter())

logger = logging.getLogger()
logger.addHandler(handler)

logger.error({"example": True}) # at=ERROR example=yes

Development

Required Software

Refer to the links provided below to install these development dependencies:

Getting Started

Setup

$ <runtimes.txt xargs -n 1 pyenv install -s
$ direnv allow
$ pip install -r requirements/dev.txt
$ pre-commit install
$ pip install -e .

Tests

Run the test suite against the active python environment.

$ pytest

Run the test suite against the active python environment and watch the codebase for any changes.

$ ptw

Run the test suite against all supported python versions.

$ tox

Publishing

Build

$ python -m build

Upload

$ twine upload dist/*
Comments
  • Question: How to include default / reserved attributes in the log message?

    Question: How to include default / reserved attributes in the log message?

    Thank you for making this library available. I have been looking at the tests and docs but could not find out how I can include some of the keys you defined as "RESERVED" in my message, such as asctime. I would like to say::

    >>> logger.info("hello")
    "asctime=2022-04-20 msg='hello'"
    

    without having to pass asctime in as an extra. I have tried the below in my dictconfig but did not work::

            'logfmt': {
                'format': 'on=%(asctime)s level=%(levelname)s module=%(name)s msg=%(message)s',
                'class': 'logfmter.Logfmter',
            },
    

    Is this possible?

    opened by meitham 3
  • Include Native Log Record Attributes

    Include Native Log Record Attributes

    We should be able to configure the logger to always include some default keys. For example, a user may want to always include a time or logger key.

    This functionality is seen in the python-json-logger.

    opened by jteppinette 1
  • Add Python 11 Official Support / Testing

    Add Python 11 Official Support / Testing

    I assume this will work on Python 11 just fine, but let's add the Python 11 image to the test scripts and see if there are any improvements that can be made.

    opened by jteppinette 0
  • Update Structure

    Update Structure

    We should move the logfmter directory inside src as suggested by:

    • https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html#choosing-a-test-layout-import-rules
    • https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure%3E

    This is highly recommend and will also support the usage of importlib import mode as recommended here.

    opened by jteppinette 0
  • Exclude Keys

    Exclude Keys

    Some third party libraries may make log calls with keys which we don't want to log. We should be able to exclude these keys from the final log line.

    For example, when overriding uvicorn logging. The color_message key is currently printed (it is used by their custom color formatter). If we could exclude this key, then we could prevent the following output:

    at=INFO msg="Started server process [97819]" color_message="Started server process [%d]"
    at=INFO msg="Waiting for application startup."
    at=INFO msg="Application startup complete."
    at=INFO msg="Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)" color_message="Uvicorn running on %s://%s:%d (Press CTRL+C to quit)"
    
    opened by jteppinette 0
Releases(v0.0.6)
  • v0.0.6(Apr 22, 2022)

  • v0.0.5(Apr 20, 2022)

    • Add support for include native log record attributes in the final log output.
    • Add support for overriding the date format used when formatting the asctime attribute.
    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Mar 30, 2022)

    • Fix the usage documentation
    • Escape newline characters in all logged values. Previously, you could generate multi-line log statements. This should never be the case.
    • Add support for auto-generating exc_info parameters. If the log record is generated with exc_info, as in when using logging.exception(...), the log message will contain properly formatted exception and traceback information.
    • Add support for logging with an empty message dictionary.
    Source code(tar.gz)
    Source code(zip)
  • v0.0.3(Feb 12, 2022)

  • v0.0.1(Feb 6, 2022)

Integrates a UPS monitored by NUT into OctoPrint

OctoPrint UPS This OctoPrint plugin interfaces with a UPS monitored by NUT (Network UPS Tools). Requirements NUT must be configured by the user. This

Shawn Bruce 11 Jul 05, 2022
Robust and effective logging for Python 2 and 3.

Robust and effective logging for Python 2 and 3.

Chris Hager 1k Jan 04, 2023
Structured Logging for Python

structlog makes logging in Python faster, less painful, and more powerful by adding structure to your log entries. It's up to you whether you want str

Hynek Schlawack 2.3k Jan 05, 2023
Discord-Image-Logger - Discord Image Logger With Python

Discord-Image-Logger A exploit I found in discord. Working as of now. Explanatio

111 Dec 31, 2022
Summarize LSF job properties by parsing log files.

Summarize LSF job properties by parsing log files of workflows executed by Snakemake.

Kim 4 Jan 09, 2022
The easy way to send notifications

See changelog for recent changes Got an app or service and you want to enable your users to use notifications with their provider of choice? Working o

Or Carmi 2.4k Dec 25, 2022
A Python library that tees the standard output & standard error from the current process to files on disk, while preserving terminal semantics

A Python library that tees the standard output & standard error from the current process to files on disk, while preserving terminal semantics (so breakpoint(), etc work as normal)

Greg Brockman 7 Nov 30, 2022
Command-line tool that instantly fetches Stack Overflow results when an exception is thrown

rebound Rebound is a command-line tool that instantly fetches Stack Overflow results when an exception is thrown. Just use the rebound command to exec

Jonathan Shobrook 3.9k Jan 03, 2023
蓝鲸日志平台(BK-LOG)是为解决分布式架构下日志收集、查询困难的一款日志产品,基于业界主流的全文检索引擎

蓝鲸日志平台(BK-LOG)是为解决分布式架构下日志收集、查询困难的一款日志产品,基于业界主流的全文检索引擎,通过蓝鲸智云的专属 Agent 进行日志采集,提供多种场景化的采集、查询功能。

腾讯蓝鲸 102 Dec 22, 2022
Python bindings for g3log

g3logPython Python bindings for g3log This library provides python3 bindings for g3log + g3sinks (currently logrotate, syslog, and a color-terminal ou

4 May 21, 2021
A new kind of Progress Bar, with real time throughput, eta and very cool animations!

alive-progress :) A new kind of Progress Bar, with real-time throughput, eta and very cool animations! Ever found yourself in a remote ssh session, do

Rogério Sampaio de Almeida 4k Dec 30, 2022
Docker container log aggregation with Elasticsearch, Kibana & Filebeat

Epilog Dead simple container log aggregation with ELK stack Preface Epilog aims to demonstrate a language-agnostic, non-invasive, and straightfo

Redowan Delowar 23 Oct 26, 2022
Translating symbolicated Apple JSON format crash log into our old friends :)

CrashTranslation Translating symbolicated Apple JSON format crash log into our old friends :) Usage python3 translation.py -i {input_sybolicated_json_

Kam-To 11 May 16, 2022
A python logging library

logi v1.3.4 instolation the lib works on python 3x versions pip install logi examples import import logi log = logger(path='C:/file path', timestamp=T

2 Jul 06, 2022
A simple package that allows you to save inputs & outputs as .log files

wolf_dot_log A simple package that allows you to save inputs & outputs as .log files pip install wolf_dot_log pip3 install wolf_dot_log |Instructions|

Alpwuf 1 Nov 16, 2021
A demo of Prometheus+Grafana for monitoring an ML model served with FastAPI.

ml-monitoring Jeremy Jordan This repository provides an example setup for monitoring an ML system deployed on Kubernetes.

Jeremy Jordan 176 Jan 01, 2023
Monitor and log Network and Disks statistics in MegaBytes per second.

iometrics Monitor and log Network and Disks statistics in MegaBytes per second. Install pip install iometrics Usage Pytorch-lightning integration from

Leo Gallucci 17 May 03, 2022
A watchdog and logger to Discord for hosting ScPrime servers.

ScpDog A watchdog and logger to Discord for hosting ScPrime servers. Designed to work on Linux servers. This is only capable of sending the logs from

Keagan Landfried 3 Jan 10, 2022
Log4j alternative for Python

Log4p Log4p is the most secure logging library ever created in this and all other universes. Usage: import log4p log4p.log('"Wow, this library is sec

Isaak Uchakaev 15 Dec 16, 2022
dash-manufacture-spc-dashboard is a dashboard for monitoring read-time process quality along manufacture production line

In our solution based on plotly, dash and influxdb, the user will firstly generate the specifications for different robots, and then a wide range of interactive visualizations for different machines

Dequn Teng 1 Feb 13, 2022