Command line parser for common log format (Nginx default).

Related tags

Command-line Toolspythonnginxcommonlogging
Overview

Logparser

Command line parser for common log format (Nginx default).

Usage

It counts most important data: referrers, operating systems, browsers and daily unique visitors (IPs).

# Console output
python parse.py sitename.log.gz

# HTML output
python parse.py sitename.log.gz --html ~/sitename/logs.html

Install and update PIP packages.

pip install -U -r requirements.txt

Speed

  • logparser 24,249/s
  • GoAccess 6,234/s

Outputs

  • HTML output is based on Jinja2 templates. It can be improved as you see fit.
  • Console output for Subreply for a fews days in Sep, 2021:
----- Days
 1402 2021-09-14 Tuesday
  893 2021-09-15 Wednesday
  797 2021-09-16 Thursday
  857 2021-09-17 Friday
  951 2021-09-18 Saturday
  945 2021-09-19 Sunday
  869 2021-09-20 Monday

----- Browsers
    2 Chrome Mobile iOS
    2 Bytespider
    2 HeadlessChrome
    3 Links
    4 Firefox iOS
    4 UC Browser
    6 IE
    7 Go-http-client
    7 Opera
    8 Other
   20 Chrome Mobile WebView
   20 Edge
   28 Firefox Mobile
   30 Safari
   42 Chrome Mobile
   56 Mobile Safari
  111 Firefox
 3468 Chrome

----- Operating Systems
   15 Other
   22 Ubuntu
   39 Linux
   63 iOS
   91 Android
  101 Mac OS X
 3474 Windows

----- Referrers
    2 twtxt.net
    2 l.instagram.com
    2 lobste.rs/s/bffayk/what_are_you_doing_this_weekend
    3 ro.linkedin.com
    3 199.247.2.88:80
    3 t.co/sv4adihlka
    3 lobste.rs/s/fe2eph/what_are_you_doing_this_weekend
    4 news.ycombinator.com
    4 1mb.club
    4 old.reddit.com/r/internetisbeautiful
    4 baidu.com
    4 t.co
    5 out.reddit.com/t3_pl9hiu
    5 sjmulder.nl/en/textonly.html
    8 nt
   10 199.247.2.88
   11 lobste.rs
   13 sjmulder.nl
   15 google.com
   16 lucianmarin.com
   49 reddit.com
   85 lobste.rs/s/8fzzmz/what_are_you_doing_this_week
Owner
Lucian Marin
Python developer and all things design.
Lucian Marin
GitHub Repository
Ipylivebash - Run shell script in Jupyter with live output

ipylivebash ipylivebash is a library to run shell script in Jupyter with live ou

Ben Lau 6 Aug 27, 2022
Rdwcli - Car list cli app with python

Rdwcli - Car list cli app with python

Arie Twigt 1 Feb 02, 2022
Make tree planting a part of your daily workflow. 🌳

Continuous Reforestation Make tree planting a part of your daily workflow. 🌳 A GitHub Action for planting trees within your development workflow usin

protontypes 168 Dec 22, 2022
Neovim integration for Google Keep, built using gkeepapi

Gkeep.nvim Neovim integration for Google Keep, built using gkeepapi Requirements Neovim 0.5 Python 3.6+ A patched font (optional. Used for icons) Tabl

Steven Arcangeli 143 Jan 02, 2023
TermPair lets developers securely share and control terminals in real time🔒

View and control terminals from your browser with end-to-end encryption 🔒

Chad Smith 1.5k Jan 05, 2023
Themes for Windows Terminal

Windows Terminal Themes Preview and copy themes for the new Windows Terminal. Use the project at windowsterminalthemes.dev How to use the themes This

Tom 1.1k Jan 03, 2023
A simple CLI to convert snapshots into EAVT log, and EAVT log into SCD.

EAVT helper CLI Simple CLI to convert snapshots into eavt log, and eavt log into slowly changing dimensions Usage Installation Snapshot to EAVT log EA

2 Apr 07, 2022
Regis-ltmpt-auto - Program register ltmpt 2022 automatis

LTMPT Register Otomatis 2022 Program register ltmpt 2022 automatis dibuat untuk

1 Jan 13, 2022
Command-line tool for downloading and extending the RedCaps dataset.

Command-line tool for downloading and extending the RedCaps dataset.

RedCaps dataset 33 Dec 14, 2022
grungegirl is the hacker's drug encyclopedia. programmed in python for maximum modularity and ease of configuration.

grungegirl. cli-based drug search for girls. welcome. grungegirl is aiming to be the premier drug culture application. it is the hacker's encyclopedia

Eristava 10 Oct 02, 2022
A tool to automatically convert old string literal formatting to f-strings

flynt - string formatting converter flynt is a command line tool to automatically convert a project's Python code from old "%-formatted" and .format(.

Elijah K 551 Jan 06, 2023
Helping you manage your data science projects sanely.

PyDS CLI Helping you manage your data science projects sanely. Requirements Anaconda/Miniconda/Miniforge/Mambaforge (Mambaforge recommended!) git on y

Eric Ma 16 Apr 25, 2022
open a remote repo locally quickly

A command line tool to peek a remote repo hosted on github or gitlab locally and view it in your favorite editor. The tool handles cleanup of the repo once you exit your editor.

Rahul Nair 44 Dec 16, 2022
Analyzing the most strategic words to guess on Wordle, based on letter frequency distributions

wordle-analysis Evaluating different heuristics to determine the most effective solving strategy and building an AI-powered assistant tool to help you

Sejal Dua 9 Feb 27, 2022
git-partial-submodule is a command-line script for setting up and working with submodules while enabling them to use git's partial clone and sparse checkout features.

Partial Submodules for Git git-partial-submodule is a command-line script for setting up and working with submodules while enabling them to use git's

Nathan Reed 15 Sep 22, 2022
Declarative CLIs with argparse and dataclasses

argparse_dataclass Declarative CLIs with argparse and dataclasses. Features Features marked with a ✓ are currently implemented; features marked with a

Mike DePalatis 29 Dec 06, 2022
Customisable pharmacokinetic model accessible via bash CLI allowing for variable dose calculations as well as intravenous and subcutaneous administration calculations

Pharmacokinetic Modelling Group Project A PharmacoKinetic (PK) modelling function for analysis of injected solute dynamics over time, developed by Gro

1 Oct 24, 2021
CLI tool to develop StarkNet projects written in Cairo

⛵ Nile Navigate your StarkNet projects written in Cairo. Installation pip install cairo-nile Usage Install Cairo Use nile to install a given version o

Martín Triay 305 Dec 30, 2022
A python-based terminal application that displays current cryptocurrency prices

CryptoAssetPrices A python-based terminal application that displays current cryptocurrency prices. Covered Cryptocurrencies Bitcoin (BTC) Ethereum (ET

3 Apr 21, 2022
🎈 A Mini CLI-based Operating System simulator

Mini OS Simulator Summary 🎈 A Mini CLI-based Operating System simulator, well, not really. It simulates a file system with songs and movies and stuff

Jaiyank S. 3 Feb 14, 2022