Command line parser for common log format (Nginx default).

Related tags

Command-line Toolspythonnginxcommonlogging
Overview

Logparser

Command line parser for common log format (Nginx default).

Usage

It counts most important data: referrers, operating systems, browsers and daily unique visitors (IPs).

# Console output
python parse.py sitename.log.gz

# HTML output
python parse.py sitename.log.gz --html ~/sitename/logs.html

Install and update PIP packages.

pip install -U -r requirements.txt

Speed

  • logparser 24,249/s
  • GoAccess 6,234/s

Outputs

  • HTML output is based on Jinja2 templates. It can be improved as you see fit.
  • Console output for Subreply for a fews days in Sep, 2021:
----- Days
 1402 2021-09-14 Tuesday
  893 2021-09-15 Wednesday
  797 2021-09-16 Thursday
  857 2021-09-17 Friday
  951 2021-09-18 Saturday
  945 2021-09-19 Sunday
  869 2021-09-20 Monday

----- Browsers
    2 Chrome Mobile iOS
    2 Bytespider
    2 HeadlessChrome
    3 Links
    4 Firefox iOS
    4 UC Browser
    6 IE
    7 Go-http-client
    7 Opera
    8 Other
   20 Chrome Mobile WebView
   20 Edge
   28 Firefox Mobile
   30 Safari
   42 Chrome Mobile
   56 Mobile Safari
  111 Firefox
 3468 Chrome

----- Operating Systems
   15 Other
   22 Ubuntu
   39 Linux
   63 iOS
   91 Android
  101 Mac OS X
 3474 Windows

----- Referrers
    2 twtxt.net
    2 l.instagram.com
    2 lobste.rs/s/bffayk/what_are_you_doing_this_weekend
    3 ro.linkedin.com
    3 199.247.2.88:80
    3 t.co/sv4adihlka
    3 lobste.rs/s/fe2eph/what_are_you_doing_this_weekend
    4 news.ycombinator.com
    4 1mb.club
    4 old.reddit.com/r/internetisbeautiful
    4 baidu.com
    4 t.co
    5 out.reddit.com/t3_pl9hiu
    5 sjmulder.nl/en/textonly.html
    8 nt
   10 199.247.2.88
   11 lobste.rs
   13 sjmulder.nl
   15 google.com
   16 lucianmarin.com
   49 reddit.com
   85 lobste.rs/s/8fzzmz/what_are_you_doing_this_week
Owner
Lucian Marin
Python developer and all things design.
Lucian Marin
GitHub Repository
Dark powered asynchronous completion framework for neovim/Vim8

deoplete.nvim Dark powered asynchronous completion framework for neovim/Vim8 Note: The development of this plugin is finished. Accepts minor patches a

Shougo 5.9k Dec 30, 2022
Example of a CLI with python - know the extension of your files.

extensionCLI Example of a CLI with python - know the extension of your files. Usage: Install the CLI: pip3 install -e . Run the command with "ext" + t

ItanuRomero 5 Dec 29, 2022
A command line tool to hide and reveal information inside images (works for both PNGs and JPGs)

ImgReRite A command line tool to hide and reveal information inside images (work

Jigyasu 10 Jul 27, 2022
Python package with library and CLI tool for analyzing SeaFlow data

Seaflowpy A Python package for SeaFlow flow cytometer data. Table of Contents Install Read EVT/OPP/VCT Files Command-line Interface Configuration Inte

<a href=[email protected]"> 3 Nov 03, 2021
Python Library and CLI for exporting MySQL databases

expdb Python library and CLI for exporting MySQL databases Installation Pre-requisites MySQL server Python 3.9+ Using git Clone the repository to your

Devansh Singh 1 Nov 29, 2021
A Python module and command line utility for working with web archive data using the WACZ format specification

py-wacz The py-wacz repository contains a Python module and command line utility for working with web archive data using the WACZ format specification

Webrecorder 14 Oct 24, 2022
Convert shellcode generated using pe_2_shellcode to cdb format.

pe2shc-to-cdb This tool will convert shellcode generated using pe_to_shellcode to cdb format. Cdb.exe is a LOLBIN which can help evade detection & app

mrd0x 75 Jan 05, 2023
py-image-dedup is a tool to sort out or remove duplicates within a photo library

py-image-dedup is a tool to sort out or remove duplicates within a photo library. Unlike most other solutions, py-image-dedup intentionally uses an approximate image comparison to also detect duplica

Markus Ressel 96 Jan 02, 2023
Interact with Replit remotely with the Replit CLI

Replit CLI pip install repl-cli Welcome to Replit CLI! With the Replit CLI Application, you can work with your repls locally, including clone, pull,

Shuchir Jain 4 Aug 18, 2022
Amazon Scraper: A command-line tool for scraping Amazon product data

Amazon Product Scraper: 2021 Description A command-line tool for scraping Amazon product data to CSV or JSON format(s). Requirements Python 3 pip3 Ins

49 Nov 15, 2021
Themes for the kitty terminal emulator

Themes for the kitty terminal This is a collection of themes for the kitty terminal emulator. The themes were initially imported from dexpota/kitty-th

Kovid Goyal 190 Jan 05, 2023
A python program to detect your emotion and suggest a playlist.

mood_music A python program to detect your emotion and suggest a playlist. This program is written in python using opencv,FER() and tensorflow This pr

Aditya_Sai 1 Jan 02, 2022
This CLI give the possibility to do a queries in Star Wars API and returns a JSON in a terminal.

Star Wars CLI (swcli) This CLI give the possibility to do a queries in Star Wars API and returns a JSON in a terminal. Install $ pip install swcli Qu

Pery Lemke 5 Nov 05, 2021
Simple CLI interface for linear task manager

Linear CLI (Unmaintained) Simple CLI interface for linear task manager Usage Install: pip install linearcli Setup: Generate a pe

Mike Lyons 1 Jan 07, 2022
A minimal and ridiculously good looking command-line-interface toolkit.

Pyceo Pyceo is a Python package for creating beautiful, composable, and ridiculously good looking command-line-user-interfaces without having to write

Juan-Pablo Scaletti 21 Mar 25, 2022
🕰 The command line tool for scheduling Python scripts

hickory is a simple command line tool for scheduling Python scripts.

Max Humber 146 Dec 07, 2022
A startpage configured aesthetically with terminal-esque link formatting

Terminal-y Startpage Setup Clone the repository, then make an unformatted.txt file following the specifications in example.txt. Run format.py Open ind

belkarx 13 May 01, 2022
Hurry is a CLI tool to speed setting up MoniGoMani HyperStrategy & co. #freqtrade #hyperopting #trading #strategy

Hurry is a CLI tool to speed setting up MoniGoMani HyperStrategy & co. #freqtrade #hyperopting #trading #strategy

10 Dec 29, 2022
A webmining CLI tool & library for python.

minet is a webmining command line tool & library for python (= 3.6) that can be used to collect and extract data from a large variety of web sources

médialab Sciences Po 165 Dec 17, 2022
Projeto Reverse Shell For Python

Use com sabedoria!!! Modo de uso: Linux (inclui Android e Mac): - apt-get update - apt install python3 (ou "python" apenas) - git clone https://github

1 Jan 03, 2022