Command line parser for common log format (Nginx default).

Overview

Logparser

Command line parser for common log format (Nginx default).

Usage

It counts most important data: referrers, operating systems, browsers and daily unique visitors (IPs).

# Console output
python parse.py sitename.log.gz

# HTML output
python parse.py sitename.log.gz --html ~/sitename/logs.html

Install and update PIP packages.

pip install -U -r requirements.txt

Speed

  • logparser 24,249/s
  • GoAccess 6,234/s

Outputs

  • HTML output is based on Jinja2 templates. It can be improved as you see fit.
  • Console output for Subreply for a fews days in Sep, 2021:
----- Days
 1402 2021-09-14 Tuesday
  893 2021-09-15 Wednesday
  797 2021-09-16 Thursday
  857 2021-09-17 Friday
  951 2021-09-18 Saturday
  945 2021-09-19 Sunday
  869 2021-09-20 Monday

----- Browsers
    2 Chrome Mobile iOS
    2 Bytespider
    2 HeadlessChrome
    3 Links
    4 Firefox iOS
    4 UC Browser
    6 IE
    7 Go-http-client
    7 Opera
    8 Other
   20 Chrome Mobile WebView
   20 Edge
   28 Firefox Mobile
   30 Safari
   42 Chrome Mobile
   56 Mobile Safari
  111 Firefox
 3468 Chrome

----- Operating Systems
   15 Other
   22 Ubuntu
   39 Linux
   63 iOS
   91 Android
  101 Mac OS X
 3474 Windows

----- Referrers
    2 twtxt.net
    2 l.instagram.com
    2 lobste.rs/s/bffayk/what_are_you_doing_this_weekend
    3 ro.linkedin.com
    3 199.247.2.88:80
    3 t.co/sv4adihlka
    3 lobste.rs/s/fe2eph/what_are_you_doing_this_weekend
    4 news.ycombinator.com
    4 1mb.club
    4 old.reddit.com/r/internetisbeautiful
    4 baidu.com
    4 t.co
    5 out.reddit.com/t3_pl9hiu
    5 sjmulder.nl/en/textonly.html
    8 nt
   10 199.247.2.88
   11 lobste.rs
   13 sjmulder.nl
   15 google.com
   16 lucianmarin.com
   49 reddit.com
   85 lobste.rs/s/8fzzmz/what_are_you_doing_this_week
Owner
Lucian Marin
Python developer and all things design.
Lucian Marin
A python script that enables a raspberry pi sd card through the CLI and automates the process of configuring network details and ssh.

This project is one script (wpa_helper.py) written in python that will allow for the user to automate the proccess of setting up a new boot disk and configuring ssh and network settings for the pi

Theo Kirby 6 Jun 24, 2021
Universal Command Line Interface for Amazon Web Services

This package provides a unified command line interface to Amazon Web Services.

Amazon Web Services 13.3k Jan 07, 2023
Autosub - Command-line utility for auto-generating subtitles for any video file

Auto-generated subtitles for any video Autosub is a utility for automatic speech recognition and subtitle generation. It takes a video or an a

Anastasis Germanidis 3.9k Jan 05, 2023
QueraToCSV is a simple python CLI project to convert the Quera results file into CSV files.

Quera is an Iranian Learning management system (LMS) that has an online judge for programming languages. Some Iranian universities use it to automate the evaluation of programming assignments.

Amirmahdi Namjoo 16 Nov 11, 2022
This is a CLI utility that allows you to view RedFlagDeals.com on the command line.

RFD Description Motivation Installation Usage View Hot Deals View and Sort Hot Deals Search Advanced View Posts Shell Completion bash zsh Description

Dave G 8 Nov 29, 2022
A tool to manage the study of courses at the university.

todo-cli A tool to manage the study of courses at the university

Quentin 6 Aug 01, 2022
Python script to tabulate data formats like json, csv, html, etc

pyT PyT is a a command line tool and as well a library for visualising various data formats like: JSON HTML Table CSV XML, etc. Features Print table o

Mobolaji Abdulsalam 1 Dec 30, 2021
Pyreadline3 - Windows implementation of the GNU readline library

pyreadline3 The pyreadline3 package is based on the stale package pyreadline loc

32 Jan 06, 2023
Convert shellcode generated using pe_2_shellcode to cdb format.

pe2shc-to-cdb This tool will convert shellcode generated using pe_to_shellcode to cdb format. Cdb.exe is a LOLBIN which can help evade detection & app

mrd0x 75 Jan 05, 2023
adds flavor of interactive filtering to the traditional pipe concept of UNIX shell

percol __ ____ ___ ______________ / / / __ \/ _ \/ ___/ ___/ __ \/ / / /_/ / __/ / / /__/ /_/ / / / .__

Masafumi Oyamada 3.2k Jan 07, 2023
🎮 An easy to use tool to change the mapping of your input device buttons.

Input Remapper Formerly Key Mapper An easy to use tool to change the mapping of your input device buttons. Supports mice, keyboards, gamepads, X11, Wa

Tobi 1.9k Jan 05, 2023
Shortcut-Maker - It is a tool that can be set to run any tool with a single command

Shortcut-Maker It is a tool that can be set to run any tool with a single command Coded by Dave Smith(Owner of Sl Cyber Warriors) Command list 👇 pkg

Dave Smith 10 Sep 14, 2022
A simple CLI application helps you to find giant files that are eating up your system storage

Large file finder Sometimes it's very hard to find if some giant files are eating up your system storage. We might need to hunt those down. This simpl

Rahul Baruri 5 Nov 18, 2022
The project help you to quickly build layouts in terminal,cross-platform

The project help you to quickly build layouts in terminal,cross-platform

gojuukaze 133 Nov 30, 2022
liquidctl – liquid cooler control Cross-platform tool and drivers for liquid coolers and other devices

Cross-platform CLI and Python drivers for AIO liquid coolers and other devices

1.7k Jan 08, 2023
CLI client for RFC 4226's HOTP and RFC 6238's TOTP.

One Time Password (OTP, TOTP/HOTP) OTP serves as additional protection in case of password leaks. onetimepass allows you to manage OTP codes and gener

Apptension 4 Jan 05, 2022
Tools crack instagram + fb ayok dicoba keburu premium 😁

FITUR INSTALLASI [1] pkg update && pkg upgrade [2] pkg install git [3] pkg install python [4] pkg install python2 [5] pkg install nano [6]

Jeeck 1 Dec 11, 2021
A fantasy life simulator and role-playing game hybrid distributed as CLI, written in Python 3.

Life is Fantasy Epic (LIFE) A fantasy life simulator and role-playing game hybrid distributed as CLI, written in Python 3. This repository will be pro

Pawitchaya Chaloeijanya 2 Oct 24, 2021
Doing set operations on files considered as sets of lines

CLI tool that can be used to do set operations like union on files considering them as a set of lines. Notes It ignores all empty lines with whitespac

Partho 11 Sep 06, 2022
CLI tool to develop StarkNet projects written in Cairo

OpenZeppelin Nile ⛵ Navigate your StarkNet projects written in Cairo. Getting started Create a folder for your project and cd into it: mkdir myproject

OpenZeppelin 305 Dec 30, 2022