Command line parser for common log format (Nginx default).

Related tags

Command-line Toolspythonnginxcommonlogging
Overview

Logparser

Command line parser for common log format (Nginx default).

Usage

It counts most important data: referrers, operating systems, browsers and daily unique visitors (IPs).

# Console output
python parse.py sitename.log.gz

# HTML output
python parse.py sitename.log.gz --html ~/sitename/logs.html

Install and update PIP packages.

pip install -U -r requirements.txt

Speed

  • logparser 24,249/s
  • GoAccess 6,234/s

Outputs

  • HTML output is based on Jinja2 templates. It can be improved as you see fit.
  • Console output for Subreply for a fews days in Sep, 2021:
----- Days
 1402 2021-09-14 Tuesday
  893 2021-09-15 Wednesday
  797 2021-09-16 Thursday
  857 2021-09-17 Friday
  951 2021-09-18 Saturday
  945 2021-09-19 Sunday
  869 2021-09-20 Monday

----- Browsers
    2 Chrome Mobile iOS
    2 Bytespider
    2 HeadlessChrome
    3 Links
    4 Firefox iOS
    4 UC Browser
    6 IE
    7 Go-http-client
    7 Opera
    8 Other
   20 Chrome Mobile WebView
   20 Edge
   28 Firefox Mobile
   30 Safari
   42 Chrome Mobile
   56 Mobile Safari
  111 Firefox
 3468 Chrome

----- Operating Systems
   15 Other
   22 Ubuntu
   39 Linux
   63 iOS
   91 Android
  101 Mac OS X
 3474 Windows

----- Referrers
    2 twtxt.net
    2 l.instagram.com
    2 lobste.rs/s/bffayk/what_are_you_doing_this_weekend
    3 ro.linkedin.com
    3 199.247.2.88:80
    3 t.co/sv4adihlka
    3 lobste.rs/s/fe2eph/what_are_you_doing_this_weekend
    4 news.ycombinator.com
    4 1mb.club
    4 old.reddit.com/r/internetisbeautiful
    4 baidu.com
    4 t.co
    5 out.reddit.com/t3_pl9hiu
    5 sjmulder.nl/en/textonly.html
    8 nt
   10 199.247.2.88
   11 lobste.rs
   13 sjmulder.nl
   15 google.com
   16 lucianmarin.com
   49 reddit.com
   85 lobste.rs/s/8fzzmz/what_are_you_doing_this_week
Owner
Lucian Marin
Python developer and all things design.
Lucian Marin
GitHub Repository
ddgr is a cmdline utility to search DuckDuckGo (html version) from the terminal

ddgr is a cmdline utility to search DuckDuckGo (html version) from the terminal. While googler is extremely popular among cmdline users, in many forums the need of a similar utility for privacy-aware

Piña Colada 2.5k Dec 25, 2022
Universal Command Line Interface for Amazon Web Services

This package provides a unified command line interface to Amazon Web Services.

Amazon Web Services 13.3k Jan 07, 2023
telescope.nvim is a highly extendable fuzzy finder over lists.

telescope.nvim is a highly extendable fuzzy finder over lists. Built on the latest awesome features from neovim core. Telescope is centered around modularity, allowing for easy customization.

nvim-telescope 8.4k Jan 05, 2023
A dilligent command line tool to publish ads on ebay-kleinanzeigen.de

kleinanzeigen-bot Feedback and high-quality pull requests are highly welcome! About Installation Usage Development Notes License About kleinanzeigen-b

83 Dec 26, 2022
Euporie is a text-based user interface for running and editing Jupyter notebooks

Euporie is a text-based user interface for running and editing Jupyter notebooks

781 Jan 01, 2023
Todo list console based application. Todo's save to a seperate file.

Todo list console based application. Todo's save to a seperate file.

1 Dec 24, 2021
Pymongo based CLI client, to run operation on existing databases and collections

Mongodb-Operations-Console Pymongo based CLI client, to run operation on existing databases and collections Program developed by Gustavo Wydler Azuaga

Gus 1 Dec 01, 2021
Python3 parser for Apple's crash reports

pyCrashReport in intended for analyzing crash reports from Apple devices into a clearer view, without all the thread listing and loaded images, just the actual data you really need to debug the probl

7 Aug 19, 2022
Professor Wordlist is a free open source command line tool written in python

Professor Wordlist is a free open source command line tool written in python, With the aim of generating custom wordlists with a variety of unique parameters and functions providing many possibilitie

オークO A K Z E H オーク 1 Oct 28, 2021
Install python modules from pypi from a previous date in history

pip-rewind is a command-line tool that can rewind pypi module versions (given as command-line arguments or read from a requirements.txt file) to a previous date in time.

Amar Paul 4 Jul 03, 2021
Enriches Click with option groups, constraints, command aliases, help sections for subcommands, themes for --help and other stuff.

Enriches Click with option groups, constraints, command aliases, help sections for subcommands, themes for --help and other stuff.

Gianluca Gippetto 62 Dec 22, 2022
Command Line Manager + Interactive Shell for Python Projects

Manage Command Line Manager + Interactive Shell for Python Projects

Python Manage 123 Aug 28, 2022
Wappalyzer CLI tool to find Web Technologies

Wappalyzer CLI tool to find Web Technologies

GOKUL A.P 17 Dec 15, 2022
Basic python tools to generate shellcode runner in vba

vba_bin_runner Basic python tools to generate shellcode runner in vba. The stub use ZwAllocateVirtualMemory to allocate memory, RtlMoveMemory to write

4 Aug 24, 2021
Trans is a dependency-free CLI for Google Translate

Trans is a dependency-free CLI for Google Translate

11 Jan 04, 2022
Unconventional ways to save an Image

Unexpected Image Saves Unconventional ways to save an image 😄 Have you ever been bored by the same old .png, .jpg, .jpeg, .gif and all other image ex

Eric Mendes 15 Nov 06, 2022
Chopper: An Automated Security Headers Analyzer

____ _ _ / ___| |__ ___ _ __ _ __ ___ _ __| | | | | '_ \ / _ \| '_ \| '_ \ / _ \ '__| | | |___| | | | (_) |

Kamran Saifullah (Frog Man) 2 Nov 27, 2022
Dart Version Manager CLI implemented with Python and Typer.

Dart Version Manager Dart Version Manager CLI implemented with Python and Typer Usage: $ dvm [OPTIONS] COMMAND [ARGS]... Options: --install-completion

EducUp 6 Jun 26, 2022
A new kind of Progress Bar, with real time throughput, eta and very cool animations!

A new kind of Progress Bar, with real time throughput, eta and very cool animations!

Rogério Sampaio de Almeida 4.1k Jan 08, 2023
The command line interface for Gradient - Gradient is an an end-to-end MLOps platform

Gradient CLI Get started: Create Account • Install CLI • Tutorials • Docs Resources: Website • Blog • Support • Contact Sales Gradient is an an end-to

Paperspace 58 Dec 06, 2022