lsf_stats

Summarize LSF job properties by parsing log files of workflows executed by Snakemake.

Installation

$ pip install lsf_stats

Usage

$ lsf_stats --help
Usage: lsf_stats [OPTIONS] COMMAND [ARGS]...

  Summarize LSF job properties by parsing log files.

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  gather     Aggregate information from log files in single dataframe.
  summarize  Summarize and visualize aggregated information.

Example

Assume that you executed your Snakemake workflow using the lsf-profile and all generated log files are stored in the directory ./logs/:

$ snakemake --profile lsf
[..]

You can then quickly aggregate resource, rule and other types of information about the workflow execution into a single dataframe:

$ lsf_stats gather -o workflow_stats.csv.gz ./logs/
[..]

This dataframe can then be summarized in various ways:

$ lsf_stats summarize \
    --query 'status == "Successfully completed."' \
    --split-wildcards \
    --grouping-variable category \
    workflow_stats.csv.gz
[..]

For example, the following plots will be generated:

Job execution	Job resources

Summarize LSF job properties by parsing log files.

Related tags

Overview

lsf_stats

Installation

Usage

Example

Owner

Kim

Monitor and log Network and Disks statistics in MegaBytes per second.

Outlog it's a library to make logging a simple task

Discord-Image-Logger - Discord Image Logger With Python

Debugging-friendly exceptions for Python

A simple, transparent, open-source key logger, written in Python, for tracking your own key-usage statistics.

🐑 Syslog Simulator hazır veya kullanıcıların eklediği logları belirtilen adreslere ve port'a seçilen döngüde syslog ile gönderilmesini sağlayan araçtır. | 🇹🇷

Progressbar 2 - A progress bar for Python 2 and Python 3 - "pip install progressbar2"

A python library used to interact with webots robocup game web logs

HTTP(s) "monitoring" webpage via FastAPI+Jinja2. Inspired by https://github.com/RaymiiOrg/bash-http-monitoring

Robust and effective logging for Python 2 and 3.

This open-source python3 script is a builder to the very popular token logger that is on my github that many people use.

Token Logger with python

A demo of Prometheus+Grafana for monitoring an ML model served with FastAPI.

Lazy Profiler is a simple utility to collect CPU, GPU, RAM and GPU Memory stats while the program is running.

Structured Logging for Python

Splunk Add-On to collect audit log events from Github Enterprise Cloud

A Python library that tees the standard output & standard error from the current process to files on disk, while preserving terminal semantics

A colored formatter for the python logging module

This is a DemoCode for parsing through large log files and triggering an email whenever there's an error.

ScreenshotLogger works just like a keylogger but instead of capturing keystroke,it captures the screen, stores it or sends via email