Create matplotlib visualizations from the command-line

Overview

MatplotCLI

Create matplotlib visualizations from the command-line

MatplotCLI is a simple utility to quickly create plots from the command-line, leveraging Matplotlib.

plt "scatter(x,y,5,alpha=0.05); axis('scaled')" < sample.json

plt "hist(x,30)" < sample.json

MatplotCLI accepts both JSON lines and arrays of JSON objects as input. Look at the recipes section to learn how to handle other formats like CSV.

MatplotCLI executes python code (passed as argument) where some handy imports are already done (e.g. from matplotlib.pyplot import *) and where the input JSON data is already parsed and available in variables, making plotting easy. Please refer to matplotlib.pyplot's reference and tutorial for comprehensive documentation about the plotting commands.

Data from the input JSON is made available in the following way. Given the input myfile.json:

{"a": 1, "b": 2}
{"a": 10, "b": 20}
{"a": 30, "c$d": 40}

The following variables are made available:

data = {
    "a": [1, 10, 30],
    "b": [2, 20, None],
    "c_d": [None, None, 40]
}

a = [1, 10, 30]
b = [2, 20, None]
c_d = [None, None, 40]

col_names = ("a", "b", "c_d")

So, for a scatter plot a vs b, you could simply do:

plt "scatter(a,b); title('a vs b')" < myfile.json

Notice that the names of JSON properties are converted into valid Python identifiers whenever they are not (e.g. c$d was converted into c_d).

Execution flow

  1. Import matplotlib and other libs;
  2. Read JSON data from standard input;
  3. Execute user code;
  4. Show the plot.

All steps (except step 3) can be skipped through command-line options.

Installation

The easiest way to install MatplotCLI is from pip:

pip install matplotcli

Recipes and Examples

Plotting JSON data

MatplotCLI natively supports JSON lines:

echo '
    {"a":0, "b":1}
    {"a":1, "b":0}
    {"a":3, "b":3}' |
plt "plot(a,b)"

and arrays of JSON objects:

echo '[
    {"a":0, "b":1},
    {"a":1, "b":0},
    {"a":3, "b":3}]' |
plt "plot(a,b)"

Plotting from a csv

SPyQL is a data querying tool that allows running SQL queries with Python expressions on top of different data formats. Here, SPyQL is reading a CSV file, automatically detecting if there's an header row, the dialect and the data type of each column, and converting the output to JSON lines before handing over to MatplotCLI.

cat my.csv | spyql "SELECT * FROM csv TO json" | plt "plot(x,y)"

Plotting from a yaml/xml/toml

yq converts yaml, xml and toml files to json, allowing to easily plot any of these with MatplotCLI.

cat file.yaml | yq -c | plt "plot(x,y)"
cat file.xml | xq -c | plt "plot(x,y)"
cat file.toml | tomlq -c | plt "plot(x,y)"

Plotting from a parquet file

parquet-tools allows dumping a parquet file to JSON format. jq -c makes sure that the output has 1 JSON object per line before handing over to MatplotCLI.

parquet-tools cat --json my.parquet | jq -c | plt "plot(x,y)"

Plotting from a database

Databases CLIs typically have an option to output query results in CSV format (e.g. psql --csv -c query for PostgreSQL, sqlite3 -csv -header file.db query for SQLite).

Here we are visualizing how much space each namespace is taking in a PostgreSQL database. SPyQL converts CSV output from the psql client to JSON lines, and makes sure there are no more than 10 items, aggregating the smaller namespaces in an All others category. Finally, MatplotCLI makes a pie chart based on the space each namespace is taking.

psql -U myuser mydb --csv  -c '
    SELECT
        N.nspname,
        sum(pg_relation_size(C.oid))*1e-6 AS size_mb
    FROM pg_class C
    LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
    GROUP BY 1
    ORDER BY 2 DESC' |
spyql "
    SELECT
        nspname if row_number < 10 else 'All others' as name,
        sum_agg(size_mb) AS size_mb
    FROM csv
    GROUP BY 1
    TO json" |
plt "
nice_labels = ['{0}\n{1:,.0f} MB'.format(n,s) for n,s in zip(name,size_mb)];
pie(size_mb, labels=nice_labels, autopct='%1.f%%', pctdistance=0.8, rotatelabels=True)"

Plotting a function

Disabling reading from stdin and generating the output using numpy.

plt --no-input "
x = np.linspace(-1,1,2000);
y = x*np.sin(1/x);
plot(x,y);
axis('scaled');
grid(True)"

Saving the plot to an image

Saving the output without showing the interactive window.

cat sample.json |
plt --no-show "
hist(x,30);
savefig('myimage.png', bbox_inches='tight')"

Plot of the global temperature

Here's a complete pipeline from getting the data to transforming and plotting it:

  1. Downloading a CSV file with curl;
  2. Skipping the first row with sed;
  3. Grabbing the year column and 12 columns with monthly temperatures to an array and converting to JSON lines format using SPyQL;
  4. Exploding the monthly array with SPyQL (resulting in 12 rows per year) while removing invalid monthly measurements;
  5. Plotting with MatplotCLI .
curl https://data.giss.nasa.gov/gistemp/tabledata_v4/GLB.Ts+dSST.csv |
sed 1d |
spyql "
  SELECT Year, cols[1:13] AS temps
  FROM csv
  TO json" |
spyql "
  SELECT
    json->Year + ((row_number-1)%12)/12 AS year,
    json->temps AS temp
  FROM json
  EXPLODE json->temps
  WHERE json->temps is not Null
  TO json" |
plt "
scatter(year, temp, 2, temp);
xlabel('Year');
ylabel('Temperature anomaly w.r.t. 1951-80 (ºC)');
title('Global surface temperature (land and ocean)')"

You might also like...
These data visualizations were created for my introductory computer science course using Python
These data visualizations were created for my introductory computer science course using Python

Homework 2: Matplotlib and Data Visualization Overview These data visualizations were created for my introductory computer science course using Python

These data visualizations were created as homework for my CS40 class. I hope you enjoy!
These data visualizations were created as homework for my CS40 class. I hope you enjoy!

Data Visualizations These data visualizations were created as homework for my CS40 class. I hope you enjoy! Nobel Laureates by their Country of Birth

Generate visualizations of GitHub user and repository statistics using GitHub Actions.

GitHub Stats Visualization Generate visualizations of GitHub user and repository statistics using GitHub Actions. This project is currently a work-in-

A Python package for caclulations and visualizations in geological sciences.

geo_calcs A Python package for caclulations and visualizations in geological sciences. Free software: MIT license Documentation: https://geo-calcs.rea

Make scripted visualizations in blender
Make scripted visualizations in blender

Scripted visualizations in blender The goal of this project is to script 3D scientific visualizations using blender. To achieve this, we aim to bring

Standardized plots and visualizations in Python
Standardized plots and visualizations in Python

Standardized plots and visualizations in Python pltviz is a Python package for standardized visualization. Routine and novel plotting approaches are f

Generate visualizations of GitHub user and repository statistics using GitHub Actions.

GitHub Stats Visualization Generate visualizations of GitHub user and repository statistics using GitHub Actions. This project is currently a work-in-

Visualizations of some specific solutions of different differential equations.
Visualizations of some specific solutions of different differential equations.

Diff_sims Visualizations of some specific solutions of different differential equations. Heat Equation in 1 Dimension (A very beautiful and elegant ex

Data aggregated from the reports found at the MCPS COVID Dashboard into a set of visualizations.

Montgomery County Public Schools COVID-19 Visualizer Contents About this project Data Support this project About this project Data All data we use can

Comments
  • stats about input data

    stats about input data

    option to print simple statistics about the input data. e.g. for each field

    • number of missing values
    • number of distinct values
    • avg, min, max (if numeric)
    • number of nan, inf (if float)
    • ...
    enhancement good first issue 
    opened by dcmoura 0
Releases(v0.2.0)
Owner
Daniel Moura
Daniel Moura
NorthPitch is a python soccer plotting library that sits on top of Matplotlib

NorthPitch is a python soccer plotting library that sits on top of Matplotlib.

Devin Pleuler 30 Feb 22, 2022
The Spectral Diagram (SD) is a new tool for the comparison of time series in the frequency domain

The Spectral Diagram (SD) is a new tool for the comparison of time series in the frequency domain. The SD provides a novel way to display the coherence function, power, amplitude, phase, and skill sc

Mabel 3 Oct 10, 2022
Geocoding library for Python.

geopy geopy is a Python client for several popular geocoding web services. geopy makes it easy for Python developers to locate the coordinates of addr

geopy 3.8k Jan 02, 2023
Create Badges with stats of Scratch User, Project and Studio. Use those badges in Github readmes, etc.

Scratch-Stats-Badge Create customized Badges with stats of Scratch User, Studio or Project. Use those badges in Github readmes, etc. Examples Document

Siddhesh Chavan 5 Aug 28, 2022
Gaphas is the diagramming widget library for Python.

Gaphas Gaphas is the diagramming widget library for Python. Gaphas is a library that provides the user interface component (widget) for drawing diagra

Gaphor 144 Dec 14, 2022
PyPassword is a simple follow up to PyPassphrase

PyPassword PyPassword is a simple follow up to PyPassphrase. After finishing that project it occured to me that while some may wish to use that option

Scotty 2 Jan 22, 2022
BrowZen correlates your emotional states with the web sites you visit to give you actionable insights about how you spend your time browsing the web.

BrowZen BrowZen correlates your emotional states with the web sites you visit to give you actionable insights about how you spend your time browsing t

Nick Bild 36 Sep 28, 2022
Generate knowledge graphs with interesting geometries, like lattices

Geometric Graphs Generate knowledge graphs with interesting geometries, like lattices. Works on Python 3.9+ because it uses cool new features. Get out

Charles Tapley Hoyt 5 Jan 03, 2022
Interactive Data Visualization in the browser, from Python

Bokeh is an interactive visualization library for modern web browsers. It provides elegant, concise construction of versatile graphics, and affords hi

Bokeh 17.1k Dec 31, 2022
An open-source tool for visual and modular block programing in python

PyFlow PyFlow is an open-source tool for modular visual programing in python ! Although for now the tool is in Beta and features are coming in bit by

1.1k Jan 06, 2023
Flame Graphs visualize profiled code

Flame Graphs visualize profiled code

Brendan Gregg 14.1k Jan 03, 2023
Bioinformatics tool for exploring RNA-Protein interactions

Explore RNA-Protein interactions. RNPFind is a bioinformatics tool. It takes an RNA transcript as input and gives a list of RNA binding protein (RBP)

Nahin Khan 3 Jan 27, 2022
2021 grafana arbitrary file read

2021_grafana_arbitrary_file_read base on pocsuite3 try 40 default plugins of grafana alertlist annolist barchart cloudwatch dashlist elasticsearch gra

ATpiu 5 Nov 09, 2022
Package managers visualization

Software Galaxies This repository combines visualizations of major software package managers. All visualizations are available here: http://anvaka.git

Andrei Kashcha 1.4k Dec 22, 2022
Visualization of numerical optimization algorithms

Visualization of numerical optimization algorithms

Zhengxia Zou 46 Dec 01, 2022
Farhad Davaripour, Ph.D. 1 Jan 05, 2022
A workshop on data visualization in Python with notebooks and exercises for following along.

Beyond the Basics: Data Visualization in Python The human brain excels at finding patterns in visual representations, which is why data visualizations

Stefanie Molin 162 Dec 05, 2022
ScisorWiz: Differential Isoform Visualizer for Long-Read RNA Sequencing Data

ScisorWiz: Vizualizer for Differential Isoform Expression README ScisorWiz is a linux-based R-package for visualizing differential isoform expression

Alexander Stein 6 Oct 04, 2022
Smarthome Dashboard with Grafana & InfluxDB

Smarthome Dashboard with Grafana & InfluxDB This is a complete overhaul of my Raspberry Dashboard done with Flask. I switched from sqlite to InfluxDB

6 Oct 20, 2022
Collection of scripts for making high quality beautiful math-related posters.

Poster Collection of scripts for making high quality beautiful math-related posters. The poster can have as large printing size as 3x2 square feet wit

Nattawut Phetmak 3 Jun 09, 2022