Create matplotlib visualizations from the command-line

Overview

MatplotCLI

Create matplotlib visualizations from the command-line

MatplotCLI is a simple utility to quickly create plots from the command-line, leveraging Matplotlib.

plt "scatter(x,y,5,alpha=0.05); axis('scaled')" < sample.json

plt "hist(x,30)" < sample.json

MatplotCLI accepts both JSON lines and arrays of JSON objects as input. Look at the recipes section to learn how to handle other formats like CSV.

MatplotCLI executes python code (passed as argument) where some handy imports are already done (e.g. from matplotlib.pyplot import *) and where the input JSON data is already parsed and available in variables, making plotting easy. Please refer to matplotlib.pyplot's reference and tutorial for comprehensive documentation about the plotting commands.

Data from the input JSON is made available in the following way. Given the input myfile.json:

{"a": 1, "b": 2}
{"a": 10, "b": 20}
{"a": 30, "c$d": 40}

The following variables are made available:

data = {
    "a": [1, 10, 30],
    "b": [2, 20, None],
    "c_d": [None, None, 40]
}

a = [1, 10, 30]
b = [2, 20, None]
c_d = [None, None, 40]

col_names = ("a", "b", "c_d")

So, for a scatter plot a vs b, you could simply do:

plt "scatter(a,b); title('a vs b')" < myfile.json

Notice that the names of JSON properties are converted into valid Python identifiers whenever they are not (e.g. c$d was converted into c_d).

Execution flow

  1. Import matplotlib and other libs;
  2. Read JSON data from standard input;
  3. Execute user code;
  4. Show the plot.

All steps (except step 3) can be skipped through command-line options.

Installation

The easiest way to install MatplotCLI is from pip:

pip install matplotcli

Recipes and Examples

Plotting JSON data

MatplotCLI natively supports JSON lines:

echo '
    {"a":0, "b":1}
    {"a":1, "b":0}
    {"a":3, "b":3}' |
plt "plot(a,b)"

and arrays of JSON objects:

echo '[
    {"a":0, "b":1},
    {"a":1, "b":0},
    {"a":3, "b":3}]' |
plt "plot(a,b)"

Plotting from a csv

SPyQL is a data querying tool that allows running SQL queries with Python expressions on top of different data formats. Here, SPyQL is reading a CSV file, automatically detecting if there's an header row, the dialect and the data type of each column, and converting the output to JSON lines before handing over to MatplotCLI.

cat my.csv | spyql "SELECT * FROM csv TO json" | plt "plot(x,y)"

Plotting from a yaml/xml/toml

yq converts yaml, xml and toml files to json, allowing to easily plot any of these with MatplotCLI.

cat file.yaml | yq -c | plt "plot(x,y)"
cat file.xml | xq -c | plt "plot(x,y)"
cat file.toml | tomlq -c | plt "plot(x,y)"

Plotting from a parquet file

parquet-tools allows dumping a parquet file to JSON format. jq -c makes sure that the output has 1 JSON object per line before handing over to MatplotCLI.

parquet-tools cat --json my.parquet | jq -c | plt "plot(x,y)"

Plotting from a database

Databases CLIs typically have an option to output query results in CSV format (e.g. psql --csv -c query for PostgreSQL, sqlite3 -csv -header file.db query for SQLite).

Here we are visualizing how much space each namespace is taking in a PostgreSQL database. SPyQL converts CSV output from the psql client to JSON lines, and makes sure there are no more than 10 items, aggregating the smaller namespaces in an All others category. Finally, MatplotCLI makes a pie chart based on the space each namespace is taking.

psql -U myuser mydb --csv  -c '
    SELECT
        N.nspname,
        sum(pg_relation_size(C.oid))*1e-6 AS size_mb
    FROM pg_class C
    LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
    GROUP BY 1
    ORDER BY 2 DESC' |
spyql "
    SELECT
        nspname if row_number < 10 else 'All others' as name,
        sum_agg(size_mb) AS size_mb
    FROM csv
    GROUP BY 1
    TO json" |
plt "
nice_labels = ['{0}\n{1:,.0f} MB'.format(n,s) for n,s in zip(name,size_mb)];
pie(size_mb, labels=nice_labels, autopct='%1.f%%', pctdistance=0.8, rotatelabels=True)"

Plotting a function

Disabling reading from stdin and generating the output using numpy.

plt --no-input "
x = np.linspace(-1,1,2000);
y = x*np.sin(1/x);
plot(x,y);
axis('scaled');
grid(True)"

Saving the plot to an image

Saving the output without showing the interactive window.

cat sample.json |
plt --no-show "
hist(x,30);
savefig('myimage.png', bbox_inches='tight')"

Plot of the global temperature

Here's a complete pipeline from getting the data to transforming and plotting it:

  1. Downloading a CSV file with curl;
  2. Skipping the first row with sed;
  3. Grabbing the year column and 12 columns with monthly temperatures to an array and converting to JSON lines format using SPyQL;
  4. Exploding the monthly array with SPyQL (resulting in 12 rows per year) while removing invalid monthly measurements;
  5. Plotting with MatplotCLI .
curl https://data.giss.nasa.gov/gistemp/tabledata_v4/GLB.Ts+dSST.csv |
sed 1d |
spyql "
  SELECT Year, cols[1:13] AS temps
  FROM csv
  TO json" |
spyql "
  SELECT
    json->Year + ((row_number-1)%12)/12 AS year,
    json->temps AS temp
  FROM json
  EXPLODE json->temps
  WHERE json->temps is not Null
  TO json" |
plt "
scatter(year, temp, 2, temp);
xlabel('Year');
ylabel('Temperature anomaly w.r.t. 1951-80 (ºC)');
title('Global surface temperature (land and ocean)')"

You might also like...
These data visualizations were created for my introductory computer science course using Python
These data visualizations were created for my introductory computer science course using Python

Homework 2: Matplotlib and Data Visualization Overview These data visualizations were created for my introductory computer science course using Python

These data visualizations were created as homework for my CS40 class. I hope you enjoy!
These data visualizations were created as homework for my CS40 class. I hope you enjoy!

Data Visualizations These data visualizations were created as homework for my CS40 class. I hope you enjoy! Nobel Laureates by their Country of Birth

Generate visualizations of GitHub user and repository statistics using GitHub Actions.

GitHub Stats Visualization Generate visualizations of GitHub user and repository statistics using GitHub Actions. This project is currently a work-in-

A Python package for caclulations and visualizations in geological sciences.

geo_calcs A Python package for caclulations and visualizations in geological sciences. Free software: MIT license Documentation: https://geo-calcs.rea

Make scripted visualizations in blender
Make scripted visualizations in blender

Scripted visualizations in blender The goal of this project is to script 3D scientific visualizations using blender. To achieve this, we aim to bring

Standardized plots and visualizations in Python
Standardized plots and visualizations in Python

Standardized plots and visualizations in Python pltviz is a Python package for standardized visualization. Routine and novel plotting approaches are f

Generate visualizations of GitHub user and repository statistics using GitHub Actions.

GitHub Stats Visualization Generate visualizations of GitHub user and repository statistics using GitHub Actions. This project is currently a work-in-

Visualizations of some specific solutions of different differential equations.
Visualizations of some specific solutions of different differential equations.

Diff_sims Visualizations of some specific solutions of different differential equations. Heat Equation in 1 Dimension (A very beautiful and elegant ex

Data aggregated from the reports found at the MCPS COVID Dashboard into a set of visualizations.

Montgomery County Public Schools COVID-19 Visualizer Contents About this project Data Support this project About this project Data All data we use can

Comments
  • stats about input data

    stats about input data

    option to print simple statistics about the input data. e.g. for each field

    • number of missing values
    • number of distinct values
    • avg, min, max (if numeric)
    • number of nan, inf (if float)
    • ...
    enhancement good first issue 
    opened by dcmoura 0
Releases(v0.2.0)
Owner
Daniel Moura
Daniel Moura
Fractals plotted on MatPlotLib in Python.

About The Project Learning more about fractals through the process of visualization. Built With Matplotlib Numpy License This project is licensed unde

Akeel Ather Medina 2 Aug 30, 2022
Dipto Chakrabarty 7 Sep 06, 2022
Customizing Visual Styles in Plotly

Customizing Visual Styles in Plotly Code for a workshop originally developed for an Unconference session during the Outlier Conference hosted by Data

Data Design Dimension 9 Aug 03, 2022
visualize_ML is a python package made to visualize some of the steps involved while dealing with a Machine Learning problem

visualize_ML visualize_ML is a python package made to visualize some of the steps involved while dealing with a Machine Learning problem. It is build

Ayush Singh 164 Dec 12, 2022
cqMore is a CadQuery plugin based on CadQuery 2.1.

cqMore (under construction) cqMore is a CadQuery plugin based on CadQuery 2.1. Installation Please use conda to install CadQuery and its dependencies

Justin Lin 36 Dec 21, 2022
Generate a 3D Skyline in STL format and a OpenSCAD file from Gitlab contributions

Your Gitlab's contributions in a 3D Skyline gitlab-skyline is a Python command to generate a skyline figure from Gitlab contributions as Github did at

Félix Gómez 70 Dec 22, 2022
Generate SVG (dark/light) images visualizing (private/public) GitHub repo statistics for profile/website.

Generate daily updated visualizations of GitHub user and repository statistics from the GitHub API using GitHub Actions for any combination of private and public repositories, whether owned or contri

Adam Ross 2 Dec 16, 2022
matplotlib: plotting with Python

Matplotlib is a comprehensive library for creating static, animated, and interactive visualizations in Python. Check out our home page for more inform

Matplotlib Developers 16.7k Jan 08, 2023
With Holoviews, your data visualizes itself.

HoloViews Stop plotting your data - annotate your data and let it visualize itself. HoloViews is an open-source Python library designed to make data a

HoloViz 2.3k Jan 02, 2023
3D plotting and mesh analysis through a streamlined interface for the Visualization Toolkit (VTK)

PyVista Deployment Build Status Metrics Citation License Community 3D plotting and mesh analysis through a streamlined interface for the Visualization

PyVista 1.6k Jan 08, 2023
Using SQLite within Python to create database and analyze Starcraft 2 units data (Pandas also used)

SQLite python Starcraft 2 English This project shows the usage of SQLite with python. To create, modify and communicate with the SQLite database from

1 Dec 30, 2021
Realtime Viewer Mandelbrot set with Python and Taichi (cpu, opengl, cuda, vulkan, metal)

Mandelbrot-set-Realtime-Viewer- Realtime Viewer Mandelbrot set with Python and Taichi (cpu, opengl, cuda, vulkan, metal) Control: "WASD" - movement, "

22 Oct 31, 2022
Create matplotlib visualizations from the command-line

MatplotCLI Create matplotlib visualizations from the command-line MatplotCLI is a simple utility to quickly create plots from the command-line, levera

Daniel Moura 46 Dec 16, 2022
JupyterHub extension for ContainDS Dashboards

ContainDS Dashboards for JupyterHub A Dashboard publishing solution for Data Science teams to share results with decision makers. Run a private on-pre

Ideonate 179 Nov 29, 2022
Moscow DEG 2021 elections plots

Построение графиков на основе публичных данных о ДЭГ в Москве в 2021г. Описание Скрипты в данном репозитории позволяют собственноручно построить графи

9 Jul 15, 2022
Visualization of the World Religion Data dataset by Correlates of War Project.

World Religion Data Visualization Visualization of the World Religion Data dataset by Correlates of War Project. Mostly personal project to famirializ

Emile Bangma 1 Oct 15, 2022
Rockstar - Makes you a Rockstar C++ Programmer in 2 minutes

Rockstar Rockstar is one amazing library, which will make you a Rockstar Programmer in just 2 minutes. In last decade, people learned C++ in 21 days.

4k Jan 05, 2023
Backend app for visualizing CANedge log files in Grafana (directly from local disk or S3)

CANedge Grafana Backend - Visualize CAN/LIN Data in Dashboards This project enables easy dashboard visualization of log files from the CANedge CAN/LIN

13 Dec 15, 2022
Resources for teaching & learning practical data visualization with python.

Practical Data Visualization with Python Overview All views expressed on this site are my own and do not represent the opinions of any entity with whi

Paul Jeffries 98 Sep 24, 2022
1900-2016 Olympic Data Analysis in Python by plotting different graphs

🔥 Olympics Data Analysis 🔥 In Data Science field, there is a big topic before creating a model for future prediction is Data Analysis. We can find o

Sayan Roy 1 Feb 06, 2022