Python package for hypergraph analysis and visualization.

Overview

HyperNetX

The HyperNetX library provides classes and methods for the analysis and visualization of complex network data. HyperNetX uses data structures designed to represent set systems containing nested data and/or multi-way relationships. The library generalizes traditional graph metrics to hypergraphs.

The current version is preliminary. We are actively testing and would be grateful for comments and suggestions. Expect changes in both class names and methods as many of the requirements demanded of the library are worked out.

HypernetX was developed by the Pacific Northwest National Laboratory for the Hypernets project as part of its High Performance Data Analytics (HPDA) program. PNNL is operated by Battelle Memorial Institute under Contract DE-AC05-76RL01830.

  • Principle Developer and Designer: Brenda Praggastis
  • Graphics Developer and Designer: Dustin Arendt
  • Principle Investigator: Emilie Purvine
  • Program Manager: Cliff Joslyn, Mark Raugas
  • Mathematics, methods, and algorithms: Sinan Aksoy, Dustin Arendt, Cliff Joslyn, Brenda Praggastis, and Emilie Purvine
  • Software support: Kyle Monson

For questions and comments you may contact the developers directly at:
[email protected]

Documentation is available at: https://pnnl.github.io/HyperNetX/

Tutorials may be run in your browser using Google Colab

Open In Colab Tutorial 1 - HNX Basics
Open In Colab Tutorial 2 - Visualization Methods
Open In Colab Tutorial 3 - LesMis Case Study
Open In Colab Tutorial 4 - LesMis Visualizations-Book Tour
Open In Colab Tutorial 5 - Homology mod2 for TriLoop Example

Installation Instructions

HyperNetX requires Python>=3.6. We recommend installation in a virtual environment.

To install in an Anaconda environment

>>> conda create -n  python=3.6
>>> source activate  

Mac Users: If you wish to build the documentation you will need the conda version of matplotlib:

>>> conda install matplotlib

To install in a virtualenv environment

>>> virtualenv --python= 

This will create a virtual environment in the specified location using the specified python executable. For example:

>>> virtualenv --python=C:\Anaconda3\python.exe hnx

This will create a virtual environment in .\hnx using the python that comes with Anaconda3.

>>> \Scripts\activate

If you are running in Windows PowerShell use =.ps1

If you are running in Windows Command Prompt use =.bat

Otherwise use =NULL (no file extension).

Once activated continue to follow the installation instructions below.

Install using Pip

For a minimal installation:

>>> pip install hypernetx

For an editable installation with access to jupyter notebooks:

>>> pip install [-e] .

To install with the tutorials:

>>> pip install -e .['tutorials']

To install with the documentation:

>>> pip install -e .['documentation']
>>> sphinx-build -b html docs/source docs/build 
## This will generate the documentation in /docs/build
## Open them in your browser with /docs/build/index.html

To install and test using pytest:

>>> pip install -e .['testing']
>>> pytest

To install the whole shabang:

>>> pip install -e .['all']

License

Released under the 3-Clause BSD license (see License.rst)

Comments
  • Not drawing the hypergraph

    Not drawing the hypergraph

    I tried the programs in the tutorial and also the code of the guy that reported the other issue and i get the following error:

    /usr/local/lib/python3.7/site-packages/hypernetx/drawing/rubber_band.py:97: FutureWarning: arrays to stack must be passed as a "sequence" type such as list or tuple. Support for non-sequence iterables such as generators is deprecated as of NumPy 1.16 and will raise an error in the future.

    I don't know if the problem is that I use python3

    opened by ChrisGeorgakidis 10
  • Started work on support for weighted hypernetworks

    Started work on support for weighted hypernetworks

    Here is a new PR for the weighted hypernetworks code, with all the new commits in a separate branch. Here is the original comment for reference:

    I thought I would try to make it a bit easier to create a weighted hypergraph, using a similar syntax to that in networkx. Now you can add weighted edges like this:

    w = 0.019281
    G = hnx.Hypergraph()
    G.add_edge(("A", "B"), weight=w)
    

    I've also added support for calculating the weighted ~~node~~ adjacency matrices:

    w0 = 0.98279384
    w1 = 0.2309
    G = hnx.Hypergraph()
    e0 = [("A", "B"), ("B", "C")]
    e1 = [("A", "B", "C")]
    G.add_edges_from(e0, weight=w0)
    G.add_edges_from(e1, weight=w1)
    	
    print(G.adjacency_matrix())
    print(G.adjacency_matrix(weight="weight"))
    print(G.edge_adjacency_matrix(weight="weight"))
    

    ~~I have an issue with the edge adjacency matrix, because I would ordinarily calculate it with the following matrix formula: \sqrt(W) * M^T * M * \sqrt(W), but this won't currently work as edge_adjacency_matrix calls __incidence_to_adjacency, the same method as is called by adjacency_matrix. I would be grateful for your thoughts on this, I've currently left the weighted edge adjacency to throw a NotImplementedError.~~

    I have modified the code to do what I commented in the original PR. When the weighted adjacency matrix is requested, M.dot(weight_matrix) is passed into __incidence_to_adjacency so both adjacency matrix functions work.

    All of these changes play quite nicely with the existing code, but I wanted to submit them for your consideration before doing too much more.

    Many thanks. :)

    opened by jim-rafferty 6
  • Started work on support for weighted hypergraphs

    Started work on support for weighted hypergraphs

    Hi,

    I thought I would try to make it a bit easier to create a weighted hypergraph, using a similar syntax to that in networkx. Now you can add weighted edges like this:

    w = 0.019281
    G = hnx.Hypergraph()
    G.add_edge(("A", "B"), weight=w)
    

    I've also added support for calculating the weighted node adjacency matrix:

    w0 = 0.98279384
    w1 = 0.2309
    G = hnx.Hypergraph()
    e0 = [("A", "B"), ("B", "C")]
    e1 = [("A", "B", "C")]
    G.add_edges_from(e0, weight=w0)
    G.add_edges_from(e1, weight=w1)
    	
    print(G.adjacency_matrix())
    print(G.adjacency_matrix(weight="weight"))
    

    I have an issue with the edge adjacency matrix, because I would ordinarily calculate it with the following matrix formula: \sqrt(W) * M^T * M * \sqrt(W), but this won't currently work as edge_adjacency_matrix calls __incidence_to_adjacency, the same method as is called by adjacency_matrix. I would be grateful for your thoughts on this, I've currently left the weighted edge adjacency to throw a NotImplementedError.

    All of these changes play quite nicely with the existing code, but I wanted to submit them for your consideration before doing too much more.

    Many thanks. :)

    opened by jim-rafferty 6
  • Please provide support for weighted hypergraphs

    Please provide support for weighted hypergraphs

    Hi, this is a really good package :)

    It would be loads more useful if you could provide support for weighted hypergraphs. It seems to be possible to create a weighted hypergraph at the moment like this:

    import itertools
    import hypernetx as hnx
    import numpy as np
    
    labels = "ABCD"
    nodes = set(labels)
    edges = list(
        itertools.chain(
            *[list(itertools.combinations(set(labels), ii)) for ii in range(2, len(labels))]
        )
    )
    hnx_edges = hnx.EntitySet(
        "Edges", 
        [hnx.Entity(
                "".join(ee), 
                ee,
                weight=np.random.rand() 
            ) for ee in edges]
    )
    
    h = hnx.Hypergraph(hnx_edges)
    hnx.draw(h)
    print(h.edges.elements)
    

    but it's not possible to do any analysis with the weights as far as I can tell (see for example, the output of hnx.dist_stats(h) )

    Are there any plans to add this type of functionality?

    Thanks!

    opened by jim-rafferty 6
  • Import fails due to missing `decorator` package in clean install

    Import fails due to missing `decorator` package in clean install

    Installing hypernetx in a clean virtual environment fails due to a missing package: decorator.

    Step to reproduce (after initializing a new empty virtual environment):

    pip install hypernetx
    

    giving an error like this:

    >>> import hypernetx as hnx
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/tmp/hnx-version/venv/lib/python3.8/site-packages/hypernetx/__init__.py", line 7, in <module>
        from hypernetx.classes import *
      File "/tmp/hnx-version/venv/lib/python3.8/site-packages/hypernetx/classes/__init__.py", line 2, in <module>
        from .hypergraph import Hypergraph
      File "/tmp/hnx-version/venv/lib/python3.8/site-packages/hypernetx/classes/hypergraph.py", line 13, in <module>
        from hypernetx.classes.staticentity import StaticEntity, StaticEntitySet
      File "/tmp/hnx-version/venv/lib/python3.8/site-packages/hypernetx/classes/staticentity.py", line 10, in <module>
        from hypernetx.utils import (
      File "/tmp/hnx-version/venv/lib/python3.8/site-packages/hypernetx/utils/__init__.py", line 8, in <module>
        from .decorators import not_implemented_for
      File "/tmp/hnx-version/venv/lib/python3.8/site-packages/hypernetx/utils/decorators.py", line 4, in <module>
        from decorator import decorator
    ModuleNotFoundError: No module named 'decorator'
    

    Of course, installing the decorator package manually fixes the issue and hypernetx imports fine. I think the decorator package should just be added to the install_requires entry in setup.cfg and setup.py since this package is required to even load the package at all.

    Tested using Python versions 3.8 and 3.9, using hypernetx 1.2.2 (on Ubuntu).

    opened by tmthyln 3
  • Dependencies are not correct

    Dependencies are not correct

    For pip install hypernetx, the packages igraph and celluloid are not installed by default. One has to do it manually. This is also the reason why the tutorial fails (#73). What about adding them to setup.py?

    opened by michaeldorner 3
  • Find all hyperedges for a node

    Find all hyperedges for a node

    Hi, Maybe I am just blind (and/or stupid), but how can I identify (easily) all hyperedges of a given node n in a (static) hypergraph H in HyperNetX?

    The solution I came up is:

    [edge for edge, nodes in H.incidence_dict.items() if n in nodes]

    But probably you had another, more performant solution in mind? Thank you!

    opened by michaeldorner 2
  • Import error on module hypernetx.utils.toys

    Import error on module hypernetx.utils.toys

    Hello, I have the following error when I try to import hypernetx on Colab (with your tutorials) and also on my conda env:

    ModuleNotFoundError                       Traceback (most recent call last)
    <ipython-input-3-b7c66bb90caa> in <module>()
          3 import matplotlib.pyplot as plt
          4 import networkx as nx
    ----> 5 import hypernetx as hnx
          6 
    
    4 frames
    /usr/local/lib/python3.7/dist-packages/hypernetx/__init__.py in <module>()
          5 )
          6 from hypernetx.read_write import to_pickle, load_from_pickle
    ----> 7 from hypernetx.classes import *
          8 from hypernetx.reports import *
          9 from hypernetx.drawing import *
    
    /usr/local/lib/python3.7/dist-packages/hypernetx/classes/__init__.py in <module>()
          1 from .entity import Entity, EntitySet
    ----> 2 from .hypergraph import Hypergraph
          3 from .staticentity import StaticEntity, StaticEntitySet
    
    /usr/local/lib/python3.7/dist-packages/hypernetx/classes/hypergraph.py in <module>()
         11 from collections import OrderedDict, defaultdict
         12 from hypernetx.classes.entity import Entity, EntitySet
    ---> 13 from hypernetx.classes.staticentity import StaticEntity, StaticEntitySet
         14 from hypernetx.exception import HyperNetXError
         15 from hypernetx.utils.decorators import not_implemented_for
    
    /usr/local/lib/python3.7/dist-packages/hypernetx/classes/staticentity.py in <module>()
          8 from hypernetx.exception import HyperNetXError
          9 from hypernetx.classes.entity import Entity, EntitySet
    ---> 10 from hypernetx.utils import HNXCount, DefaultOrderedDict, remove_row_duplicates
         11 from scipy.sparse import coo_matrix, csr_matrix, issparse
         12 import itertools as it
    
    /usr/local/lib/python3.7/dist-packages/hypernetx/utils/__init__.py in <module>()
          2 from .decorators import not_implemented_for
          3 # from .toys import HarryPotter, LesMis, lesmis_hypergraph_from_df, book_tour, TransmissionProblem
    ----> 4 from .toys import *
    
    ModuleNotFoundError: No module named 'hypernetx.utils.toys'
    

    I'm doing something wrong?

    opened by RemiG3 2
  • add_node_to_edge does not update the node's membership

    add_node_to_edge does not update the node's membership

    When adding an existing node to an existing edge, the memberships property of the node stored in _nodes is not updated

    Code to reproduce the issue:

    system = hnx.Hypergraph()
    edge0 = hnx.Entity('edge0')
    edge1 = hnx.Entity('edge1')
    node0 = hnx.Entity('node0')
    node1 = hnx.Entity('node1')
    
    system.add_edge(edge0)
    system.add_edge(edge1)
    system.add_node_to_edge(node0,edge0)
    system.add_node_to_edge(node0,edge1)
    system.add_node_to_edge(node1,edge1)
    
    #Memberships seen from the nodes stored within _edges
    print("Memberships seen from the nodes stored within _edges")
    for el in system.edges:
        for edge in system.edges[el]:
            print(edge,system.edges[el][edge].memberships)
            
    #Memberships seen from the nodes stored within _nodes
    print("Memberships seen from the nodes stored within _nodes")
    for el in system.nodes:
        print(el,system.nodes[el].memberships)
    
    hnx.drawing.rubber_band.draw(system)
    
    opened by dncolomer 2
  • Overlapping node labels

    Overlapping node labels

    When drawing hypergraphs using the rubber band layout, node labels often overlap. This is especially prevalent when using longer strings as node labels; however, this problem also often occurs with very short labels as well. Due to the overlap, many labels are unreadable. Are there any settings within HyperNetX or Matplotlib to correct this? If not, are there any known workarounds? Thanks!

    opened by epideveloper 2
  • Can I adjust how close hyperedges can be drawn in proximity to nodes?

    Can I adjust how close hyperedges can be drawn in proximity to nodes?

    I made the following diagram using Hypernetworkx's draw command.

    image

    The node fst_score node appears really close to the 233 hyperedge, making it hard to distinguish whether the node is a member of the hyperedge. Is there a setting an argument somewhere that would allow me to ensure that the hyperedges maintain a minimal distance from the nodes?

    opened by galenseilis 1
  • Same edge name ?

    Same edge name ?

    Is it possible for edges with the same name but different nodes to coexist. Something like this :

    {
      "is" : ["red","apple","rotten"],
      "is" : ["black","pen"]
    }
    

    How do you achieve this ?

    opened by vsraptor 1
  • Comparison between weighted and unweighted hypergraphs in clustering

    Comparison between weighted and unweighted hypergraphs in clustering

    Hello, thank you for sharing this excellent library.

    I want to compare the performance of clustering on weighted and unweighted hypergraphs as described in K. Hayashi, S. Aksoy, C. Park, H. Park, "Hypergraph random walks, Laplacians, and clustering". I tried to do this based on Tutorial 11. However, I think some modifications need. h = hnx.Hypergraph(hnx.StaticEntitySet(data=data, weights=w)) I think this code does not make a weighted hypergraph. Instead, this should be revised as below. h = hnx.Hypergraph(hnx.StaticEntitySet(data=data, weights=w), weights=w)

    Additionally, I cannot get the satisfying result that weighted hypergraphs perform better than unweighted ones. Thus, I want to ask whether the below code is a correct way to make clustering on an unweighted hypergraph and evaluate clustering algorithms by NMI scores.

    # get data
    categories = all_categories[[1,15]]
    twenty_train = fetch_20newsgroups(subset='test', categories=categories, shuffle=True, random_state=42)
    doc_types=dict()
    for i,x in enumerate(twenty_train.filenames):
        doc_types[i]=x.split('/')[-2]
    tfidf_vect = TfidfVectorizer()
    X_tfidf = tfidf_vect.fit_transform(twenty_train.data)
    
    # construct hypergraph
    mat = coo_matrix(X_tfidf)
    edges = mat.col
    nodes = mat.row
    data = np.array([edges,nodes]).T
    weights = mat.data
    # clustering on weighted hypergraph
    h = hnx.Hypergraph(hnx.StaticEntitySet(data=data,weights=weights),weights=weights)
    clusters=hnx.spec_clus(h,num_clus,weights=True)
    # clustering on unweighted hypergraph
    uwh =  hnx.Hypergraph(hnx.StaticEntitySet(data=data,weights=None))
    uw_clusters = hnx.spec_clus(uwh,num_clus,weights=False)
    
    # construct clustering label list
    categoryindexing = {}
    _labels = {} # answer
    for i in range(X_tfidf.shape[0]):
        if doc_types[i] not in categoryindexing:
            categoryindexing[doc_types[i]] = len(categoryindexing)
        ci = categoryindexing[doc_types[i]]
        _labels[i] = ci
    labels = [_labels[i] for i in range(X_tfidf.shape[0])]
    _w_pred = {} #  labels from weighted hypergraph
    for i in clusters:
        for v in clusters[i]:
            _w_pred[v] = i
    w_pred = [_w_pred[i] for i in range(X_tfidf.shape[0])]
    _uw_pred = {} # labels from unweighted hypergraph
    for i in uw_clusters:
        for v in uw_clusters[i]:
            _uw_pred[v] = i
    uw_pred = [_uw_pred[i] for i in range(X_tfidf.shape[0])]
    
    # evaluation on NMI score
    from sklearn.metrics.cluster import normalized_mutual_info_score as NMI_SCORE
    score = NMI_SCORE(labels, w_pred)
    print(score) # 0.73
    score = NMI_SCORE(labels, uw_pred)
    print(score) # 0.78
    
    opened by young917 3
  • Modularity function returns error: AttributeError: 'Entity' object has no attribute 'strength'

    Modularity function returns error: AttributeError: 'Entity' object has no attribute 'strength'

    When I call print('qH =',hmod.modularity(H, K, strict)) it returns AttributeError: 'Entity' object has no attribute 'strength'

    The full code:

    import igraph as ig
    import hypernetx as hnx
    import hypernetx.algorithms.hypergraph_modularity as hmod
    
    H = hnx.Hypergraph(lookup)
    H = hmod.precompute_attributes(H)
    #%%
    K = hmod.kumar(H)
    #%%
    strict = hmod.strict
    ## Compute qH
    print('qH =',hmod.modularity(H, K, strict))
    

    Lookup is a dictionary of the form: {key1 : (hedge1, hedge2...), key2 : (hedge1, hedge3...)}

    opened by thosvarley 3
  • Nodes in a Hypergarph via H.nodes

    Nodes in a Hypergarph via H.nodes

    I tried to generate a hypergraph using node neighborhood information from a matrix K of size |N| x |N|. Using the simple graph edges list, if the length (distance) of two given nodes is less than or equal to k, then I put 1 in the K matrix, 0 otherwise. 1 denotes that the two given nodes are neighbors, and 0 denotes that nodes are not neighbors. After generating the K matrix, I used it to represent hyperedges where each row of K represents the neighborhood of a particular node. (a hyperedge).

    The problem is, after generating a hypergraph, it shows the correct number of hyperedges using the H.edges method, however, it gives the wrong number of nodes using the H.nodes method. The total nodes are very less than the total nodes given by the H.nodes method,

    Also, I tried the example graph given in the documentation of the library. It has 13 nodes and 8 hyperedges. H.nodes and H.edges methods show the correct number of nodes and edges for this graph.

    Can't I use the defined K matrix for hypergraph generation?

    opened by khizer-hayat 1
Releases(v1.2.5)
Owner
Pacific Northwest National Laboratory
Pacific Northwest National Laboratory
A research of IT labor market based especially on hh.ru. Salaries, rate of technologies and etc.

hh_ru_research Проект реализован в учебных целях анализа рынка труда, в особенности по hh.ru Input data В качестве входных данных используются сериали

3 Sep 07, 2022
Visualization of the World Religion Data dataset by Correlates of War Project.

World Religion Data Visualization Visualization of the World Religion Data dataset by Correlates of War Project. Mostly personal project to famirializ

Emile Bangma 1 Oct 15, 2022
Curvipy - The Python package for visualizing curves and linear transformations in a super simple way

Curvipy - The Python package for visualizing curves and linear transformations in a super simple way

Dylan Tintenfich 55 Dec 28, 2022
Visualization of numerical optimization algorithms

Visualization of numerical optimization algorithms

Zhengxia Zou 46 Dec 01, 2022
Standardized plots and visualizations in Python

Standardized plots and visualizations in Python pltviz is a Python package for standardized visualization. Routine and novel plotting approaches are f

Andrew Tavis McAllister 0 Jul 09, 2022
Altair extension for saving charts in a variety of formats.

Altair Saver This packge provides extensions to Altair for saving charts to a variety of output types. Supported output formats are: .json/.vl.json: V

Altair 85 Dec 09, 2022
Typical: Fast, simple, & correct data-validation using Python 3 typing.

typical: Python's Typing Toolkit Introduction Typical is a library devoted to runtime analysis, inference, validation, and enforcement of Python types

Sean 171 Jan 02, 2023
Streaming pivot visualization via WebAssembly

Perspective is an interactive visualization component for large, real-time datasets. Originally developed for J.P. Morgan's trading business, Perspect

The Fintech Open Source Foundation (www.finos.org) 5.1k Dec 27, 2022
Designed a greedy algorithm based on Markov sequential decision-making process in MATLAB/Python to optimize using Gurobi solver

Designed a greedy algorithm based on Markov sequential decision-making process in MATLAB/Python to optimize using Gurobi solver, the wheel size, gear shifting sequence by modeling drivetrain constrai

Sabbella Prasanna 1 Jan 11, 2022
Using SQLite within Python to create database and analyze Starcraft 2 units data (Pandas also used)

SQLite python Starcraft 2 English This project shows the usage of SQLite with python. To create, modify and communicate with the SQLite database from

1 Dec 30, 2021
A library for bridging Python and HTML/Javascript (via Svelte) for creating interactive visualizations

A library for bridging Python and HTML/Javascript (via Svelte) for creating interactive visualizations

Anthropic 98 Dec 27, 2022
Graphical visualizer for spectralyze by Lauchmelder23

spectralyze visualizer Graphical visualizer for spectralyze by Lauchmelder23 Install Install matplotlib and ffmpeg. Put ffmpeg.exe in same folder as v

Matthew 1 Dec 21, 2021
An easy to use burndown chart generator for GitHub Project Boards.

Burndown Chart for GitHub Projects An easy to use burndown chart generator for GitHub Project Boards. Table of Contents Features Installation Assumpti

Joseph Hale 15 Dec 28, 2022
Collection of scripts for making high quality beautiful math-related posters.

Poster Collection of scripts for making high quality beautiful math-related posters. The poster can have as large printing size as 3x2 square feet wit

Nattawut Phetmak 3 Jun 09, 2022
Simple plotting for Python. Python wrapper for D3xter - render charts in the browser with simple Python syntax.

PyDexter Simple plotting for Python. Python wrapper for D3xter - render charts in the browser with simple Python syntax. Setup $ pip install PyDexter

D3xter 31 Mar 06, 2021
The Python ensemble sampling toolkit for affine-invariant MCMC

emcee The Python ensemble sampling toolkit for affine-invariant MCMC emcee is a stable, well tested Python implementation of the affine-invariant ense

Dan Foreman-Mackey 1.3k Jan 04, 2023
YOPO is an interactive dashboard which generates various standard plots.

YOPO is an interactive dashboard which generates various standard plots.you can create various graphs and charts with a click of a button. This tool uses Dash and Flask in backend.

ADARSH C 38 Dec 20, 2022
The visual framework is designed on the idea of module and implemented by mixin method

Visual Framework The visual framework is designed on the idea of module and implemented by mixin method. Its biggest feature is the mixins module whic

LEFTeyes 9 Sep 19, 2022
A python visualization of the A* path finding algorithm

A python visualization of the A* path finding algorithm. It allows you to pick your start, end location and make obstacles and then view the process of finding the shortest path. You can also choose

Kimeon 4 Aug 02, 2022
Python Data Validation for Humans™.

validators Python data validation for Humans. Python has all kinds of data validation tools, but every one of them seems to require defining a schema

Konsta Vesterinen 670 Jan 09, 2023