Efficient and Scalable Physics-Informed Deep Learning and Scientific Machine Learning on top of Tensorflow for multi-worker distributed computing

Overview

TensorDiffEq logo

Package Build Package Release pypi downloads python versions

Notice: Support for Python 3.6 will be dropped in v.0.2.1, please plan accordingly!

Efficient and Scalable Physics-Informed Deep Learning

Collocation-based PINN PDE solvers for prediction and discovery methods on top of Tensorflow 2.X for multi-worker distributed computing.

Use TensorDiffEq if you require:

  • A meshless PINN solver that can distribute over multiple workers (GPUs) for forward problems (inference) and inverse problems (discovery)
  • Scalable domains - Iterated solver construction allows for N-D spatio-temporal support
    • support for N-D spatial domains with no time element is included
  • Self-Adaptive Collocation methods for forward and inverse PINNs
  • Intuitive user interface allowing for explicit definitions of variable domains, boundary conditions, initial conditions, and strong-form PDEs

What makes TensorDiffEq different?

  • Completely open-source

  • Self-Adaptive Solvers for forward and inverse problems, leading to increased accuracy of the solution and stability in training, resulting in less overall training time

  • Multi-GPU distributed training for large or fine-grain spatio-temporal domains

  • Built on top of Tensorflow 2.0 for increased support in new functionality exclusive to recent TF releases, such as XLA support, autograph for efficent graph-building, and grappler support for graph optimization* - with no chance of the source code being sunset in a further Tensorflow version release

  • Intuitive interface - defining domains, BCs, ICs, and strong-form PDEs in "plain english"

*In development

If you use TensorDiffEq in your work, please cite it via:

@article{mcclenny2021tensordiffeq,
  title={TensorDiffEq: Scalable Multi-GPU Forward and Inverse Solvers for Physics Informed Neural Networks},
  author={McClenny, Levi D and Haile, Mulugeta A and Braga-Neto, Ulisses M},
  journal={arXiv preprint arXiv:2103.16034},
  year={2021}
}

Thanks to our additional contributors:

@marcelodallaqua, @ragusa, @emiliocoutinho

Comments
  • Latest version of package

    Latest version of package

    The examples in the doc use the latest code of master branch but the library on Pypi is still the version in May. Can you build the lib and update the version on Pypi?

    opened by devzhk 5
  • ADAM training on batches

    ADAM training on batches

    It is possible to define a batch size and this will be applied to the calculation of the residual loss function, in splitting the collocation points in batches during the training.

    opened by emiliocoutinho 3
  • Pull Request using PyCharm

    Pull Request using PyCharm

    Dear Levi,

    I tried to make a Pull Request on this repository using PyCharm, and I received the following message:

    Although you appear to have the correct authorization credentials, the tensordiffeq organization has enabled OAuth App access restrictions, meaning that data access to third-parties is limited. For more information on these restrictions, including how to whitelist this app, visit https://help.github.com/articles/restricting-access-to-your-organization-s-data/

    I would kindly ask you to authorize PyCharm to access your organization data to use the GUI to make future pull requests.

    Best Regards

    opened by emiliocoutinho 1
  • Update method def get_sizes of utils.py

    Update method def get_sizes of utils.py

    Fix bug on the method def get_sizes(layer_sizes) of utils.py. The method was only allowing neural nets with an identical number of nodes in each hidden layer. Which was making the L- BFGS optimization to crash.

    opened by marcelodallaqua 1
  • model.save ?

    model.save ?

    Sometimes, it's useful to save the model for later use. I couldn't find a .save method and pickle (and dill) didn't let me dump the object for later re-use. (example of error with pickle: Can't pickle local object 'make_gradient_clipnorm_fn..').

    Is it currently possible to save the model? Thanks!

    opened by ragusa 1
  • add model.save and model.load_model

    add model.save and model.load_model

    Add model.save and model.load_model to CollocationSolverND class ref #3

    Will be released in the next stable.

    currently this can be done by using the Keras integration via running model.u_model.save("path/to/file"). This change will allow a direct save by calling model.save() on the CollocationSolverND class. Same with load_model().

    The docs will be updated to reflect this change.

    opened by levimcclenny 0
  • 2D Burgers Equation

    2D Burgers Equation

    Hello @levimcclenny and thanks for recommending this library!

    I have modified the 1D burger example to be in 2D, but I did not get good comparison results. Any suggestions?

    import math
    import scipy.io
    import tensordiffeq as tdq
    from tensordiffeq.boundaries import *
    from tensordiffeq.models import CollocationSolverND
    
    Domain = DomainND(["x", "y", "t"], time_var='t')
    
    Domain.add("x", [-1.0, 1.0], 256)
    Domain.add("y", [-1.0, 1.0], 256)
    Domain.add("t", [0.0, 1.0], 100)
    
    N_f = 10000
    Domain.generate_collocation_points(N_f)
    
    
    def func_ic(x,y):
        p =2
        q =1
        return np.sin (p * math.pi * x) * np.sin(q * math.pi * y)
        
    
    init = IC(Domain, [func_ic], var=[['x','y']])
    upper_x = dirichletBC(Domain, val=0.0, var='x', target="upper")
    lower_x = dirichletBC(Domain, val=0.0, var='x', target="lower")
    upper_y = dirichletBC(Domain, val=0.0, var='y', target="upper")
    lower_y = dirichletBC(Domain, val=0.0, var='y', target="lower")
    
    BCs = [init, upper_x, lower_x, upper_y, lower_y]
    
    
    def f_model(u_model, x, y, t):
        u = u_model(tf.concat([x, y, t], 1))
        u_x = tf.gradients(u, x)
        u_xx = tf.gradients(u_x, x)
        u_y = tf.gradients(u, y)
        u_yy = tf.gradients(u_y, y)
        u_t = tf.gradients(u, t)
        f_u = u_t + u * (u_x + u_y) - (0.01 / tf.constant(math.pi)) * (u_xx+u_yy)
        return f_u
    
    
    layer_sizes = [3, 20, 20, 20, 20, 20, 20, 20, 20, 1]
    
    model = CollocationSolverND()
    model.compile(layer_sizes, f_model, Domain, BCs)
    
    # to reproduce results from Raissi and the SA-PINNs paper, train for 10k newton and 10k adam
    model.fit(tf_iter=10000, newton_iter=10000)
    
    model.save("burger2D_Training_Model")
    #model.load("burger2D_Training_Model")
    
    #######################################################
    #################### PLOTTING #########################
    #######################################################
    
    data = np.load('py-pde_2D_burger_data.npz')
    
    Exact = data['u_output']
    Exact_u = np.real(Exact)
    
    x = Domain.domaindict[0]['xlinspace']
    y = Domain.domaindict[1]['ylinspace']
    t = Domain.domaindict[2]["tlinspace"]
    
    X, Y, T = np.meshgrid(x, y, t)
    
    X_star = np.hstack((X.flatten()[:, None], Y.flatten()[:, None], T.flatten()[:, None]))
    u_star = Exact_u.T.flatten()[:, None]
    
    u_pred, f_u_pred = model.predict(X_star)
    
    error_u = tdq.helpers.find_L2_error(u_pred, u_star)
    print('Error u: %e' % (error_u))
    
    lb = np.array([-1.0, -1.0, 0.0])
    ub = np.array([1.0, 1.0, 1])
    
    tdq.plotting.plot_solution_domain2D(model, [x, y, t], ub=ub, lb=lb, Exact_u=Exact_u.T)
    
    
    Screen Shot 2022-03-04 at 11 15 31 PM Screen Shot 2022-03-04 at 11 15 44 PM Screen Shot 2022-03-04 at 11 15 18 PM
    opened by engsbk 3
  • 2D Wave Equation

    2D Wave Equation

    Thank you for the great contribution!

    I'm trying to extend the 1D example problems to 2D, but I want to make sure my changes are in the correct place:

    1. Dimension variables. I changed them like so:

    Domain = DomainND(["x", "y", "t"], time_var='t')

    Domain.add("x", [0.0, 5.0], 100) Domain.add("y", [0.0, 5.0], 100) Domain.add("t", [0.0, 5.0], 100)

    1. My IC is zero, but for the BCs I'm not sure how to define the left and right borders, please let me know if my implementation is correct:
    
    def func_ic(x,y):
        return 0
    
    init = IC(Domain, [func_ic], var=[['x','y']])
    upper_x = dirichletBC(Domain, val=0.0, var='x', target="upper")
    lower_x = dirichletBC(Domain, val=0.0, var='x', target="lower")
    upper_y = dirichletBC(Domain, val=0.0, var='y', target="upper")
    lower_y = dirichletBC(Domain, val=0.0, var='y', target="lower")
            
    BCs = [init, upper_x, lower_x, upper_y, lower_y]
    

    All of my BCs and ICs are zero. And my equation has a (forcing) time-dependent source term as such:

    
    def f_model(u_model, x, y, t):
        c = tf.constant(1, dtype = tf.float32)
        Amp = tf.constant(2, dtype = tf.float32)
        freq = tf.constant(1, dtype = tf.float32)
        sigma = tf.constant(0.2, dtype = tf.float32)
    
        source_x = tf.constant(0.5, dtype = tf.float32)
        source_y = tf.constant(2.5, dtype = tf.float32)
    
        GP = Amp * tf.exp(-0.5*( ((x-source_x)/sigma)**2 + ((y-source_y)/sigma)**2 ))
        
        S = GP * tf.sin( 2 * tf.constant(math.pi)  * freq * t )
        u = u_model(tf.concat([x,y,t], 1))
        u_x = tf.gradients(u,x)
        u_xx = tf.gradients(u_x, x)
        u_y = tf.gradients(u,y)
        u_yy = tf.gradients(u_y, y)
        u_t = tf.gradients(u,t)
        u_tt = tf.gradients(u_t,t)
    
    
        f_u = u_xx + u_yy - (1/c**2) * u_tt + S
        
        return f_u
    

    Please advise.

    Looking forward to your reply!

    opened by engsbk 13
  • Reproducibility

    Reproducibility

    Dear @levimcclenny,

    Have you considered in adapt TensorDiffEq to be deterministic? In the way the code is implemented, we can find two sources of randomness:

    • The function Domain.generate_collocation_points has a random number generation
    • The TensorFlow training procedure (weights initialization and possibility of the use o random batches)

    Both sources of randomness can be solved with not much effort. We can define a random state for the first one that can be passed to the function Domain.generate_collocation_points. For the second, we can use the implementation provided on Framework Determinism. I have used the procedures suggested by this code, and the results of TensorFlow are always reproducible (CPU or GPU, serial or distributed).

    If you want, I can implement these two features.

    Best Regards

    opened by emiliocoutinho 3
Releases(v0.2.0)
Owner
tensordiffeq
Scalable PINN solvers for PDE Inference and Discovery
tensordiffeq
Namish Khanna 40 Oct 11, 2022
FluxTraining.jl gives you an endlessly extensible training loop for deep learning

A flexible neural net training library inspired by fast.ai

86 Dec 31, 2022
NeuroLKH: Combining Deep Learning Model with Lin-Kernighan-Helsgaun Heuristic for Solving the Traveling Salesman Problem

NeuroLKH: Combining Deep Learning Model with Lin-Kernighan-Helsgaun Heuristic for Solving the Traveling Salesman Problem Liang Xin, Wen Song, Zhiguang

xinliangedu 33 Dec 27, 2022
SelfAugment extends MoCo to include automatic unsupervised augmentation selection.

SelfAugment extends MoCo to include automatic unsupervised augmentation selection. In addition, we've included the ability to pretrain on several new datasets and included a wandb integration.

Colorado Reed 24 Oct 26, 2022
PyTorch implementation of Pay Attention to MLPs

gMLP PyTorch implementation of Pay Attention to MLPs. Quickstart Clone this repository. git clone https://github.com/jaketae/g-mlp.git Navigate to th

Jake Tae 34 Dec 13, 2022
hySLAM is a hybrid SLAM/SfM system designed for mapping

HySLAM Overview hySLAM is a hybrid SLAM/SfM system designed for mapping. The system is based on ORB-SLAM2 with some modifications and refactoring. Raú

Brian Hopkinson 15 Oct 10, 2022
Implementation of CVAE. Trained CVAE on faces from UTKFace Dataset to produce synthetic faces with a given degree of happiness/smileyness.

Conditional Smiles! (SmileCVAE) About Implementation of AE, VAE and CVAE. Trained CVAE on faces from UTKFace Dataset. Using an encoding of the Smile-s

Raúl Ortega 3 Jan 09, 2022
Official Implementation of "LUNAR: Unifying Local Outlier Detection Methods via Graph Neural Networks"

LUNAR Official Implementation of "LUNAR: Unifying Local Outlier Detection Methods via Graph Neural Networks" Adam Goodge, Bryan Hooi, Ng See Kiong and

Adam Goodge 25 Dec 28, 2022
Large Scale Multi-Illuminant (LSMI) Dataset for Developing White Balance Algorithm under Mixed Illumination

Large Scale Multi-Illuminant (LSMI) Dataset for Developing White Balance Algorithm under Mixed Illumination (ICCV 2021) Dataset License This work is l

DongYoung Kim 33 Jan 04, 2023
Test-Time Personalization with a Transformer for Human Pose Estimation, NeurIPS 2021

Transforming Self-Supervision in Test Time for Personalizing Human Pose Estimation This is an official implementation of the NeurIPS 2021 paper: Trans

41 Nov 28, 2022
A Flow-based Generative Network for Speech Synthesis

WaveGlow: a Flow-based Generative Network for Speech Synthesis Ryan Prenger, Rafael Valle, and Bryan Catanzaro In our recent paper, we propose WaveGlo

NVIDIA Corporation 2k Dec 26, 2022
Repository for RNNs using TensorFlow and Keras - LSTM and GRU Implementation from Scratch - Simple Classification and Regression Problem using RNNs

RNN 01- RNN_Classification Simple RNN training for classification task of 3 signal: Sine, Square, Triangle. 02- RNN_Regression Simple RNN training for

Nahid Ebrahimian 13 Dec 13, 2022
rliable is an open-source Python library for reliable evaluation, even with a handful of runs, on reinforcement learning and machine learnings benchmarks.

Open-source library for reliable evaluation on reinforcement learning and machine learning benchmarks. See NeurIPS 2021 oral for details.

Google Research 529 Jan 01, 2023
Relative Human dataset, CVPR 2022

Relative Human (RH) contains multi-person in-the-wild RGB images with rich human annotations, including: Depth layers (DLs): relative depth relationsh

Yu Sun 112 Dec 02, 2022
Change Detection in SAR Images Based on Multiscale Capsule Network

SAR_CD_MS_CapsNet Code for the paper "Change Detection in SAR Images Based on Multiscale Capsule Network" , IEEE Geoscience and Remote Sensing Letters

Feng Gao 21 Nov 29, 2022
This repository is the official implementation of Open Rule Induction. This paper has been accepted to NeurIPS 2021.

Open Rule Induction This repository is the official implementation of Open Rule Induction. This paper has been accepted to NeurIPS 2021. Abstract Rule

Xingran Chen 16 Nov 14, 2022
A numpy-based implementation of RANSAC for fundamental matrix and homography estimation. The degeneracy updating and local optimization components are included and optional.

Description A numpy-based implementation of RANSAC for fundamental matrix and homography estimation. The degeneracy updating and local optimization co

AoxiangFan 9 Nov 10, 2022
Official Pytorch implementation of "Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Video", CVPR 2021

TCMR: Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Video Qualtitative result Paper teaser video Introduction This r

Hongsuk Choi 215 Jan 06, 2023
A Pytorch implementation of SMU: SMOOTH ACTIVATION FUNCTION FOR DEEP NETWORKS USING SMOOTHING MAXIMUM TECHNIQUE

SMU_pytorch A Pytorch Implementation of SMU: SMOOTH ACTIVATION FUNCTION FOR DEEP NETWORKS USING SMOOTHING MAXIMUM TECHNIQUE arXiv https://arxiv.org/ab

Fuhang 36 Dec 24, 2022
PyTorch Implementation of Daft-Exprt: Robust Prosody Transfer Across Speakers for Expressive Speech Synthesis

PyTorch Implementation of Daft-Exprt: Robust Prosody Transfer Across Speakers for Expressive Speech Synthesis

Ubisoft 76 Dec 30, 2022