Approximating functions, functionals, and operators using deep neural networks for diverse applications

Presenter: George Em Karniadakis (GS h-index 105)
The Charles Pitts Robinson and John Palmer Barstow Professor of Applied Mathematics and Engineering, Brown University; Also @MIT & PNNL

Abstract: We will present a new approach to develop a data-driven, learning-based framework for predicting outcomes of physical and biophysical systems and for discovering hidden physics from noisy data. We will introduce a deep learning approach based on neural networks (NNs) and generative adversarial networks (GANs). Unlike other approaches that rely on big data, here we “learn” from small data by exploiting the information provided by the physical conservation laws, which are used to obtain informative priors or regularize the neural networks. We will also make connections between Gauss Process Regression and NNs, and discuss the new powerful concept of meta-learning. We will demonstrate the power of PINNs for several inverse problems, and we will demonstrate how we can use multi-fidelity modeling in monitoring ocean acidification levels in the Massachusetts Bay. We will also introduce new NNs that learn functionals and nonlinear operators from functions and corresponding responses for system identification. The universal approximation theorem of operators is suggestive of the potential of NNs in learning from scattered data any continuous operator or complex system. We first generalize the theorem to deep neural networks, and subsequently we apply it to design a new composite NN with small generalization error, the deep operator network (DeepONet), consisting of a NN for encoding the discrete input function space (branch net) and another NN for encoding the domain of the output functions (trunk net). We demonstrate that DeepONet can learn various explicit operators, e.g., integrals, Laplace transforms and fractional Laplacians, as well as implicit operators that represent deterministic and stochastic differential equations. More generally, DeepOnet can learn multiscale operators spanning across many scales and trained by diverse sources of data simultaneously. There are many versions of PINNs, e.g., variational (VPINNs), stochastic (sPINNs), conservative (cPINNs), nonlocal (nPINNs), generalized (xPINNs), etc, and we will provide some highlights. In addition, we will present our recent theoretical results on the convergence and generalization of PINNs.