JOURNAL OF COMPUTATIONAL MATHEMATICS

Browse issues
Improving Weak PINNs for Hyperbolic Conservation Laws: Dual Norm Computation, Boundary Conditions and Systems
Aidan Chaumet1; Jan Giesselmann1
1 Department of Mathematics, Technical University of Darmstadt, Germany
The SMAI Journal of computational mathematics, Volume 10 (2024), pp. 373-401.

We consider the approximation of entropy solutions of nonlinear hyperbolic conservation laws using neural networks. We provide explicit computations that highlight why classical PINNs will not work for discontinuous solutions to nonlinear hyperbolic conservation laws and show that weak (dual) norms of the PDE residual should be used in the loss functional. This approach has been termed “weak PINNs” recently. We suggest some modifications to weak PINNs that make their training easier, which leads to smaller errors with less training, as shown by numerical experiments. Additionally, we extend wPINNs to scalar conservation laws with weak boundary data and to systems of hyperbolic conservation laws. We perform numerical experiments in order to assess the accuracy and efficiency of the extended method.

Published online:
DOI: 10.5802/smai-jcm.116
Classification: 35L65, 65M99
Keywords: physics-informed learning, PINNs, hyperbolic conservation laws, entropy solution

Aidan Chaumet 1; Jan Giesselmann 1

1 Department of Mathematics, Technical University of Darmstadt, Germany
License: CC-BY 4.0
Copyrights: The authors retain unrestricted copyrights and publishing rights 
@article{SMAI-JCM_2024__10__373_0,
     author = {Aidan Chaumet and Jan Giesselmann},
     title = {Improving {Weak} {PINNs} for {Hyperbolic} {Conservation} {Laws:} {Dual} {Norm} {Computation,} {Boundary} {Conditions} and {Systems}},
     journal = {The SMAI Journal of computational mathematics},
     pages = {373--401},
     publisher = {Soci\'et\'e de Math\'ematiques Appliqu\'ees et Industrielles},
     volume = {10},
     year = {2024},
     doi = {10.5802/smai-jcm.116},
     language = {en},
     url = {https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.116/}
}
TY  - JOUR
AU  - Aidan Chaumet
AU  - Jan Giesselmann
TI  - Improving Weak PINNs for Hyperbolic Conservation Laws: Dual Norm Computation, Boundary Conditions and Systems
JO  - The SMAI Journal of computational mathematics
PY  - 2024
SP  - 373
EP  - 401
VL  - 10
PB  - Société de Mathématiques Appliquées et Industrielles
UR  - https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.116/
DO  - 10.5802/smai-jcm.116
LA  - en
ID  - SMAI-JCM_2024__10__373_0
ER  - 
%0 Journal Article
%A Aidan Chaumet
%A Jan Giesselmann
%T Improving Weak PINNs for Hyperbolic Conservation Laws: Dual Norm Computation, Boundary Conditions and Systems
%J The SMAI Journal of computational mathematics
%D 2024
%P 373-401
%V 10
%I Société de Mathématiques Appliquées et Industrielles
%U https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.116/
%R 10.5802/smai-jcm.116
%G en
%F SMAI-JCM_2024__10__373_0
Aidan Chaumet; Jan Giesselmann. Improving Weak PINNs for Hyperbolic Conservation Laws: Dual Norm Computation, Boundary Conditions and Systems. The SMAI Journal of computational mathematics, Volume 10 (2024), pp. 373-401. doi : 10.5802/smai-jcm.116. https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.116/

[1] Claude Bardos; Alain-Yves Leroux; Jean-Claude Nedelec First order quasilinear equations with boundary conditions, Commun. Partial Differ. Equations, Volume 4 (1979) no. 9, pp. 1017-1034 | DOI

[2] Pavel B. Bochev; Max D. Gunzburger Least-Squares Finite Element Methods, Applied Mathematical Sciences, Springer, 2009 https://books.google.de/books?id=5ze_xil-fnqc

[3] Constantine M. Dafermos Hyperbolic conservation laws in continuum physics, Grundlehren der Mathematischen Wissenschaften, 325, Springer, 2016, xxxviii+826 pages | DOI | MR

[4] Camillo De Lellis; Felix Otto; Michael Westdickenberg Minimal entropy conditions for Burgers equation, Q. Appl. Math., Volume 62 (2004) no. 4, pp. 687-700 | DOI | Zbl

[5] Tim De Ryck; Samuel Lanthaler; Siddhartha Mishra On the approximation of functions by tanh neural networks, Neural Netw., Volume 143 (2021), pp. 732-750 | DOI | Zbl

[6] Tim De Ryck; Siddhartha Mishra Generic bounds on the approximation error for physics-informed (and) operator learning, Advances in Neural Information Processing Systems 35 (NeurIPS 2022), Curran Associates, Inc. (2022), pp. 10945-10958 https://proceedings.neurips.cc/paper_files/paper/2022/file/46f0114c06524debc60ef2a72769f7a9-paper-conference.pdf

[7] Tim De Ryck; Siddhartha Mishra; Roberto Molinaro wPINNs: Weak Physics Informed Neural Networks for Approximating Entropy Solutions of Hyperbolic Conservation Laws, SIAM J. Numer. Anal., Volume 62 (2024) no. 2, pp. 811-841 | DOI | Zbl

[8] Waleed Diab; Mohammed Al Kobaisi PINNs for the Solution of the Hyperbolic Buckley-Leverett Problem with a Non-convex Flux Function (2021) | arXiv

[9] George Dinca; Petru Jebelean; Jean L. Mawhin Variational and topological methods for Dirichlet problems with p-Laplacian, Port. Math. (N.S.), Volume 58 (2001) no. 3, pp. 339-378 | MR

[10] Gamini Dissanayake; Nhan Phan-Thien Neural-network-based approximations for solving partial differential equations, Commun. Numer. Methods Eng., Volume 10 (1994) no. 3, pp. 195-201 | DOI | Zbl

[11] Weinan E; Bing Yu The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems, Commun. Math. Stat., Volume 6 (2018) no. 1, pp. 1-12 | DOI | Zbl

[12] Antonio Ferrer-Sánchez; José D. Martín-Guerrero; Roberto Ruiz de Austri-Bazan; Alejandro Torres-Forné; José A. Font Gradient-annihilated PINNs for solving Riemann problems: Application to relativistic hydrodynamics, Comput. Methods Appl. Mech. Eng., Volume 424 (2024), 116906 | DOI | Zbl

[13] Ulrik S. Fjordholm; Samuel Lanthaler; Siddhartha Mishra Statistical Solutions of Hyperbolic Conservation Laws: Foundations, Arch. Ration. Mech. Anal., Volume 226 (2017) no. 2, pp. 809-849 | DOI | Zbl

[14] Cedric G. Fraces; Hamdi Tchelepi Physics Informed Deep Learning for Flow and Transport in Porous Media, SPE Reservoir Simulation Conference (2021) | DOI

[15] Ian Gemp; Brian McWilliams The unreasonable effectiveness of adam on cycles, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada (2019) https://sgo-workshop.github.io/cameraready2019/11.pdf

[16] Jean-Luc Guermond A Finite Element Technique for Solving First-Order PDEs in L p , SIAM J. Numer. Anal., Volume 42 (2004) no. 2, pp. 714-737 | DOI | Zbl

[17] Jean-Luc Guermond; Fabien Marpeau; Bojan Popov A fast algorithm for solving first-order PDEs by L 1 -minimization, Commun. Math. Sci., Volume 6 (2008) no. 1, pp. 199-216 | DOI | MR | Zbl

[18] Jean-Luc Guermond; Bojan Popov L 1 -minimization methods for Hamilton–Jacobi equations: the one-dimensional case, Numer. Math., Volume 109 (2008) no. 2, pp. 269-284 | DOI | Zbl

[19] Zheyuan Hu; Ameya D. Jagtap; George Em Karniadakis; Kenji Kawaguchi When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?, SIAM J. Sci. Comput., Volume 44 (2022) no. 5, p. A3158-A3182 | DOI | Zbl

[20] Chen Jingrun A Comparison Study of Deep Galerkin Method and Deep Ritz Method for Elliptic Problems with Different Boundary Conditions, Commun. Math. Res., Volume 36 (2020) no. 3, pp. 354-376 | DOI | Zbl

[21] David I. Ketcheson; Kyle T. Mandli; Aron J. Ahmadia; Amal Alghamdi; Manuel Quezada de Luna; Matteo Parsani; Matthew G. Knepley; Matthew Emmett PyClaw: Accessible, Extensible, Scalable Tools for Wave Propagation Problems, SIAM J. Sci. Comput., Volume 34 (2012) no. 4, p. C210-C231 | DOI | Zbl

[22] Ehsan Kharazmi; Zhongqiang Zhang; George Em Karniadakis Variational Physics-Informed Neural Networks For Solving Partial Differential Equations (2019) | arXiv

[23] Diederik P. Kingma; Jimmy Ba Adam: A Method for Stochastic Optimization, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (Yoshua Bengio; Yann LeCun, eds.) (2015) | arXiv

[24] Cezar I. Kondo; Philippe G. LeFloch Measure-valued solutions and well-posedness of multi-dimensional conservation laws in a bounded domain, Port. Math. (N.S.), Volume 58 (2001) no. 2, pp. 171-193 | MR | Zbl

[25] Isaac E. Lagaris; Aristidis Likas; Dimitrios I. Fotiadis Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Netw., Volume 9 (1998) no. 5, pp. 987-1000 | DOI

[26] Isaac E. Lagaris; Aristidis Likas; D. G. Papageorgiou Neural-network methods for boundary value problems with irregular boundaries, IEEE Trans. Neural Netw., Volume 11 (2000) no. 5, pp. 1041-1049 | DOI

[27] John E. Lavery Nonoscillatory solution of the steady-state inviscid burgers’ equation by mathematical programming, J. Comput. Phys., Volume 79 (1988) no. 2, pp. 436-448 | DOI | Zbl

[28] Yulei Liao; Pingbing Ming Deep Nitsche Method: Deep Ritz Method with Essential Boundary Conditions, Commun. Comput. Phys., Volume 29 (2021) no. 5, pp. 1365-1384 | DOI | Zbl

[29] Li Liu; Shengping Liu; Hui Xie; Fansheng Xiong; Tengchao Yu; Mengjuan Xiao; Lufeng Liu; Heng Yong Discontinuity Computing with Physics-Informed Neural Network, J. Sci. Comput., Volume 98 (2024) no. 1, 22 | DOI | Zbl

[30] Emmanuel Lorin; Arian Novruzi Non-diffusive neural network method for hyperbolic conservation laws, J. Comput. Phys., Volume 513 (2024), 113161 | DOI | Zbl

[31] Kjetil O. Lye; Siddhartha Mishra; Deep Ray Deep learning observables in computational fluid dynamics, J. Comput. Phys., Volume 410 (2020), 109339 | DOI | Zbl

[32] Piotr Minakowski; Thomas Richter A priori and a posteriori error estimates for the Deep Ritz method applied to the Laplace and Stokes problem, J. Comput. Appl. Math., Volume 421 (2023), 114845 | DOI | Zbl

[33] Siddhartha Mishra; Roberto Molinaro Estimates on the generalization error of physics-informed neural networks for approximating PDEs, IMA J. Numer. Anal., Volume 43 (2022) no. 1, pp. 1-43 | DOI

[34] Guofei Pang; Lu Lu; George Em Karniadakis fPINNs: Fractional Physics-Informed Neural Networks, SIAM J. Sci. Comput., Volume 41 (2019) no. 4, p. A2603-A2626 | DOI | Zbl

[35] Evgueni Yu. Panov Uniqueness of the solution of the Cauchy problem for a first order quasilinear equation with one admissible strictly convex entropy, Math. Notes, Volume 55 (1994) no. 5, pp. 517-525 | DOI | Zbl

[36] Ravi G. Patel; Indu Manickam; Nathaniel A. Trask; Mitchell A. Wood; Myoungkyu Lee; Ignacio Tomas; Eric C. Cyr Thermodynamically consistent physics-informed neural networks for hyperbolic systems, J. Comput. Phys., Volume 449 (2022), 110754 | DOI | Zbl

[37] Maziar Raissi; Paris Perdikaris; George Em Karniadakis Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., Volume 378 (2019), pp. 686-707 | DOI

[38] Sashank J. Reddi; Satyen Kale; Sanjiv Kumar On the Convergence of Adam and Beyond, 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings (2018) https://openreview.net/forum?id=ryqu7f-rz

[39] Yeonjong Shin; Zhongqiang Zhang; George Em Karniadakis Error Estimates of Residual Minimization using Neural Networks for Linear PDEs, J. Mach. Learn. Model. Comput., Volume 4 (2023) no. 4, pp. 73-101 | DOI

[40] Justin Sirignano; Konstantinos Spiliopoulos DGM: A deep learning algorithm for solving partial differential equations, J. Comput. Phys., Volume 375 (2018), pp. 1339-1364 | DOI | Zbl

[41] Leslie N. Smith Cyclical Learning Rates for Training Neural Networks, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) (2017), pp. 464-472 | DOI

[42] Gary A. Sod A survey of several finite difference methods for systems of nonlinear hyperbolic conservation laws, J. Comput. Phys., Volume 27 (1978) no. 1, pp. 1-31 | DOI | Zbl

[43] Erik Laurin Strelow; Alf Gerisch; Jens Lang; Marc E. Pfetsch Physics informed neural networks: A case study for gas transport problems, J. Comput. Phys., Volume 481 (2023), 112041 | DOI | Zbl

[44] Magnus Svärd Entropy solutions of the compressible Euler equations, BIT Numer. Math., Volume 56 (2016) no. 4, pp. 1479-1496 | DOI | Zbl

[45] Chuwei Wang; Shanda Li; Di He; Liwei Wang Is L 2 Physics Informed Loss Always Suitable for Training Physics Informed Neural Network?, Adv. Neural Inf. Process. Syst., Volume 35 (2022), pp. 8278-8290

[46] Sifan Wang; Shyam Sankaran; Paris Perdikaris Respecting causality for training physics-informed neural networks, Comput. Methods Appl. Mech. Eng., Volume 421 (2024), 116813 | DOI | Zbl

[47] Sifan Wang; Yujun Teng; Paris Perdikaris Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks, SIAM J. Sci. Comput., Volume 43 (2021) no. 5, p. A3055-A3081 | DOI | Zbl

[48] Sifan Wang; Xinling Yu; Paris Perdikaris When and why PINNs fail to train: A neural tangent kernel perspective, J. Comput. Phys., Volume 449 (2022), 110768 | DOI | Zbl

Cited by Sources: