Recent advances in the literature show promising potential of deep learning methods, particularly neural operators, in obtaining numerical solutions to partial differential equations (PDEs) beyond the reach of current numerical solvers. However, existing data-driven approaches often rely on training data produced by numerical PDE solvers (e.g., finite difference or finite element methods). We introduce a “backward” data generation method that avoids solving the PDE numerically: by randomly sampling candidate solutions $u_j$ from the appropriate solution space (e.g., $H_0^1(\Omega )$), we compute the corresponding right-hand side $f_j$ directly from the equation by differentiation. This produces training pairs ${(f_j, u_j)}$ by computing derivatives rather than solving a PDE numerically for each data point, enabling fast, large-scale data generation consisting of exact solutions. Experiments indicate that models trained on this synthetic data generalize well when tested on data produced by standard solvers. While the idea is simple, we hope this method will expand the potential of neural PDE solvers that do not rely on classical numerical solvers for data generation.
Keywords: synthetic data, numerical PDEs, neural operators
Erisa Hasani 1; Rachel A. Ward 1
@article{SMAI-JCM_2025__11__497_0,
author = {Erisa Hasani and Rachel A. Ward},
title = {Generating synthetic data for neural operators},
journal = {The SMAI Journal of computational mathematics},
pages = {497--516},
publisher = {Soci\'et\'e de Math\'ematiques Appliqu\'ees et Industrielles},
volume = {11},
year = {2025},
doi = {10.5802/smai-jcm.132},
language = {en},
url = {https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.132/}
}
TY - JOUR AU - Erisa Hasani AU - Rachel A. Ward TI - Generating synthetic data for neural operators JO - The SMAI Journal of computational mathematics PY - 2025 SP - 497 EP - 516 VL - 11 PB - Société de Mathématiques Appliquées et Industrielles UR - https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.132/ DO - 10.5802/smai-jcm.132 LA - en ID - SMAI-JCM_2025__11__497_0 ER -
%0 Journal Article %A Erisa Hasani %A Rachel A. Ward %T Generating synthetic data for neural operators %J The SMAI Journal of computational mathematics %D 2025 %P 497-516 %V 11 %I Société de Mathématiques Appliquées et Industrielles %U https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.132/ %R 10.5802/smai-jcm.132 %G en %F SMAI-JCM_2025__11__497_0
Erisa Hasani; Rachel A. Ward. Generating synthetic data for neural operators. The SMAI Journal of computational mathematics, Volume 11 (2025), pp. 497-516. doi : 10.5802/smai-jcm.132. https://smai-jcm.centre-mersenne.org/articles/10.5802/smai-jcm.132/
[1] Variational analysis in Sobolev and BV spaces: applications to PDEs and optimization, Society for Industrial and Applied Mathematics, 2014 | DOI | MR | Zbl
[2] Model reduction and neural networks for parametric PDEs (2020) | arXiv
[3] GCN-FFNN: A two-stream deep model for learning solution to partial differential equations, Neurocomputing, Volume 511 (2022), pp. 131-141 | DOI
[4] Data-driven discovery of Green’s functions with human-understandable deep learning, Sci. Rep., Volume 12 (2022) no. 1, 4824
[5] Laplace neural operator for solving differential equations, Nat. Mach. Intell., Volume 6 (2024) no. 6, pp. 631-640
[6] Constructive approximation, Grundlehren der Mathematischen Wissenschaften, 303, Springer, 1993 | DOI | MR | Zbl
[7] Partial Differential Equations, Graduate Studies in Mathematics, American Mathematical Society, 2010 https://books.google.com/books?id=xnu0o_ejrcqc | Zbl
[8] Spectral neural operators, Doklady Mathematics, Volume 108 (2023), p. S226-S232 | DOI | MR | Zbl
[9] Geometrical structure of Laplacian eigenfunctions, SIAM Rev., Volume 55 (2013) no. 4, pp. 601-667 | DOI | MR | Zbl
[10] Poseidon: Efficient foundation models for pdes, Adv. Neural Inf. Process. Syst., Volume 37 (2024), pp. 72525-72624
[11] LordNet: An efficient neural network for learning to solve parametric partial differential equations without simulated data, Neural Netw., Volume 176 (2024), 106354
[12] Neural operator: Learning maps between function spaces with applications to pdes, J. Mach. Learn. Res., Volume 24 (2023) no. 89, pp. 1-97 | MR
[13] Leçons sur la théorie mathématique de l’élasticité des corps solides, Bachelier, 1852 https://books.google.com/books?id=tfedaaaaqaaj
[14] Fourier neural operator with learned deformations for pdes on general geometries, J. Mach. Learn. Res., Volume 24 (2023) no. 388, pp. 1-26 | MR
[15] Neural operator: Graph kernel network for partial differential equations (2020) | arXiv
[16] Multipole graph neural operator for parametric partial differential equations, Adv. Neural Inf. Process. Syst., Volume 33 (2020), pp. 6755-6766
[17] Non-equispaced fourier neural solvers for pdes (2022) | arXiv
[18] Pde-net: Learning pdes from data, International conference on machine learning, PMLR (2018), pp. 3208-3216
[19] Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators (2019) | arXiv
[20] Neural inverse operators for solving PDE inverse problems (2023) | arXiv | Zbl
[21] A metalearning approach for physics-informed neural networks (PINNs): Application to parameterized PDEs, J. Comput. Phys., Volume 477 (2023), 111912 | MR | Zbl
[22] A graph convolutional autoencoder approach to model order reduction for parametrized PDEs, J. Comput. Phys., Volume 501 (2024), 112762 | MR | Zbl
[23] U-no: U-shaped neural operators (2022) | arXiv
[24] Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations (2017) | arXiv
[25] Convolutional neural operators for robust and accurate learning of PDEs, Adv. Neural Inf. Process. Syst., Volume 36 (2023), pp. 77187-77200
[26] Code Verification by the Method of Manufactured Solutions (2000) https://www.osti.gov/biblio/759450 (Technical report) | DOI
[27] NOMAD: Nonlinear manifold decoders for operator learning, Adv. Neural Inf. Process. Syst., Volume 35 (2022), pp. 5601-5613
[28] Pseudo-differential neural operator: Generalized fourier neural operator for learning solution operators of partial differential equations (2022) | arXiv
[29] DGM: A deep learning algorithm for solving partial differential equations, J. Comput. Phys., Volume 375 (2018), pp. 1339-1364 | DOI | MR | Zbl
[30] Towards foundation models for scientific machine learning: Characterizing scaling and transfer behavior, Adv. Neural Inf. Process. Syst., Volume 36 (2023), pp. 71242-71262
[31] Enhanced deeponet for modeling partial differential operators considering multiple input functions (2022) | arXiv
[32] From finite differences to finite elements: A short history of numerical analysis of partial differential equations, J. Comput. Appl. Math., Volume 128 (2001) no. 1, pp. 1-54 (Numerical Analysis 2000. Vol. VII: Partial Differential Equations) | DOI | MR | Zbl
[33] Factorized fourier neural operators (2021) | arXiv
[34] Guaranteed approximation bounds for mixed-precision neural operators (2023) | arXiv
[35] Random grid neural processes for parametric partial differential equations, International Conference on Machine Learning, PMLR (2023), pp. 34759-34778
[36] Learning the solution operator of parametric partial differential equations with physics-informed DeepONets, Sci. Adv., Volume 7 (2021) no. 40, eabi8605
[37] et al. The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems, Commun. Math. Stat., Volume 6 (2018) no. 1, pp. 1-12 | MR | Zbl
[38] et al. MOD-Net: A machine learning approach via model-operator-data network for solving PDEs (2021) | arXiv
[39] Bayesian neural networks for weak solution of PDEs with uncertainty quantification (2021) | arXiv
Cited by Sources: