Translation: Adaptive Loss Balanced Physical Information Neural Network

-- Neurocomputing -- 2022

I. Introduction

Considering that each weight of the loss function of PINN is fixed, and some studies have also observed that the training efficiency of PINN sensitively depends on the weights associated with different loss terms.But the general method of adjusting loss weights is time-consuming, laborious, and prone to errors and omissions.So the author hopes to find a more convenient way to adaptively learn loss weights - adaptive loss balanced physical information neural network (IbPINNs).

Second, method

2.1. Motivation

First of all, the loss of PINN is:

where the size of the loss weight λ is fixed, and

The author studies the effect of loss weights on PINN accuracy through numerical experiments.Here we demonstrate the results of learning the one-dimensional Poisson equation by PINN with different loss weights in Table 1.

The error curve is shown in Figure 2:

It is obvious that the performance of PINN is affected by the choice of loss weights.Therefore, it is necessary to propose a more convenient method to adaptively learn these loss weights, thereby improving the accuracy and robustness of PINN.

2.2, IbPINNs

The author has established a Gaussian probability model whose output is u.Gaussian likelihood is defined as Gaussian with mean, approximated by PINN and uncertainty parameter:

Based on the minimization objective, minimize the negative log-likelihood of the model:

Similarly, a Gaussian probability model with output g or h can also be built to define the BC and IC losses of PINN.Further, suppose the output of the Gaussian probability model consists of two vectors u, g, each following a Gaussian distribution:

This leads to the minimization objective of the multi-output model:

Therefore, the author builds a multi-output model of four vectors to define the loss function.The loss function of adaptive loss-balanced PINN (IbPINNs) can be expressed as:

On the one hand, when the adaptive weight decreases, the total weightIncrease, which means that the The penalty is bigger; on the other hand, the last item Prevents adaptive weights from dropping too much.In general, a larger adaptive weight reduces the contribution of the loss term, while a smaller adaptive weight increases its contribution and penalizes the model.That is, the method of automatically adjusting the adaptive weight of each loss item.

Transform the above loss:

With exponential mapping, the minimization of the loss function can be made unconstrained.Since exp(-s) resolves to the positive domain, the adaptive weights do not converge to zero very quickly.Training will be numerically more stable.A schematic diagram of ibpinn is shown in Figure 3.The method is summarized as Algorithm 1.

Three, experiment

The two-dimensional Poisson equation

Results: