Applied Math Colloquium: Alena Kopaničáková (Brown U)
Location
Mathematics/Psychology
Date & Time
October 6, 2023, 11:00 am – 12:00 pm
Description
Title: Enhancing Training of Deep Neural Networks Using Multilevel and Domain Decomposition Strategies
Abstract: The training of deep neural networks (DNNs) is traditionally accomplished using stochastic gradient descent or its variants. While these methods have demonstrated certain robustness and accuracy, their convergence speed deteriorates for large-scale, highly ill-conditioned, and stiff problems, such as ones arising in scientific machine learning applications. Consequently, there is a growing interest in adopting more sophisticated training strategies that can not only accelerate convergence but may also provide parallelism, convergence control, and automatic selection of certain hyper-parameters. In this talk, we propose to enhance the training of DNNs by leveraging nonlinear multilevel and domain decomposition strategies. We will discuss how to construct a multilevel hierarchy and how to decompose the parameters of the network by exploring the structure of the DNN architecture, properties of the loss function, and characteristics of the dataset. Further- more, the dependency on a large number of hyper-parameters will be reduced by employing a trust-region globalization strategy. The effectiveness of the proposed training strategies will be demonstrated through a series of numerical experiments from the field of image classification and physics- informed neural networks.
We will have the Departmental Coffee and Tea from 10 to 10:45 in M&P 422.
Tags: