Optimal convergence rates in the context of neural networks
Vortrag von Prof. Dr. Michael Feischl
Datum: 14.05.25 Zeit: 16.30 - 18.00 Raum: ETH HG G 19.2
We present two recent results on the convergence rates of algorithms involving neural networks: First, we propose a hierarchical training algorithm for standard feed-forward neural networks that adaptively extends the network architecture as soon as the optimization reaches a stationary point. By solving small (low-dimensional) optimization problems, the extended network provably escapes any local minimum or stationary point. Under some assumptions on the approximability of the data with stable neural networks, we show that the algorithm achieves an optimal convergence rate s in the sense that loss is bounded by the number of parameters to the -s. Second, we show that quadrature with neural network integrands is inherently hard and that no higher-order algorithms can exist, even if the algorithm has access to the weights of the network.