Some Topics at the Intersection of Control, Dynamics, and Learning

Data-driven modeling typically involves simplifications of systems through dimensionality reduction (less variables) or through dimensionality enlargement (more variables, but simpler, perhaps linear, dynamics).  Autoencoders with narrow bottleneck layers are a typical approach to the former (allowing the discovery of dynamics taking place in a lower-dimensional manifold), while autoencoders with wide layers provide an approach to the latter, with "neurons" in these layers thought of as "observables" in Koopman representations. In the first part of this talk, I'll briefly discuss some theoretical results about each of these topics. (Joint work with M.D. Kvalheim on dimension reduction and with Z. Liu and N. Ozay on Koopman representations.)  The training of autoencoders, and more generally the solution of other optimization problems, including policy optimization in reinforcement learning, typically relies upon some variant of gradient descent. There has been much recent work in the machine learning, control, and optimization communities in the application of the Polyak-Łojasiewicz Inequality (PŁI) to such problems in order to establish exponential (a.k.a. “linear” in the local-iteration language of numerical analysis) convergence of loss functions to their minima under the gradient flow. A somewhat surprising fact is that the exponential rate, at least in the continuous-time LQR problem, vanishes for large initial conditions, resulting in a mixed globally linear / locally exponential behavior. This is in sharp contrast with the discrete-time LQR problem, where there is global exponential convergence. The gap between CT and DT behaviors motivated our work on generalizations of the PŁI condition, and the second part of the talk will address that topic. In fact, these generalizations are key to understanding the effect of errors in the estimation of the gradient. Such errors might arise from adversarial attacks, wrong evaluation by an oracle, early stopping of a simulation, inaccurate and very approximate digital twins, stochastic computations (algorithm "reproducibility"), or learning by sampling from limited data. We will suggest an input to state stability (ISS) analysis of this issue. Time permitting, we will also mention some initial results on the performance of linear feedforward networks in feedback control.  (Joint work with A.C.B. de Oliveira, L. Cui, Z.P. Jiang, and M. Siami).