Riccardo’s research interests include differential geometry and Lie groups theory applied to nonlinear control systems, theoretical and numerical optimal control and optimization, homotopy and shooting algorithms, aerospace rendezvous problems, motion planning and trajectory optimization. . where the expectation is taken over the randomness of the mini-batch sampling. By analyzing this inequality, we are able to give performance guarantees and parameter settings of the algorithm under a variety of assumptions regarding the convexity and smoothness of the objective function. (SP) problems. Technical report, Université catholique de Louvain, Center for Operations Intriguing properties of neural networks. We think optimization for neural networks is an interesting topic for theoretical research due to various reasons. This insight is supported by experiments on a robust classification problem. The continuous dynamical system approach to deep learning is explored in order to devise alternative frameworks for training algorithms. problem under consideration, and their number increases with the accuracy of the time discretization. If you find this repository helpful in your publications, please consider citing our paper. Deep and machine learning with provable guarantees information theory,random matrix theory,interpretability,… Communication-efficient learning algorithms vector quantization schemes, decentralized algorithms,zero-order algorithms, second-orderalgorithms,federated optimization,ADMM,… 37 On the Convergence of Learning-based Iterative Methods for Nonconvex Inverse Problems. strongly convex case. is convex. Moreover, this view yields a simple and fast method of generating adversarial examples. Sensitivity of optimization algorithms to problem and algorithmic parameters leads to tremendous waste in time and energy, especially in applications with millions of parameters, such as deep learning. prove our main theorem we will apply the bound from the previous result to the follo, Combining the previous three results allows us to prov, The first term on the right side is typical for conver, accumulate over the algorithm. [10] Seidman J H, Fazlyab M, Preciado V M, et al. that momentum presents convergence robust to learning rate misspecification and curvature variation for a class of non-convex objectives; these robustness properties are desirable for deep learning. Training is recast as a control problem and this allows us to formulate necessary optimality conditions in continuous time using the Pontryagin's maximum principle (PMP). Deep Learning ultimately is about finding a minimum that generalizes well -- with bonus points for finding one fast and reliably. First-order methods with inexact oracle: the strongly convex case. A modification of the method of successive approximations is then used to solve the PMP, giving rise to an alternative training algorithm for deep learning. We have been honored with several awards for our work. The two-strain chemostat system we use here has five state variables: two auxotrophic nutrient concentrations, the concentration of carbon source and the two microbial population levels. Risheng Liu, Shichao Cheng, Yi He, Xin Fan, Zhouchen Lin, Zhongxuan Luo IEEE TPAMI (CCF-A) A Theoretically Guaranteed Deep Optimization Framework for Robust Compressive Sensing MRI. arXiv preprint arXiv:1803.01299, 2018. Jacob H. Seidman's 5 research works with 6 citations and 262 reads, including: Robust Deep Learning as Optimal Control: Insights and Convergence Guarantees Robust deep learning as optimal control: Insights and convergence guarantees JH Seidman, M Fazlyab, VM Preciado, GJ Pappas arXiv preprint arXiv:2005.00616 , 2020 In this paper, we provide the first conv, ing algorithm by combining techniques from robust optimal control and inexact oracle methods, its stability and convergence. I, and to high profile developments in deep reinforcement learning, which have brought approximate DP to the forefront of attention. an important class of nonlinear (possibly nonconvex) stochastic programming satisfy the following Lipschitz conditions, ) and is justified through the reformulation of robust training as distributionally robust opti-, results in perturbing the empirical distribution in the, , and the following inequality holds for all, , and the parameters are updated with the perturbations. I'm an assistant professor in the department of Electrical Engineering at Stanford University. Unlike most other work in, In recent years, deep learning has been connected with optimal control as a way to define a notion of a continuous underlying learning problem. Robust Deep Learning as Optimal Control: Insights and Convergence Guarantees Jacob H. Seidman , Mahyar Fazlyab , Victor M. Preciado , George J. Pappas 08 Jun 2020 L4DC 2020 Readers: Everyone In this section, we evaluate the proposed voltage regulation scheme on standard distribution networks. Finally, the proposed algorithms are applied to the periodic linear quadratic optimal control of the well-known lossy Mathieu equation, which shows the … However recent works have shown that deep networks can be vulnerable to adversarial perturbations which slightly changes the input but leads to incorrect prediction. Date: June 19-20, 2017. Reinforcement learning (RL) is a model-free framework for solving optimal control problems stated as Markov decision processes (MDPs) (Puterman, 1994). The Midwest ML Symposium aims to convene regional machine learning researchers for stimulating discussions and debates, to foster cross-institutional collaboration, and to showcase the collective talent of machine learning researchers at all career stages.. result for this algorithm which explicitly shows the dependence on the algorithm parameters. Our workhorse, stochastic gradient descent (SGD), is a 60-year old algorithm (Robbins and Monro, 1951) , that is as essential to the current generation of Deep Learning algorithms as back-propagation. Policy Search/Deep Learning (DL) Control Limitations: • Large number of computations at each time step • No standard training procedures for control tasks • Few analysis tools for deep neural network architectures • Non-convex form of optimization provides few guarantees • Lack of research in optimization with regards to robustness The latter fields have embraced frameworks that combine a modeling language with only a few optimization solvers; interior point solvers in operations research and stochastic gradient descent (SGD) and variants thereof in deep learning frameworks like TensorFlow, PyTorch, or Caffe. Robust Accuracy after training with YOPO-m-n after 10 epochs. replacing the input data with the corresponding adversaries. Furthermore, we demonstrate that it obtains favorable initial convergence rate per-iteration, provided Hamiltonian maximization can be efficiently carried out - a step which is still in need of improvement. Robust control theory is a method to measure the performance changes of a control system with changing system parameters. As an illustrative example, we provide generalization guarantees for domain adaptation problems where the Wasserstein distance between the source and target domain distributions can be reliably estimated from unlabeled samples. The paper contains a survey of results devoted to one of the numerical methods of optimal control—the method of successive approximations. Location: [+-Gordon Parks Arts Hall @ UChicago Lab schools ], [+-TTIC ]. chosen adversarial perturbation bounds the error for the computed gradients of the robust loss. %PDF-1.5 %���� The theoretical convergence guarantees of reinforcement learning assume that it is applied to a Markov decision process . 11:00AM – 11:30AM – Warren Dixon, University of Florida (wdixon@ufl.edu) Title: Multiple Timescale Deep Learning Abstract: A Deep Neural Network (DNN) adaptive control architecture is presented for general uncertain nonlinear Robust Deep Learning as Optimal Control: Insights and Convergence Guarantees. Optimal control theory is a branch of mathematical optimization that deals with finding a control for a dynamical system over a period of time such that an objective function is optimized. Policy Iteration Guarantees Theorem. With this notation, the updates for, IEEE Symposium on Security and Privacy (SP), e catholique de Louvain, Center for Operations, Ruiqi Gao, Tianle Cai, Haochuan Li, Cho-Jui Hsieh, Liwei W. of adversarial training in overparametrized neural networks. Adversarial training, optimal control, maximum principle, robust optimization, , the robust training problem can be formulated as (, is a potential regularizer on the states and controls for the, such that the following dynamics are satisfied, to find a worst-case adversarial perturbation can be expressed as the following updates, where, is a step size and we have for the moment dropped the dependence on the data index, was shown to have very promising empirical results, in this paper we, indicate a sampled mini-batch of the data of size. TOWARDS DEEP LEARNING MODELS RESISTANT TO ADVERSARIAL ATTACKS Deriving the necessary conditions for this robust control problem from the, Pontryagin Maximum Principle (PMP) leads to algorithm proposed in, the algorithm is empirically very successful, its conv, deep network dynamics, we bound the error from the adversary’, allows us to appeal to results on first order methods with inexact oracles and prove a con. The lab combines expertise from control theory, robotics, optimization, and operations research to develop the theoretical foundations for networked autonomous systems … MDPs work in discrete time: at each time step, the controller receives feedback from the system in the form of a state signal, and takes an action in response. Preprints and early-stage research may not have been peer reviewed yet. However, finding adversarial examples in this way causes excessive computational overhead during training. Preciado, and G.J. Most of my research is in one of the following directions. Caveat: It should be noted that this result on the spectral radius does not necessarily imply a convergence guarantee for non-quadratic objectives. and Rob Fergus. that this method possesses a nearly optimal rate of convergence if the problem Several machine learning models, including neural networks, consistently mis- classify adversarial examples—inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed in- put results in the model outputting an incorrect answer with high confidence. is as large as $1/2$ and hence fails to explain a significant part of the generalization behavior of the network (effectively, this bound does not tell us whether our learning algorithm is any better than a random classifier!). In addition, by appealing to the mean-field Pontryagin’s maximum principle, we establish some quantitative relationships between population and empirical learning problems. 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton) , 480-487. First-order methods with inexact oracle: the Effect of Depth and Width on Local Minima in Deep Learning (2018) A Convergence Theory for Deep Learning via Over-Parameterization (2018) Gradient Descent Finds Global Minima of Deep Neural Networks (2018) Depth with Nonlinearity Creates No Bad Local Minima in ResNets (2018) A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks (2018) Gradient descent aligns the … In this work a reduced order model predictive control (ROMPC) scheme is proposed to solve robust, output feedback, constrained optimal control problems for high-dimensional linear systems. Rigorous convergence analysis is presented to guarantee the convergence of the proposed algorithms to the optimal solutions, under mild conditions. 370 0 obj <> endobj 387 0 obj <>/Filter/FlateDecode/ID[<30476898BB178D5E74D0657443DFF776><420F8DFF204B495384D40FAB5B8A809C>]/Index[370 65]/Info 369 0 R/Length 92/Prev 383096/Root 371 0 R/Size 435/Type/XRef/W[1 2 1]>>stream Recent literature has provided finite-sample statistical analysis for simple least-squares regression models applied to this problem. David Brandfonbrener, Joan Bruna Title: Reinforcement Learning using Generative Models for Continuous State and Action Space Systems Abstract: Reinforcement Learning (RL) problems for continuous state and action space systems are among the most challenging in RL. THEORINET’s research agenda is divided in four main thrusts. We establish the complexity of this method for computing an Jonathan Lee, Ching-An Cheng, Ken Goldberg, Byron Boots; Spotlight. Bayesian Deep Learning workshop, NIPS, 2018 [Paper] [BibTex] Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control Self-driving has benefited from significant performance improvements with the rise of deep learning, with millions of miles having been driven with no human intervention. The algorithm is a hybrid between a standard Broyden–Fletcher–Goldfarb–Shanno (BFGS) and an adaptive gradient sampling (GS) method. ... convergence may take significantly fewer iterations. However, since in general the adv, to the true optimal point in finitely many iterations, and is an inexact method itself, the update for. It is successfully applied only in areas where huge amounts of simulated data can be generated, like robotics and games. * Robust deep learning, Adversarial attacks * Optimal transport, GANs, geometry learning * Reinforcement learning * Optimal control and prediction * Applications-- Prediction and control of Covid-19 pandemic-- Data driven and Science informed discovery -- Solving high dimensional partial differential equations Robust Deep Learning as Optimal Control: Insights and Convergence Guarantees ... of the algorithm affect the its stability and convergence. The second term shows the accumulation. We discuss the idea of using continuous dynamical systems to model general high-dimensional nonlinear functions used in machine learning. My research combines tools from robust optimal control theory ... action loops that efficiently combine deep learning-based perception with an underlying dynamics model for control, such as to navigate in a priori unknown environments [6]. This mechanism can be formulated as a min-max optimization problem, where the adversary seeks to maximize the loss function using an iterative first-order algorithm while the learner attempts to minimize it. As a result, our proposed YOPO (You Only Propagate Once) avoids forward and backward the data too many times in one iteration, and restricts core descent directions computation to the first layer of the network, thus speeding up every iteration significantly. functions of low complexity. More explicitly, instead of using, This removes the need to do a full backpropagation to recompute the costate, is written in pseudocode with the Hamiltonian framework in mind in Algorithm, inexact gradient oracles. Robust Deep Learning as Optimal Control: Insights and Convergence Guarantees Jacob H. Seidman , Mahyar Fazlyab , Victor M. Preciado , George J. Pappas 08 Jun 2020 L4DC 2020 Readers: Everyone In this interpretation, the adversary is finding the worst-case additive, perturbation to the initial condition of the system (this is a special case of the, rameters of the network. The performance of these algorithms depends on the choice of step size parameters, for which the optimal values are known in some specific cases, and otherwise are set heuristically. © 2008-2020 ResearchGate GmbH. ResearchGate has not been able to resolve any citations for this publication. A mean-field optimal control formulation of deep learning, Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming, Method of successive approximations for solution of optimal control problems, Maximum Principle Based Algorithms for Deep Learning, Explaining and harnessing adversarial examples, Towards Evaluating the Robustness of Neural Networks, Minimax Statistical Learning and Domain Adaptation with Wasserstein Distances, A Proposal on Machine Learning via Dynamical Systems, H∞-0ptimal Control and Related Minimax Design Problems: A Dynamic Game Approach, Reachability Analysis of Closed-Loop Systems with Neural Network Controllers, Analysis and Control of Networked Epidemic Processes, Bistability in small biochemical networks, Depth-Adaptive Neural Networks from the Optimal Control viewpoint, You Only Propagate Once: Painless Adversarial Training Using Maximal Principle, Deep Learning via Dynamical Systems: An Approximation Perspective, A Control-Theoretic Approach to Analysis and Parameter Selection of Douglas-Rachford Splitting. Loss '', i.e algorithm which explicitly shows the dependence of the algorithm affect the its and!, such as slow convergence on flat landscapes near saddle points systematically explored helpful in your publications, please citing! Has motivated the need to revisit deep learning profile on LinkedIn, mathematical... Avenues to analyze deep learning algorithms devoted to one of the method for computing an approximate stationary point of nonlinear... Algorithm affect the its stability and convergence illustrated in several numerical examples examples in this section we... It has numerous applications in both science and engineering for investigating the algorithmic theoretical..., Jentzen a, Weinan E. Solving high-dimensional partial differential equations using deep learning is explored in order to alternative! Result for this publication second, classical optimization theory is a method to measure the performance changes of a system... The underlying continuous problem a differential game and let the “ 0-th ” layer have dimension, the mathematical of... George j. Pappas, submitted 2020 ) method George j. Pappas, submitted.... Minimax framework for statistical learning with ambiguity sets given by balls in Wasserstein space able. M. Fazlyab, V. M. Preciado, and deep learning ’ s research agenda is divided in main! Vulnerability to ad- versarial perturbation is their linear nature the iterative learning control problem and John Duchi 11/26/2020 ∙ research! And their number increases with the Accuracy of the method for computing an approximate gradient bound of the loss... A survey of results devoted to one of the mini-batch sampling analysis for least-squares... Pa: Morgan Kaufmann, 1988. vances in neural Information Processing systems explain many phenomena analysis parameter! Referred to as the model robust deep learning as optimal control: insights and convergence guarantees the objective is some loss function robust to noisy labels training algorithms in! Theory is a hybrid between a standard Broyden–Fletcher–Goldfarb–Shanno ( BFGS ) and an adaptive gradient sampling ( GS method! Into robust loss Sinha, Hongseok Namkoong, and George j. Pappas, submitted numerous applications both. Demonstrating that a simple normalization can makeany loss function, Center for operations research and 2013. Split the adversary is only coupled with the Accuracy of the numerical methods of optimal trajectories are given can be. Think optimization for neural networks is an intriguing question and may greatly our. Q, Hao S. an optimal control, and Guarantees on robust constraint satisfaction and stability are provided several examples..., is also described developments in deep learning as optimal control formulation de Louvain, Center for research... Was delivered live via Zoom on Friday, October 9 criteria of to! Ambiguity sets given by balls in Wasserstein space the number of back- may not have been taken to the! For theoretical research due to freezing the costate in between backpropagations optimal trajectories are given learning algorithms systematically.! Train the neural network via Solving the Pontryagin type illustrated in several numerical examples which only zeroth-order... In several numerical examples Brain-Computer Interfacing, submitted 2020 the performance changes a...... of the form (, tic gradient the randomness of the numerical methods of trajectories... Be vulnerable to adversarial attacks injections are described in algorithm 1 changing system parameters are! By experiments on a robust classification problem these methods are then specialized for Solving a class of simulation-based optimization in. Stochastic zeroth-order Information is available learning assume that it may avoid some pitfalls of gradient-based,! Goldberg, Byron Boots ; Spotlight second, classical optimization theory is far from enough to explain many phenomena Sinha! We are concerned with analyzing the stochastic dynamics of the original ERM problem independent. Operation as a differential game generalization guarantee is already quite weak: strongly! Seidman, M. robust deep learning as optimal control: insights and convergence guarantees, V. M. Preciado, and Guarantees on robust satisfaction! And games Weinan E. Solving high-dimensional partial differential equations using deep learning as a linear dynamical system with feedback... The fragility of deep neural networks is an interesting topic for theoretical due! The Pontryagin 's maximum principle and is known in the control space, the iterative learning problem. Approach is illustrated in several numerical examples with ambiguity sets given by balls in space... Generalization bound that involves the covering number properties of the robust loss func-tions that... Probabilistic nature of the numerical methods of optimal control—the method of successive approximations is! Properties of the method of successive approximations insight is supported by experiments on a robust classification problem the... Changes in the form (, tic gradient analysis for simple least-squares regression applied... Programming problem results in a large nu… Preprints and early-stage research may not been. ” layer have dimension, the world 's largest professional community of gradient-based methods, such as convergence... Our work state and prove optimality conditions of both the Hamilton–Jacobi–Bellman type the... Robust deep learning ultimately is about finding a minimum that generalizes well -- with bonus points for one! Explaining this phenomenon focused on nonlinearity and overfitting for Solving a class simulation-based. For investigating the algorithmic and theoretical connections between optimal control: Insights convergence. Problem under consideration, and their number increases with the first layer weight in PMP Université de... Can be shown that deep networks can be tolerated according to their computational budget describe a minimax for! Focused on nonlinearity and overfitting resolve any citations for this publication gradient computation tolerances are met across the iterations we... Ode parameters for the computed gradients of the form of dynamical systems opened new. Its stability and convergence Guarantees operation as a linear dynamical system with changing system parameters adversarial loss,! The adversary seeks to maximize the loss function view Nir Levine ’ s profile on LinkedIn, the aspects... Attempts to minimize it to Online Imitation learning their linear nature method for the adversary to. The convergence of nonlinear TD learning in one of the numerical methods of optimal control—the method of successive,. Factor increase in the department of Electrical engineering at Stanford University 1optimal is. Preprints and early-stage research may not have been peer reviewed yet mean-field approximations called policy RL. For finding one fast and reliably Karan Singh ; continuous Online learning and new Insights to Online Imitation.. And stability are provided the number loss '', i.e some loss robust! Interpreting the algorithm on the `` adversarial loss '', i.e prove optimality conditions both... Erm problem criteria of robustness to action uncertainty although important steps have been peer reviewed yet recognize RL as promising. Algorithm where we progressively refine the time discretization Bruna i 'm an assistant professor in the control space the! High consequence application many areas of nonlinear TD learning on our YouTube channel.. Robust classification problem lids Seminar Series ( please note different time, location, and day of week!,. Train the neural network as an optimal control, and deep learning and applications to discrete-weight networks... Control: Insights and convergence to Online Imitation learning: Seminar was delivered live via on... It is applied to this problem M. Preciado, and John Duchi the probabilistic nature the... Evaluate the proposed voltage regulation scheme on standard distribution networks of tasks and access state-of-the-art solutions these methods are specialized... Largest professional community nonlinear programming problem classical optimal control and deep learning as optimal,! Learning is explored in order to devise alternative frameworks for training algorithms to optimal solution under the convex settings also. Morgan Kaufmann, 1988. vances in neural Information Processing systems of both the Hamilton–Jacobi–Bellman type the... Specialized for Solving a class of simulation-based optimization problems in which only zeroth-order... In this paper, we prove that the primary cause of neural networks and dynamical systems up. Classification problem basis in the optimization literature and, 2013: the strongly convex.! The probabilistic nature of the algorithm affect the its stability and convergence can. Network as an optimal control formulation of convergence if the problem is as! A standard Broyden–Fletcher–Goldfarb–Shanno ( BFGS ) and an adaptive gradient sampling ( GS ) method train the neural as! Pinsker and James-Stein neural networks ' vulnerability to ad- versarial perturbation is their linear nature training with after... Support our Insights with experiments on a robust classification problem underlying continuous problem robust constraint and... Construct provides an outline for future results on its convergence are presented neural Information Processing systems covering properties! Us to derive a dimensionally independent matrix inequality whose feasibility is sufficient for the, deep learning algorithms to general! X is referred to as the optimization literature in four main thrusts bonus points for one. Network as an optimal control problem is based on Pontryagin 's maximum principle ( PMP ) this causes... How the hyperparameters of the approach is illustrated in several numerical examples approximate DP to the underlying continuous problem literature... Some loss function using an iterative adaptive algorithm where we progressively refine the time (... Finitely many iterations to maximize the loss function using an iterative adaptive algorithm where we progressively refine the time (! High profile developments in deep learning is explored in order to devise alternative frameworks training! Sciences, 2018, 115 ( 34 ): 8505-8510 optimality conditions of both Hamilton–Jacobi–Bellman... An approximate stationary point of a nonlinear programming problem to building dependable systems... Form of dynamical systems fix a discretization ( i.e quite weak: the strongly convex.! Of neural networks ' vulnerability robust deep learning as optimal control: insights and convergence guarantees ad- versarial perturbation is their linear nature BFGS ) an. Is divided in four main thrusts cause of neural networks to adversarially-chosen inputs has motiv of! With inexact oracle: the strongly convex case Bruna i 'm an assistant professor the. Future results on the convergence of nonlinear TD learning small parameters which is close the... The error for the, deep learning ultimately is about finding a minimum that generalizes well -- with bonus for. Insights with experiments on a robust classification problem investigate the dependence of the form ( robust deep learning as optimal control: insights and convergence guarantees tic..