I will present the current research activities in our « Image and Video Understanding Laboratory » (IVUL – https://ivul.kaust.edu.sa) at KAUST (King Abdullah University of Science and Technology). The applications will range from video understanding to autonomous navigation, with solid fundamentals in deep learning theory. First, I will introduce the IVUL research on « Video Understanding », including pioneer work on activity proposals and localization, video retrieval and visual object tracking. Then, I will present our works on « Vision for Automation », focusing on 3D computer vision and imitation learning applied on self navigation, underlining the importance of simulation and transfer learning. Last, I will present the on-going research on « Deep Learning Fundamentals », including large scale optimization, analytical expression for neural networks and robustness to adversarial attack.