Recent & Upcoming Talks

In their seminal paper in 2002, Joe Cartwright and Mads Huuse referred to 3D seismic reflection data as the ‘Geological Hubble’, arguing these data had the potential to revolutionise our understanding of the genesis and evolution of sedimentary basins. Almost two decades later, I will here outline just some of the key recent advances made in our understanding of basin structure and stratigraphy, focusing on: (i) the intrusion and extrusion of igneous rocks; (ii) salt tectonics; (iii) the geometry, growth, and seismogenic hazard posed by normal faults; and (iv) the structure and emplacement of submarine landslides. I will stress that future advances in these and other areas relies on energy companies and government agencies continuing to make their data freely available via free, easy-to-access data portals. I will issue a clarion call to academics, stressing that ‘geodynamicists’, sedimentologists, structural geologists and geomorphologists, amongst many, many others, can benefit from using what I believe are currently an underused data type. Most fundamentally, seismic reflection data should be used across the undergraduate syllabus, such that future generations of geoscientists are aware of its power and limitations.

Robots are envisioned as capable machines who easily navigate and interact in a world built for humans. However, looking around us we see robots mainly confined to factories only performing repetitive tasks in environments built such as to circumvent their limitations. The central question of my research is how we can create robots that are capable of adapting such that they can co-inhabit our world. This means designing systems that are capable of functioning in unstructured environments that are continuously changing with unlimited combination of shapes, sizes, appearance, and positions of objects, able to understand, adapt and learn from humans, and importantly do so from small amounts of data. In specific my work focuses on grasping and manipulation, fundamental aspects to enable a robot to interact with humans in our environment, along with dexterity (e.g. to use objects/tools successfully) and high-level reasoning (e.g. to decide about which object/tool to use). Despite decades of research, robust autonomous grasping and manipulation approaching human skills remains an elusive goal. One main difficulty lies in dealing with the inevitable uncertainties in how a robot perceives the world. Our environment is dynamic, has a complex structure and sensory measurements are noisy and associated with a large degree of uncertainty which poses challenges to avoid failures. In my research I have developed methodologies that enable a robot to interact with natural objects and learn about object properties and relations between tasks and sensory streams. I have developed tools that allow a robot to use multiple streams of sensory data in a complementary fashion. In this talk I will specifically address how a robot can use vision and touch to address grasp related questions e.g. estimating unknown object properties such as shape, grasp stability estimation before manipulating objects that can be used to trigger plan corrections, and grasp adaptation/correction.

The Earth’s system dynamics has an intrinsically multi-scale and nonlinear nature, which fundamentally affects the ability to perform weather forecasting and climate projections. In the face of complex dynamics, conventional data analysis techniques (such as EOF analysis) have frequently ambiguous physical interpretation. Moreover, validation of the extracted signals against observations is oftentimes limited by the time span of the observational record, which can be shorter than the timescales of interest, and also significantly altered by anthropogenic factors. Here, we introduce a new data analysis technique for complex dynamical systems and demonstrate its potential by investigating Indo-Pacific climate variability in models and observations. Through this technique, drawbacks associated with ad-hoc filtering are avoided as the extracted signals span many temporal scales without pre-processing the input data. In addition, we discuss the applicability of our approach in non-stationary signals (i.e. trends), and also describe a modification suitable for skillful extraction of coherent patterns in systems lacking spatiotemporal separability (e.g. turbulence). As an extension of this work, we present a framework for data assimilation and forecasting of non-linear time series.

To achieve long-term climate change goals, such as limiting global warming to 1.5 or 2°C, there must be a global effort to decide and act upon effective but realistic emission scenarios. This requires an understanding of the consequences of various scenarios and the climate response associated with these. In particular, different quantities and types of emissions, such as greenhouse gases and aerosols, are responsible for diverse changes in climate on a global and local scale. However, state-of-the-art climate models required for long-term climate change projections are computationally expensive. Here I will introduce a novel machine learning approach, which uses a unique dataset of existing simulations to learn relationships between short- and long-term temperature responses to different emission scenarios. We have explored several statistical techniques for this supervised learning task and here I present predictions made with Ridge regression and Gaussian process regression. These methods outperform pattern scaling, a standard simplified approach for estimating spatial patterns of temperature response. I will highlight challenges and opportunities for data-driven climate model emulation, especially concerning the incorporation of even larger model datasets in the future.

Way too often, observations from weather stations fall outside of the ensemble generated by initial uncertainty in weather models. This is because as of yet, many important physical processes that contribute to the weather dynamics are not resolved in numerical models, despite the increase in computing power in recent years. It is therefore necessary to incorporate uncertainty at the level of the model, without destroying the underlying physics. To this end, we introduce SALT (Stochastic Advection by Lie Transport), a framework that derives stochastic PDEs for weather models from a variational principle. This introduces a stochastic term that models the statistics of sub-grid scale turbulence, which is to be learned from data, while still preserving fundamental invariants of the flow. Techniques from machine learning are necessary to learn such stochastic parameterisations. I will also briefly discuss at the end a recent work we did on representing the climate system (as opposed to weather) as a stochastic PDE that is non-local in probability space, and its connection to turbulence closure theory.

Robots are envisioned as capable machines who easily navigate and interact in a world built for humans. However, looking around us we see robots mainly confined to factories only performing repetitive tasks in environments built such as to circumvent their limitations. The central question of my research is how we can create robots that are capable of adapting such that they can co-inhabit our world. This means designing systems that are capable of functioning in unstructured environments that are continuously changing with unlimited combination of shapes, sizes, appearance, and positions of objects, able to understand, adapt and learn from humans, and importantly do so from small amounts of data. In specific my work focuses on grasping and manipulation, fundamental aspects to enable a robot to interact with humans in our environment, along with dexterity (e.g. to use objects/tools successfully) and high-level reasoning (e.g. to decide about which object/tool to use). Despite decades of research, robust autonomous grasping and manipulation remains an elusive goal. The difficulty lies in dealing with the inevitable uncertainties in how a robot perceives the world. Our environment is dynamic, has a complex structure and sensory measurements are noisy and associated with a large degree of uncertainty. In real-world settings, these issues can lead to grasp failures with serious consequences. In my research I have developed methodologies that enable a robot to interact with natural objects and learn about object properties and relations between tasks and sensory streams. I have developed tools that allow a robot to use multiple streams of sensory data in a complementary fashion. In this talk I will specifically address how a robot can use vision and touch to address grasp related questions e.g. estimating unknown object properties or grasp success before manipulating objects that can be used to trigger plan corrections.

Recent advancement of AI technologies like deep learning or deep reinforcement learning greatly enhance robot’s capacities in many practical applications. In this talk, I will focus on how computer vision, deep learning and sensor fusion can be leveraged for better robotic perception and manipulation which enables the robots to better serve for industrial application and healthcare.I will start with our current work that exploits deep learning-based method to achieve the accurate estimation of object postures in the real scene. Bin picking robots have been explored for years.With the advancement in deep learning on point cloud, the object detection and pose estimation are intelligently achieved without explicit feature extraction. However, existing methods are only capable of estimating the bounding box but not the precise 3D position of an object. We propose anew learning-based object detection and pose estimation method for bin picking problem. We leverage PointNet and its following ideas for estimating 6 DoF parameters of objects. Our network estimates the translations and then rotations, and finally, we integrate the estimated postures by using mean-shift clustering. We trained our network using the data generated by physical simulations and stereo measurement system simulations. We demonstrate that our network can achieve the accurate estimation of object postures in the real scene without any transfer learning techniques. Furthermore, I will present our recent work on Robot-Enhanced therapy for children with autism spectrum disorder (ASD). Traditionally, diagnosis of ASD depend on doctor’s looking at a patient’s behavior and development, which is subjective and time consuming. We present a novel sensing-enhanced therapy system including three cameras and two Kinects to automatically, accurately and non-intrusively monitor ASD children in multiple scenarios. Involved technics are including but not limited to face expression, human motion analysis and gaze estimation.Furthermore, we propose a multimodal approach to detect ASD severity in participants undergoing treatment for ASD.Finally, I will present our earlier work on an ultrasound-based human-machine interface(HMI) for dexterous prosthetic control. Surface electromyography (EMG) is widely investigated in HMI by decoding movement intention to intuitively control intelligent devices, but could not offer satisfactory solutions for finger motion classification. We build a novel B mode ultrasound image-based HMI. Then, I present experiments on a finger motion task where this ultrasound-based HMI has a better average accuracy (95.88%) than the EMG-based HMI(90.14%) for finer motion recognition. Furthermore, I will discuss how to use machine learning and image process methods to improve the robustness of our proposed HMI in practice.

Variational inference (VI) is a versatile framework to perform approximate inference in probabilistic models. Recent progress made it widely applicable to Gaussian Process (GP) models: the use of sparse GPs as approximate posterior processes and the use of automatic differentiation. I will briefly review VI and sparse GPs and present how I contributed to the effort to scale VI for two classes of GP models: multi-GP models and time series models. I will illustrate these contributions with examples from neuroscience.

In the real world, a robot might face any unanticipated situation, such as damaged motors, unseen terrain conditions, faults in the sensors, and so on. Instead of aborting its mission when such situations occur, a robot could learn to adapt to those situations using reinforcement learning and continue its mission. However, the current reinforcement learning algorithms require prohibitively long interaction time (from several hours to days) to allow a complex physical robot to learn a new skill. In this talk, we present how we can accelerate the learning process on a real physical robot by leveraging prior knowledge derived from a simulator of the robot. More precisely, we use the model-based robot learning together with the priors from a simulator to allow relatively complex robots, such as a hexapod, to adapt to broken legs and fault in the sensors within a minute of interaction and accomplish the task.

This talk will describe a framework for constructing simulation-based kernels for ultra data-efficient Bayesian optimization (BO). We consider challenging settings that allow only 10-20 hardware trials, with access to only mid/low-fidelity simulators. First, I will describe how we can construct an informed kernel by embedding the space of simulated trajectories into a lower-dimensional space of latent paths. Our sequential variational autoencoder handles large-scale learning from ample simulated data; its modular design ensures quick adaptation to close the sim-to-real gap. The approach does not require domain-specific knowledge. Hence, we are able to demonstrate on hardware that the same architecture works for different areas of robotics: locomotion and manipulation. For domains with severe sim-to-real mismatch, I will describe our variant of BO which ensures that discrepancies between simulation and reality do not hinder online adaption. Using task-oriented grasping as an example, I will demonstrate how this approach helps quick recovery in case of corrupted/degraded simulation. My longer-term research vision is to build priors from simulation without requiring a specific simulation scenario. So, I will conclude by providing the motivation for this direction, and will describe our initial work on ’timescale fidelity’ priors. Such priors could help transfer-aware models and simulators to automatically adjust their timestep/frequency or planning horizon.