Learning-Based Calibration, Estimation and Stage Control for Electron Microscopy
Speaker
Jilles van Hulst
About this event
How much can you learn from a handful of noisy images? For electron microscopes, the answer decides how sharp their images get and how steadily they can hold the object under study. An electron microscope can photograph individual atoms, an ability indispensable for new medicines, better batteries, and the chips in every phone. In the COVID-19 pandemic, electron microscopes revealed the structure of the spike protein within weeks and accelerated vaccine development. Yet these instruments cannot focus themselves. Keeping them sharp is the daily work of scarce, highly trained operators. The PhD research of Jilles van Hulst, at the Eindhoven University of Technology with microscope manufacturer Thermo Fisher Scientific, teaches the machines to do this themselves. Atomic resolution No microscope can show details smaller than the wavelength it uses, and visible light is thousands of times wider than an atom. Electrons offer a way out: a fast electron behaves as a quantum wave far smaller than an atom. If a light wave were stretched to the length of a football field, the electron wave would span just a few sheets of paper. Electrons are also electrically charged, so magnetic fields can act as lenses for them. However, a theorem from 1936 proves that these lenses always distort the image. Correctors have existed since 1998, but they add dozens of settings that drift and need constant retuning. Learning from a handful of images Retuning these settings is hard, and anyone who has focused an old camera by hand knows why. Looking at a single blurry photo, you cannot tell which way to turn the focus ring. Only once you turn it do you see where you were, and perfect focus is precisely the point where turning either way makes it worse. An electron microscope poses this puzzle not for one focus ring but for dozens of settings at once, and every image costs time and can damage the specimen. Van Hulst therefore trained artificial intelligence (AI) to get the most out of each image. The AI learns from a simulator, which can generate unlimited practice images, but a simulation is never exactly the real machine. So the full method also learns that difference during operation, from the few real images it collects. The symmetry of the blur around perfect focus is what pins that point down uniquely. The result is a microscope that calibrates itself in about twenty seconds, twice as accurately as the best automated methods. Standing still at the scale of atoms Seeing atoms also demands stillness. For a three-dimensional image, the specimen is tilted step by step while dozens of pictures are taken, and the point under study must stay in view. The stage carrying the specimen can move across millimeters, yet must hold the imaged point to within nanometers. The actuators that move the stage are imperfect, and no sensor measures where the specimen actually is. Van Hulst used the images that the microscope already records as the missing sensor. They reveal exactly how far the specimen has shifted, and a learning controller removes errors it has seen before. On an operational microscope this reached nanometer accuracy, up to ten times better than the existing approach. More knowledge from less data The common thread is a lesson from statistics: what you can learn from data depends on what you already know. Van Hulst therefore developed general mathematical methods that build known structure, such as physical laws, symmetries, and models of motion, into the learning itself, so that fewer measurements suffice. Together, these results bring the self-driving microscope closer. It tunes itself, holds its specimen steady, and collects data on its own, so that researchers can focus on the discoveries.
Host
