Parmida Davarmanesh, a senior mathematics major and the University of Michigan, utilized deep learning methods, specifically a Generative Adversarial Network, to find the point spread function of 3D fluorescent microscopy images.
Recent advancements in multiplexed in situ molecular profiling techniques such as MERFISH and STARmap have enabled large-scale molecular assessment of thousands of cells at single-molecule resolution. Accurate quantification of molecular states in deep 3D tissue microscopy hinges on our ability to localize the individual molecules, which is challenging due to the spreading of the fluorescence signal via an unknown point spread function (PSF). The Broad is a unique place where some of the most talented and passionate scientists from around the globe are collaborating to solve some of the most important and interesting biomedical problems. Having the opportunity to interact with and learn from the amazing people behind the cutting-edge research that happens here, and even better, being a part of that research and contributing to it, was a dream come true.Here, we propose a novel framework for non-parametric reconstruction of the 3D PSF directly from the images using Generative Adversarial Networks (GANs). We explicitly model the probability density of point sources in 3D space, PSF, and Poisson-Gaussian microscopy noise in the data generating process. The critic scores the local and global consistency of images, both within 2D slices and across the stack. We further exploit the sparsity of point sources in 3D space to constrain the solution space using appropriate regularization terms. The GAN setup allows us to learn the PSF while side-stepping the problem of image refocusing and source localization. Accurate reconstruction of the 3D PSF of the optical setup enables image refocusing and deblurring as a subsequent step, and is an effective strategy for increasing the sensitivity and accuracy of image-based molecular assays.
Project: Blind Three-Dimensional Point Spread Function Estimation from Fluorescent Microscopy Image Stacks using Generative Adversarial Networks
Mentor: Dr. Mehrtash Babadi, Data Science Platform, CellBender Group Leader