Our lab, the Broad’s Imaging Platform, aims to make perturbations in cell morphology as computable as other large-scale functional genomics data. We began by creating model-based segmentation algorithms to identify regions of interest in images (usually, individual cells or compartments within them) and produced software that has become the world standard for image analysis from high-throughput microscopy experiments (CellProfiler, cited in 3000+ scientific papers). We have taken on a new challenge – using cell images to identify signatures of genes and chemicals, with the ultimate goal of finding the cause and potential cures of diseases. High-throughput microscopy enables imaging several thousand cells per chemical or genetic perturbation, and identifying multiple organelles using fluorescent markers yields hundreds of image features per cell. We use this rich information to construct perturbation signatures or “profiles”. Our goals in these profiling experiments include identifying drug targets and mechanisms of action, determining the functional impact of disease-related alleles, creating performance-diverse chemical libraries, categorizing mechanisms of drug toxicity, and uncovering diagnostic markers for psychiatric disease.The technical challenges we encounter include dealing with cellular subpopulation heterogeneity, interpreting and visualizing statistical models, learning better representations of the data, and integrating imaging information with other data modalities.