3D maps of the human genome have revealed a hierarchical organization of DNA within the nucleus. This organization, including chromatin loops, topologically associated domains (TADs) and active and inactive compartments, has a strong association with transcription. In cancer, the genome is known to undergo profound epigenetic alterations, but the relationship between these changes and 3D organization has not been characterized. We will describe a maximum entropy approach to construct an optimal ensemble of chromosome structures that reproduce a Hi-C contact probability map. This method is highly efficient and can be applied to the whole genome for modeling intra- and inter-chromosomal interactions at a range of scales. It can be generalized to incorporate other experimental data such as imaging. Using these methods we show how to generate nuclear organization maps. We show an example where we have investigated tumor-specific aberrations to 3D genome structure showing widespread disruption of heterochromatin organization in colon cancer.